system

Abstract

We provide the system. [Solution] A terminal means for inputting information, A server means for receiving input information and calculating an initial reliability score, A server means for comparing received information with past data and performing a step-by-step detailed evaluation, A server means for determining the confidence level based on the confidence score and generating a report, A terminal means for sending the generated report to a terminal and displaying it to the user, A terminal device for receiving user feedback and sending it to a server, A server for analyzing feedback and updating machine learning models, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, it is important to provide highly reliable information quickly and accurately. However, there is a lot of low-reliability information on the Internet, and the accuracy of information is particularly emphasized in business and public use. Therefore, a system for evaluating and appropriately classifying the reliability of information is required. In the current system, there is a problem that the reliability of information may not be appropriately evaluated, and users have to rely on information sources that can be trusted.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides the following means: a system including terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, server means for comparing the received information with past data and performing a step-by-step detailed evaluation, server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to the terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, and server means for analyzing the feedback and updating the machine learning model. By providing this system, it becomes possible to evaluate and classify the reliability of information with high accuracy. As a result, users can easily obtain reliable information and use it for business and public purposes.

[0006] "Information input terminal means" refers to a collection of devices and software for which a user inputs information and transmits that information to a server.

[0007] A "server system" is a collection of computer systems and software for receiving, processing, and analyzing information.

[0008] The "initial reliability score" is an evaluation score that initially quantifies the reliability of the input information based on the information source and metadata.

[0009] "Comparing with past data" means comparing newly entered information with previously stored data to verify the degree of similarity and any differences.

[0010] "Step-by-step detailed evaluation" is a method for evaluating the reliability of information in a step-by-step, detailed manner, and may include machine learning models or expert reviews.

[0011] A "reliability score" is a comprehensive numerical score obtained by evaluating the reliability of information.

[0012] "Confidence level" refers to a level that categorizes the reliability of information based on a reliability score.

[0013] A "report" is a document format output that provides users with evaluation results and classification levels.

[0014] "Feedback" refers to the act or result of a user reporting to the server the usefulness or problems of the information provided.

[0015] A "machine learning model" is an algorithm and system that learns on data and performs evaluations and predictions. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information. The system of the present invention is built around the interaction between a terminal, a server, and a user.

[0038] Information gathering

[0039] The user accesses the device and enters information. Specifically, the user enters text information such as market analysis reports or news articles into an input form. Based on this, the device temporarily stores the entered information and sends it to the server.

[0040] Initial assessment of information

[0041] The server analyzes the information received from the terminal. The server extracts metadata from the information source (such as the source and creation date) and calculates an initial reliability score using an initial assessment algorithm. For example, if the information comes from a government database, the initial reliability score will be set higher.

[0042] Graded evaluation of reliability

[0043] The server then compares the input information with historical databases, evaluating its accuracy, consistency, and historical reliability scores. For a more detailed, step-by-step evaluation, machine learning models and expert reviews are used to determine the accuracy and any bias in the information.

[0044] Level Classification

[0045] The server calculates a reliability score based on the results of a step-by-step evaluation. This reliability score is used to classify the information into reliability levels. For example, it might be divided into A-level (very reliable), B-level (quite reliable), C-level (average reliability), etc. The classification results are then formatted into a report.

[0046] User Feedback

[0047] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[0048] Information provision

[0049] Users can utilize information that has been evaluated for reliability in business and public settings. A specific example would be a user using this information as presentation material in an internal company meeting.

[0050] Feedback collection and updating machine learning models

[0051] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The server analyzes the received feedback and updates the machine learning model based on the results. This improves the accuracy of information reliability assessments in subsequent instances.

[0052] In this way, the system of the present invention evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information.

[0053] The following describes the processing flow.

[0054] Step 1:

[0055] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form.

[0056] Step 2:

[0057] The terminal temporarily stores the entered information and sends it to the server using a secure communication protocol.

[0058] Step 3:

[0059] The server saves the information received from the terminal to the database.

[0060] Step 4:

[0061] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[0062] Step 5:

[0063] The server uses an initial assessment algorithm to calculate an initial reliability score based on metadata. For example, if the information source is a highly reliable government database, the initial reliability score will be set higher.

[0064] Step 6:

[0065] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[0066] Step 7:

[0067] The server conducts a step-by-step detailed assessment. In this step, machine learning models and expert reviews are used to evaluate the accuracy and absence of bias in the information.

[0068] Step 8:

[0069] The server aggregates the evaluation results in stages and calculates an overall reliability score.

[0070] Step 9:

[0071] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[0072] Step 10:

[0073] The server formats the categorized information into a report format and sends it to the terminal.

[0074] Step 11:

[0075] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[0076] Step 12:

[0077] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[0078] Step 13:

[0079] Users provide feedback on the information they receive via their device. Specifically, they write comments regarding the usefulness and accuracy of the information.

[0080] Step 14:

[0081] The device sends feedback to the server.

[0082] Step 15:

[0083] The server analyzes the received feedback and saves the results.

[0084] Step 16:

[0085] The server incorporates the feedback data into the machine learning model and updates it. This improves the accuracy of reliability evaluations in subsequent tests.

[0086] The above outlines the specific processing steps for a system that evaluates and classifies the reliability of information. By using this system, users can easily obtain reliable information and use it with confidence in business and public settings.

[0087] (Example 1)

[0088] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0089] Conventional information evaluation systems have challenges such as the susceptibility to subjectivity in assessing the reliability of information, and the difficulty in evaluating consistency and accuracy. In particular, there is a need to provide users with rapid and accurate feedback on evaluation results, but this is not being adequately achieved. Furthermore, improving the accuracy of evaluations after receiving feedback is also a practical problem.

[0090] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0091] In this invention, the server includes means for receiving information and calculating an initial reliability score, means for comparing the received information with past data and performing a step-by-step detailed evaluation, and means for determining a reliability level based on the reliability score and generating a report. This enables a multifaceted evaluation of the reliability of information, allowing users to quickly and accurately obtain reliable information.

[0092] "Device means for inputting information" refers to devices or terminals used by users to input information.

[0093] "Processor means for calculating initial reliability score" refers to a central processing unit (CPU) or microprocessor used to evaluate the initial reliability of received information and calculate a score.

[0094] "Processor means for performing step-by-step detailed evaluation" refers to a central processing unit (CPU) or microprocessor that compares received information with a past database and performs a detailed evaluation based on multiple evaluation criteria.

[0095] "Processor means for determining reliability levels based on reliability scores and generating reports" refers to a central processing unit (CPU) or microprocessor that calculates a final reliability score based on evaluation results, classifies information into reliability levels, and generates reports.

[0096] "Device means for sending generated reports to a device and displaying them to the user" refers to a device or terminal that sends reports generated by a server to a terminal so that the user can view and display them.

[0097] "Device means for receiving user feedback and sending it to a processor" refers to a device or terminal that receives feedback information provided by the user and sends it to a server.

[0098] "Processor means for analyzing feedback and updating machine learning models" refers to a central processing unit (CPU) or microprocessor that analyzes feedback information received from users and updates machine learning models based on the results obtained.

[0099] An "algorithm for evaluating the degree of agreement" is a computational procedure for evaluating how well received information matches past data or existing evaluation scores.

[0100] A "machine learning model" is a trained artificial intelligence model that improves the accuracy of information reliability evaluation based on feedback data.

[0101] The embodiments for carrying out the present invention will be described in detail below. The invention is a system for evaluating and classifying the reliability of information. This system is built around the interaction between terminals, servers, and users.

[0102] Information gathering

[0103] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. The terminal temporarily stores this information and sends it to the server. In this process, the input data is converted to JSON format and an HTTP POST request is used. For example, if a market analysis report is entered, the terminal sends it to the server.

[0104] Initial assessment of information

[0105] The server analyzes the information received from the terminal. First, it extracts metadata from the information source (such as the source and creation date), and then calculates an initial reliability score using an initial evaluation algorithm. In this process, for example, if the source is a government database, the initial reliability score is set higher. The server can use a central processing unit (CPU) or cloud-based analysis software.

[0106] Graded evaluation of reliability

[0107] The server then compares the received information with historical database data to assess its accuracy and consistency. Machine learning models (e.g., the BERT model) are used in this process. Evaluation may also be conducted through an expert review system. The server evaluates the agreement rate and consistency and collects the evaluation results.

[0108] Level Classification

[0109] The server calculates a final reliability score based on the results of a step-by-step evaluation and classifies the information into reliability levels. For example, if the agreement rate is 90% and the consistency is 85%, the reliability score will be 90 points, classifying it as A level (very reliable). The generated report is formatted in PDF format.

[0110] User Feedback

[0111] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. For example, the assessment results might be displayed on the user's screen as "Reliability: A level, 90% reliability." HTTP GET requests are used for communication between the server and the terminal.

[0112] Information provision

[0113] Users can utilize information that has been evaluated for reliability in business and public settings. For example, a user might use this information as part of a presentation at an internal company meeting. They can download the generated PDF report and incorporate it into their presentation.

[0114] Feedback collection and updating machine learning models

[0115] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server, which analyzes the feedback and updates the machine learning model. In this process, user-entered feedback is sent in JSON format and analyzed by the server. This improves the accuracy of information reliability evaluations in subsequent instances.

[0116] Examples of specific cases and prompt statements

[0117] For example, a user of a market analysis report:

[0118] Market Analysis Report:

[0119] Company A's sales increased by 20% compared to last year.

[0120] New product B's market share reached 15% in 2022.

[0121] If you input this information, the following prompt may be entered into the generating AI model:

[0122] Please evaluate the reliability of the following market analysis report.

[0123] Company A's sales increased by 20% compared to last year.

[0124] New product B's market share reached 15% in 2022.

[0125] Please classify the information from A (very reliable) to C (average reliability), taking into account the reliability of the source, the consistency of the information, and its accuracy.

[0126] By inputting this prompt into the generating AI model, its reliability is evaluated, and the results are fed back to the user.

[0127] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0128] Step 1: Enter information

[0129] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. When the user presses the "Submit" button, the terminal converts the input data into JSON format and sends an HTTP POST request to the server. The input is text information provided by the user, and the output is data converted into JSON format.

[0130] Specific actions:

[0131] When a user enters a market analysis report and clicks the "Submit" button, the terminal internally serializes the entered text information into JSON format and sends it to the server via an HTTP POST request.

[0132] Step 2: Initial information reception and analysis

[0133] The server parses the JSON data received from the terminal and extracts metadata of the information source (source, creation date, etc.). The server then uses an initial evaluation algorithm to calculate an initial confidence score. The input is the JSON data sent from the terminal, and the output is the analysis results, including the initial confidence score.

[0134] Specific actions:

[0135] The server receives an HTTP POST request, deserializes the JSON data to extract text information, then extracts metadata (e.g., source and creation date) and calculates an initial confidence score.

[0136] Step 3: Comparison with historical data

[0137] The server compares the information against historical databases to assess its accuracy and consistency. This is done using machine learning models (e.g., the BERT model). Inputs are metadata and text information, and outputs are detailed evaluation results (e.g., agreement rate, consistency score).

[0138] Specific actions:

[0139] The server uses the BERT model to compare the received text information with historical data in the database. As a result, it calculates the match rate and consistency score.

[0140] Step 4: Calculation and leveling of reliability scores

[0141] The server calculates a final reliability score based on the detailed evaluation results and classifies the information into reliability levels. It then generates a report based on the evaluation results. The input is the detailed evaluation results, and the output is the final reliability score, reliability level, and the generated report.

[0142] Specific actions:

[0143] The server integrates the detailed evaluation results and calculates a final reliability score, for example, "90 points." Based on this, the information is classified as A-level (highly reliable), and a report is generated in PDF format.

[0144] Step 5: Submit and view the report

[0145] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. The input is the generated report, and the output is the reliability assessment results displayed on the user's screen.

[0146] Specific actions:

[0147] The server sends a PDF report to the terminal via an HTTP GET request, and the terminal displays it on the user's screen. For example, a result such as "Confidence level: A, 90% reliability" might be displayed.

[0148] Step 6: Gathering Feedback

[0149] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The input is the feedback provided by the user, and the output is the feedback data sent to the server.

[0150] Specific actions:

[0151] When a user enters "Very helpful, highly accurate" into the feedback form and clicks the submit button, the device converts the feedback data into JSON format and sends it to the server.

[0152] Step 7: Analyze feedback and update the machine learning model

[0153] The server analyzes the received feedback and updates the machine learning model. The input is the feedback data sent from the terminal, and the output is the updated machine learning model.

[0154] Specific actions:

[0155] The server analyzes the feedback data and saves the results to a database. This data is then used to retrain the machine learning model during the next model update, improving the accuracy of the evaluation.

[0156] (Application Example 1)

[0157] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0158] Currently, many users access a wide variety of information through the internet, but there is a lack of means to evaluate its reliability. Especially in today's world, where fake news and misinformation spread easily, there is a need for technology to quickly and accurately identify reliable information. Conventional systems are limited to initial and incremental evaluations of information, making real-time evaluation difficult as users browse web pages and digital content. Therefore, a system is needed to evaluate the reliability of information in real time and provide this information to users.

[0159] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0160] In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, and server means for comparing the received information with past data and performing a step-by-step detailed evaluation. This makes it possible to easily and quickly evaluate the reliability of information. Furthermore, the server includes server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to a terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, client means for evaluating the reliability of web pages and digital content in real time and displaying it on a browser application, metadata extraction means for extracting metadata of web content accessed by the user and calculating an initial reliability score, detailed evaluation means for evaluating the reliability score in detail using a machine learning algorithm based on the input information, and client means for providing feedback on the reliability results of the web content to the user and collecting feedback on the reliability of the information. This makes it possible to evaluate the reliability of information in real time when a user views web pages and digital content and to provide reliable information quickly and accurately.

[0161] "Terminal means" refers to a device that includes hardware and software for a user to input information or to display information received from a server.

[0162] A "server system" is a system that analyzes information, calculates reliability scores, performs step-by-step detailed evaluations, and determines the final reliability level.

[0163] A "metadata extraction method" is a system for analyzing and extracting metadata such as the source and creation date of input information.

[0164] A "machine learning model" is an algorithmic model that learns patterns based on a vast dataset and evaluates the reliability of information.

[0165] A "browser application" is software used by users to access web pages and digital content, and it includes a function for evaluating the reliability of information.

[0166] A "client device" is a mechanism that acts as a terminal device, sending user input information to a server and displaying feedback from the server.

[0167] The "detailed evaluation method" is a system that, after the initial evaluation, compares the information entered with historical data in detail and uses a machine learning model to evaluate reliability with high accuracy.

[0168] A "report generation method" is a system that determines the reliability level based on a reliability score, formats the result, and generates a report.

[0169] A "feedback collection mechanism" is a system for receiving feedback from users, sending it to a server, and analyzing it.

[0170] This invention provides a system that evaluates the reliability of information in real time and provides it to the user. This system evaluates the reliability of web pages and digital content viewed by the user and presents the user with a reliability score and evaluation report.

[0171] Specifically, this system will be implemented using the following hardware and software.

[0172] Hardware:

[0173] Smartphone (iOS or Android®)

[0174] Server (a cloud server equipped with a high-performance processor and a large-capacity database)

[0175] software:

[0176] Client applications: iOS (Swift), Android (Kotlin)

[0177] Server-side: Python, Flask / Django (Web frameworks)

[0178] Databases: PostgreSQL, Elasticsearch (registered trademark)

[0179] Machine learning models: Scikit-learn, TENSORFLOW®, PyTorch

[0180] Program processing

[0181] The terminal device (smartphone) inputs information about the web page accessed by the user and retrieves the URL and text content. The retrieved information is then sent to the server.

[0182] The server analyzes the received information and extracts metadata. In particular, it calculates an initial reliability score based on important metadata such as the source and creation date. At this stage, information from highly reliable sources such as government agencies and major news organizations receives a high score.

[0183] Next, the server compares the information with past database data and performs a step-by-step detailed evaluation. Specifically, it uses a machine learning model to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that was previously evaluated as highly reliable, it will receive a high score.

[0184] Based on these evaluation results, the server calculates a final reliability score and classifies the information into reliability levels (e.g., A level, B level, C level). The reliability level and evaluation results are generated in a report format and sent to the client application.

[0185] The terminal device (a smartphone browser application) displays the provided report to the user. The user can view the reliability assessment results in real time and make decisions based on the reliability of the information.

[0186] Furthermore, the terminal device sends user feedback to the server. The server device analyzes this feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations improves.

[0187] As a concrete example, the user can use the following prompt statement.

[0188] Please analyze the content of the provided URL and calculate a reliability score.

[0189] URL: "https: / / example.com / news / some-article"

[0190] In this way, the system of the present invention can evaluate the reliability of information from multiple perspectives and provide users with highly reliable information quickly and accurately. This enables users to make decisions based on highly reliable information.

[0191] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0192] Step 1:

[0193] The device (smartphone) obtains information about the web pages accessed by the user. When the user enters a URL into the browser application, the device accesses that URL and collects the text content and metadata (source, creation date, etc.) of the web page. The collected information is sent to the server.

[0194] Input: URL entered by the user

[0195] Output: Text content and metadata of the webpage

[0196] Step 2:

[0197] The server analyzes the information from the received web page and extracts metadata. In particular, it calculates an initial reliability score based on metadata such as the source and creation date. For example, information from government sources or highly reliable sources is assigned a high score.

[0198] Input: Text content and metadata of the webpage sent from the device.

[0199] Output: Initial reliability score

[0200] Step 3:

[0201] The server compares the input information with past database data and performs a step-by-step detailed evaluation. A machine learning model is used to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that has been previously evaluated as highly reliable, it will receive a high score.

[0202] Input: Initial reliability score and historical database

[0203] Output: Detailed evaluation score

[0204] Step 4:

[0205] The server calculates a final reliability score based on the detailed evaluation score. It then classifies the results into reliability levels (e.g., A level, B level, C level) and generates a report.

[0206] Input: Detailed evaluation score

[0207] Output: Report including reliability score and reliability level

[0208] Step 5:

[0209] The report generated by the server is sent to the terminal. The terminal displays the provided report to the user. The user can view the reliability evaluation results in real time and make decisions based on the reliability of the information.

[0210] Input: Report sent from server

[0211] Output: Display of reliability evaluation results to the user.

[0212] Step 6:

[0213] The terminal device collects user feedback and sends it to the server.

[0214] Input: User feedback

[0215] Output: Sending feedback data to the server

[0216] Step 7:

[0217] The server analyzes the received feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations is improved.

[0218] Input: User feedback data

[0219] Output: Updated machine learning model

[0220] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0221] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information, and by combining it with an emotion engine that recognizes the user's emotions, it improves the accuracy and reliability of information evaluation.

[0222] Information gathering

[0223] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions.

[0224] Initial assessment of information

[0225] The device temporarily stores the entered information and emotional data, and then sends it to the server using a secure communication protocol.

[0226] The server stores the information and sentiment data received from the terminal in a database. The server extracts metadata from the received information (source, creation date, etc.) and calculates an initial confidence score using an initial evaluation algorithm. Here, sentiment data obtained from the sentiment engine is also reflected in the calculation of the initial confidence score. For example, if the user shows very positive sentiment, the initial confidence score is adjusted.

[0227] Graded evaluation of reliability

[0228] The server compares the input information with historical databases. It evaluates the accuracy, consistency, and historical reliability scores of the information. Machine learning models and expert reviews are used for a step-by-step, detailed evaluation. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes.

[0229] Level Classification

[0230] The server calculates an overall reliability score based on its step-by-step evaluation. Sentimental data is also included in this score calculation. Based on the reliability score, the information is categorized into levels according to its reliability. For example, it may be classified into A level (very reliable), B level (quite reliable), etc. The classification results are then formatted into a report.

[0231] User Feedback

[0232] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[0233] Information provision

[0234] Users use information that has been evaluated for reliability in business and public settings. For example, this information might be used as presentation material in an internal company meeting.

[0235] Feedback collection and updating machine learning models

[0236] Users input feedback on the provided information via their device. Here too, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the emotions (positive or negative) the user feels when giving feedback.

[0237] The device sends emotional data along with feedback to the server. The server analyzes the received feedback and saves the results. The emotional data is also included in the analysis and is used to interpret the feedback. The server incorporates the feedback data into a machine learning model and updates the model. This improves the accuracy of reliability evaluations in subsequent attempts.

[0238] In this way, the system of the present invention, by combining an emotion engine, evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information. Furthermore, by utilizing emotion data, it achieves further improvements in the accuracy of the evaluation process and enhances the user experience.

[0239] The following describes the processing flow.

[0240] Step 1:

[0241] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine is activated and begins to recognize the user's emotions based on their facial expressions and tone of voice during input.

[0242] Step 2:

[0243] The terminal temporarily stores the emotional data recognized by the emotion engine along with the input information, and then sends it to the server using a secure communication protocol.

[0244] Step 3:

[0245] The server stores the information and sentiment data received from the terminal in a database.

[0246] Step 4:

[0247] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[0248] Step 5:

[0249] The server uses an initial assessment algorithm to calculate an initial confidence score based on metadata and sentiment data. For example, if a user expresses positive sentiment, the initial confidence score is adjusted accordingly.

[0250] Step 6:

[0251] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[0252] Step 7:

[0253] The server conducts a step-by-step detailed evaluation. In this step, machine learning models and expert reviews are used to assess the accuracy and absence of bias in the information. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes and used as complementary material for the assessment.

[0254] Step 8:

[0255] The server aggregates the evaluation results in stages and calculates an overall reliability score. Sentimental data is also reflected in this score calculation.

[0256] Step 9:

[0257] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[0258] Step 10:

[0259] The server formats the categorized information into a report format and sends it to the terminal.

[0260] Step 11:

[0261] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[0262] Step 12:

[0263] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[0264] Step 13:

[0265] When a user provides feedback on the information provided via their device, the emotion engine restarts to recognize and record the user's emotions at the time of feedback.

[0266] Step 14:

[0267] The device sends emotional data to the server along with feedback.

[0268] Step 15:

[0269] The server analyzes the feedback it receives and saves the results. Sentimental data is also included in the analysis and can be used to interpret the feedback.

[0270] Step 16:

[0271] The server incorporates feedback and sentiment data into the machine learning model, updating it. This improves the accuracy of reliability evaluations in subsequent attempts.

[0272] The above outlines the specific processing steps of a system that evaluates and classifies the reliability of information by combining it with an emotion engine. By using this system, users can obtain reliable information with greater accuracy and use it with confidence in business and public settings.

[0273] (Example 2)

[0274] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0275] In recent years, numerous systems have emerged to evaluate the reliability of information. However, these systems often lack consideration for the emotions users feel when providing information, resulting in limitations in their accuracy. Furthermore, there are challenges in the speed at which evaluation results are delivered to users. Against this backdrop, there is a growing demand for information reliability evaluation systems that incorporate user sentiment data.

[0276] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, server means for comparing the received information with past data and performing a stepwise detailed evaluation, server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to the terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, terminal means for recognizing the user's emotional data in real time and recording it, and server means for incorporating the recorded emotional data into the reliability evaluation process. This enables highly accurate reliability evaluation that reflects the user's emotions and rapid information provision.

[0277] The "terminal means" is a device including hardware and software for a user to input information and receive the processing results thereof.

[0278] The "server means" is a central computer system designed to receive, store, process information, and generate and transmit evaluation results and reports.

[0279] The "initial reliability score" is a numerical index calculated to evaluate the reliability of received information at an initial stage.

[0280] The "past data" is a set of data including information previously collected and evaluated that is collated and used as a reference in the evaluation process.

[0281] The "step - by - step detailed evaluation" is a process of evaluating the accuracy, consistency, and reliability of information in multiple steps.

[0282] The "reliability score" is a numerical index indicating the result of quantitatively evaluating the overall reliability of received information.

[0283] The "reliability level" is a category for classifying the reliability of information based on the reliability score.

[0284] The "report" is a document - formatted output including the results of the evaluation process, the reliability score, and the reliability level.

[0285] The "feedback" is input data representing opinions and feelings about the information provided by the user.

[0286] The "machine learning model" is an algorithm and structure for learning from data and making future predictions and classifications.

[0287] The "emotion data" is data that recognizes and records the emotional state of the user in real - time during input and feedback.

[0288] This invention is a system for evaluating and categorizing the reliability of information, and by combining it with an emotion engine that recognizes user emotions, it improves the accuracy and reliability of information evaluation. Specific embodiments of this system are described below.

[0289] Information gathering

[0290] Users access a terminal and input text information, such as market analysis reports and news articles, into a dedicated information input form. Here, an emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions. Emotional data is a crucial element in reliability evaluation.

[0291] Initial assessment of information

[0292] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (e.g., HTTPS). The server stores the received information and sentiment data in a database. Here, the database may be an RDBMS such as MySQL® or PostgreSQL. Furthermore, the server extracts metadata from the received information (source, creation date and time, etc.) and calculates an initial confidence score using an initial evaluation algorithm. Sentiment data is also taken into consideration here.

[0293] Graded evaluation of reliability

[0294] The server compares the input information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability scores of the information. For a stepwise, detailed evaluation, machine learning models and expert reviews are used. For example, machine learning libraries such as scikit-learn and TensorFlow are used to build models and further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation.

[0295] Level Classification

[0296] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This reliability score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Based on this, the server classifies the information and formats it into a report format (e.g., PDF or HTML). The Python ReportLab library is used to generate the report.

[0297] User Feedback

[0298] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability."

[0299] Information provision

[0300] Users use information that has been evaluated for reliability in business and public settings. For example, they might use this information as presentation material in an internal company meeting.

[0301] Feedback collection and updating machine learning models

[0302] Users input feedback on the provided information through their device. Here, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. The device sends the emotion data along with the feedback to the server. The server analyzes the received feedback and saves the results to a database. The emotion data is also included in the analysis and is used to interpret the content of the feedback. Finally, the server updates its machine learning model using the feedback data to improve the accuracy of reliability evaluations in the future.

[0303] Specific example

[0304] 1. When a user enters a "New Product Market Analysis Report" into an input form, the emotion engine detects the user's positive emotion.

[0305] 2. The terminal sends this information and emotion data to the server.

[0306] 3. The server analyzes the received data and calculates an initial reliability score.

[0307] 4. The server conducts a detailed evaluation and calculates a reliability score.

[0308] 5. The server classifies the information based on the reliability score (e.g., Level A) and generates a report.

[0309] 6. The server sends the generated report to the terminal, and the user checks the evaluation result.

[0310] 7. The user utilizes the information in business and provides feedback on the result.

[0311] 8. The server receives the feedback and updates the machine learning model.

[0312] Example of prompt text

[0313] "Please evaluate the reliability of the New Product Market Analysis Report. The emotion during data input was very positive."

[0314] With this system, users can quickly obtain highly reliable information and utilize it efficiently in business and public settings. By leveraging emotion data, even more accurate evaluations can be achieved.

[0315] The flow of specific processing in Example 2 will be described using FIG. 13.

[0316] Step 1:

[0317] The user accesses the terminal and enters text information, such as market analysis reports and news articles, into a dedicated information input form. Along with the text information, the terminal uses an emotion engine to recognize the user's emotions at the time of input in real time and record them as emotion data. Input data: market analysis reports, news articles, and emotion data. Output data: recorded text information and emotion data.

[0318] Step 2:

[0319] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (HTTPS). Data processing involves formatting the input information and sentiment data and converting them into transmittable data packets. Input data: text information and sentiment data. Output data: data packets sent to the server.

[0320] Step 3:

[0321] The server stores the information and sentiment data received from the terminal in a database. An RDBMS such as MySQL or PostgreSQL is used as the database. Input data: Received text information and sentiment data. Output data: Information and sentiment data stored in the database.

[0322] Step 4:

[0323] The server extracts metadata (source, creation date, etc.) from the received information. This process uses libraries such as Python's BeautifulSoup. As a data processing step, metadata is parsed from the text information. Input data: Stored text information. Output data: Extracted metadata.

[0324] Step 5:

[0325] The server calculates an initial confidence score using an initial assessment algorithm. Sentiment data is also considered. The score calculation uses basic statistical processing and rule-based algorithms. Input data: Stored text information, sentiment data, metadata. Output data: Initial confidence score.

[0326] Step 6:

[0327] The server compares the entered information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability score of the information. Input data: Initial reliability score, stored text information. Output data: Matching results.

[0328] Step 7:

[0329] The server performs a detailed evaluation using machine learning models and expert reviews. For example, it builds evaluation models using machine learning libraries such as scikit-learn and TensorFlow to further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation. Input data: Matching results, sentiment data. Output data: Detailed evaluation results.

[0330] Step 8:

[0331] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Input data: Detailed evaluation results. Output data: Overall reliability score.

[0332] Step 9:

[0333] The server classifies information based on reliability scores and formats it into a report format (e.g., PDF or HTML). The report is generated using a Python library such as ReportLab. Input data: Overall reliability score. Output data: Generated report.

[0334] Step 10:

[0335] The server sends the generated report to the terminal. HTTPS is used again for this communication. Input data: Generated report. Output data: Sent report.

[0336] Step 11:

[0337] The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability." Input data: The submitted report. Output data: The displayed evaluation results.

[0338] Step 12:

[0339] Users use information that has been evaluated for reliability in business and public settings. For example, this information is used as presentation material in an internal company meeting. Input data: Information that has been evaluated for reliability. Output data: Information that was used.

[0340] Step 13:

[0341] The user inputs feedback on the provided information via a terminal. The emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. Input data: Feedback content, emotion data. Output data: Recorded feedback and emotion data.

[0342] Step 14:

[0343] The device sends emotional data along with feedback to the server. Input data: Recorded feedback and emotional data. Output data: Feedback and emotional data sent to the server.

[0344] Step 15:

[0345] The server analyzes the feedback it receives and stores the results in a database. Sentimental data is also included in the analysis and used to interpret the feedback. Input data: Feedback and sentimental data sent to the server. Output data: Analysis results.

[0346] Step 16:

[0347] The server updates the machine learning model using feedback data. This improves the accuracy of subsequent reliability evaluations. Input data: Analysis results. Output data: Updated machine learning model.

[0348] (Application Example 2)

[0349] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0350] Conventional advertising information reliability evaluation systems failed to fully utilize user sentiment data when assessing the accuracy and reliability of information. Therefore, improving the accuracy of sentiment-based evaluations was difficult, resulting in insufficient reliability evaluation results. Furthermore, the lack of mechanisms to effectively incorporate user feedback made it difficult to improve the quality of information provided.

[0351] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading advertising creatives, terminal means for collecting sentiment data, and server means for calculating a reliability score using the collected sentiment data. This makes it possible to collect and analyze user sentiment data in real time and reflect it in the reliability evaluation. Furthermore, by displaying the reliability evaluation results immediately, it becomes possible to quickly adjust the marketing strategy. In addition, by updating the machine learning model using user feedback data, it becomes possible to continuously improve the accuracy and reliability of the evaluation.

[0352] "Terminal means for inputting information" refers to electronic devices used by users to input information, including, for example, smartphones and tablets.

[0353] "Server means for calculating initial reliability score" refers to a server and related software that evaluates the initial reliability and calculates a score based on the input information.

[0354] "Server means for conducting step-by-step detailed evaluations" refers to a server and related software that compares input information with past data and conducts detailed evaluations step by step.

[0355] "Server means for determining reliability levels based on reliability scores and generating reports" refers to a server and related software that categorizes the reliability of information based on calculated reliability scores and generates the results as a report.

[0356] "Terminal means for sending generated reports to a terminal and displaying them to the user" refers to electronic devices and related software for sending reports generated by the server to the user's terminal and displaying those reports on that terminal.

[0357] "Terminal means for receiving user feedback and sending it to the server" refers to electronic devices and related software for users to input feedback and send that information to the server.

[0358] "Server means for analyzing feedback and updating machine learning models" refers to a server and related software that analyzes feedback sent by users and updates machine learning models based on the results.

[0359] "Terminal means for uploading advertising creatives" refers to electronic devices and related software used by users to upload advertising creatives (images and videos).

[0360] "Terminal means for collecting emotional data" refers to electronic devices and related software used to collect user emotions in real time using video feeds or cameras.

[0361] "Server means for calculating reliability scores using collected sentiment data" refers to a server and related software for calculating reliability scores based on collected sentiment data.

[0362] "Terminal means for displaying reliability evaluation results" refers to electronic equipment and related software for displaying calculated reliability scores and evaluation results to the user.

[0363] The system that realizes this application example integrates various means for performing information reliability evaluation and sentiment feedback analysis. The specific system configuration and processing flow are described below.

[0364] The server provides a system that includes "terminal means for uploading advertising creatives," "terminal means for collecting sentiment data," "server means for calculating a reliability score using the collected sentiment data," and "terminal means for displaying the reliability evaluation results." This configuration allows for the collection and analysis of user sentiment data in real time and the evaluation of information reliability from multiple perspectives.

[0365] First, users upload advertising creatives (images and videos) from their devices to the server. These devices are common electronic devices such as smartphones and tablets. Next, video feeds and cameras are used to collect emotional data. For example, the camera captures the face of a user viewing the advertising creative, and their emotions are recognized and recorded in real time. OpenCV or specific emotion recognition engines are used for this process.

[0366] The collected sentiment data is sent to the server and used as part of the initial confidence score calculation. The server uses this data to evaluate the accuracy and consistency of the information and calculates an overall confidence score. This includes cross-referencing with existing metadata and historical evaluation data. Furthermore, the server updates the confidence score using the sentiment data as a corrective factor and generates the results in a report format.

[0367] The generated report is sent to the device and displayed to the user. The user can review it and adjust specific advertising strategies based on the reliability assessment results. The report includes specific evaluations, such as "Reliability: High" and "Reliability Score: 95%".

[0368] After the advertisement is actually delivered, user feedback is collected again. The sentiment engine collects sentiment data during the feedback process and sends it to the server along with the feedback. The server analyzes this data and updates the parameters of the machine learning model to improve the accuracy of future reliability evaluations.

[0369] A concrete example would be uploading a new ad video and conducting a test session where several consumers watch the ad. The application would collect the consumers' emotional responses in real time and calculate the ad's credibility score. Alternatively, a prompt message such as, "A new ad video has been uploaded. Please analyze the emotional responses of consumers watching this video and calculate its credibility score," could be used.

[0370] In this way, the system of the present invention can utilize emotional data from multiple perspectives and evaluate the reliability of information with high accuracy. It can also contribute to the rapid adjustment of marketing strategies and the improvement of the user experience.

[0371] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0372] Step 1:

[0373] Users upload advertising creatives (images and videos) from their devices to the server. The input data is the advertising creative file, and the output data is the file stored on the server. The device also collects the file's metadata (creation date and time, uploader information, etc.) and sends it to the server. This process allows the advertising creatives to be managed within the system.

[0374] Step 2:

[0375] The device activates a video feed or camera to collect user emotion data. The input data is real-time video footage, and the output data is recognized emotion information (e.g., positive, negative, neutral). This process uses image analysis and emotion recognition engines based on OpenCV. The video data is analyzed frame by frame, and emotion information for each frame is collected and sent to the server.

[0376] Step 3:

[0377] The server receives the collected sentiment data and calculates an initial confidence score. The input data consists of sentiment information and ad creative metadata, and the output data is the initial confidence score. The server analyzes the collected sentiment data and calculates a normalized score (e.g., in the range of 0 to 100). A sentiment data correction algorithm is used for this process.

[0378] Step 4:

[0379] The server compares the received information with historical data and performs a step-by-step detailed evaluation. Input data includes advertising creatives, initial confidence scores, and historical evaluation data, while output data is a detailed confidence score. The server evaluates the degree of information consistency and historical confidence scores to calculate an overall confidence score. Cross-referencing with existing databases is performed at this stage.

[0380] Step 5:

[0381] The server determines the confidence level based on the confidence score and generates a report. The input data is a detailed confidence score, and the output data is a report that includes the confidence level and evaluation results. The server categorizes the confidence level of the advertisements based on the confidence score and formats the evaluation results into a report format. This makes the evaluation results easier to understand visually.

[0382] Step 6:

[0383] The server sends the generated report to the terminal and displays it to the user. The input data is the report, and the output data is the evaluation result displayed on the terminal. The terminal displays the report received from the server, allowing the user to view the reliability evaluation results. This enables the user to adjust their advertising strategy based on the evaluation results.

[0384] Step 7:

[0385] Users input feedback after ad delivery via their device and send it to the server. Input data consists of user feedback and sentiment data, while output data is the feedback information stored on the server. The device receives the user feedback and sends it to the server for analysis.

[0386] Step 8:

[0387] The server analyzes the feedback and updates the machine learning model based on the results. The input data consists of user feedback and sentiment data, while the output data is the updated machine learning model. The server analyzes the specific feedback content and sentiment data and updates the parameters of the machine learning model. This update improves the accuracy of reliability assessments in subsequent uses.

[0388] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0389] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0390] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0391] [Second Embodiment]

[0392] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0393] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0394] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0395] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0396] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0398] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0399] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0400] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0401] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0402] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0403] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0404] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information. The system of the present invention is built around the interaction between a terminal, a server, and a user.

[0405] Information gathering

[0406] The user accesses the device and enters information. Specifically, the user enters text information such as market analysis reports or news articles into an input form. Based on this, the device temporarily stores the entered information and sends it to the server.

[0407] Initial assessment of information

[0408] The server analyzes the information received from the terminal. The server extracts metadata from the information source (such as the source and creation date) and calculates an initial reliability score using an initial assessment algorithm. For example, if the information comes from a government database, the initial reliability score will be set higher.

[0409] Graded evaluation of reliability

[0410] The server then compares the input information with historical databases, evaluating its accuracy, consistency, and historical reliability scores. For a more detailed, step-by-step evaluation, machine learning models and expert reviews are used to determine the accuracy and any bias in the information.

[0411] Level Classification

[0412] The server calculates a reliability score based on the results of a step-by-step evaluation. This reliability score is used to classify the information into reliability levels. For example, it might be divided into A-level (very reliable), B-level (quite reliable), C-level (average reliability), etc. The classification results are then formatted into a report.

[0413] User Feedback

[0414] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[0415] Information provision

[0416] Users can utilize information that has been evaluated for reliability in business and public settings. A specific example would be a user using this information as presentation material in an internal company meeting.

[0417] Feedback collection and updating machine learning models

[0418] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The server analyzes the received feedback and updates the machine learning model based on the results. This improves the accuracy of information reliability assessments in subsequent instances.

[0419] In this way, the system of the present invention evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information.

[0420] The following describes the processing flow.

[0421] Step 1:

[0422] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form.

[0423] Step 2:

[0424] The terminal temporarily stores the entered information and sends it to the server using a secure communication protocol.

[0425] Step 3:

[0426] The server saves the information received from the terminal to the database.

[0427] Step 4:

[0428] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[0429] Step 5:

[0430] The server uses an initial assessment algorithm to calculate an initial reliability score based on metadata. For example, if the information source is a highly reliable government database, the initial reliability score will be set higher.

[0431] Step 6:

[0432] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[0433] Step 7:

[0434] The server conducts a step-by-step detailed assessment. In this step, machine learning models and expert reviews are used to evaluate the accuracy and absence of bias in the information.

[0435] Step 8:

[0436] The server aggregates the evaluation results in stages and calculates an overall reliability score.

[0437] Step 9:

[0438] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[0439] Step 10:

[0440] The server formats the categorized information into a report format and sends it to the terminal.

[0441] Step 11:

[0442] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[0443] Step 12:

[0444] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[0445] Step 13:

[0446] Users provide feedback on the information they receive via their device. Specifically, they write comments regarding the usefulness and accuracy of the information.

[0447] Step 14:

[0448] The device sends feedback to the server.

[0449] Step 15:

[0450] The server analyzes the received feedback and saves the results.

[0451] Step 16:

[0452] The server incorporates the feedback data into the machine learning model and updates it. This improves the accuracy of reliability evaluations in subsequent tests.

[0453] The above outlines the specific processing steps for a system that evaluates and classifies the reliability of information. By using this system, users can easily obtain reliable information and use it with confidence in business and public settings.

[0454] (Example 1)

[0455] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0456] Conventional information evaluation systems have challenges such as the susceptibility to subjectivity in assessing the reliability of information, and the difficulty in evaluating consistency and accuracy. In particular, there is a need to provide users with rapid and accurate feedback on evaluation results, but this is not being adequately achieved. Furthermore, improving the accuracy of evaluations after receiving feedback is also a practical problem.

[0457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0458] In this invention, the server includes means for receiving information and calculating an initial reliability score, means for comparing the received information with past data and performing a step-by-step detailed evaluation, and means for determining a reliability level based on the reliability score and generating a report. This enables a multifaceted evaluation of the reliability of information, allowing users to quickly and accurately obtain reliable information.

[0459] "Device means for inputting information" refers to devices or terminals used by users to input information.

[0460] "Processor means for calculating initial reliability score" refers to a central processing unit (CPU) or microprocessor used to evaluate the initial reliability of received information and calculate a score.

[0461] "Processor means for performing step-by-step detailed evaluation" refers to a central processing unit (CPU) or microprocessor that compares received information with a past database and performs a detailed evaluation based on multiple evaluation criteria.

[0462] "Processor means for determining reliability levels based on reliability scores and generating reports" refers to a central processing unit (CPU) or microprocessor that calculates a final reliability score based on evaluation results, classifies information into reliability levels, and generates reports.

[0463] "Device means for sending generated reports to a device and displaying them to the user" refers to a device or terminal that sends reports generated by a server to a terminal so that the user can view and display them.

[0464] "Device means for receiving user feedback and sending it to a processor" refers to a device or terminal that receives feedback information provided by the user and sends it to a server.

[0465] "Processor means for analyzing feedback and updating machine learning models" refers to a central processing unit (CPU) or microprocessor that analyzes feedback information received from users and updates machine learning models based on the results obtained.

[0466] An "algorithm for evaluating the degree of agreement" is a computational procedure for evaluating how well received information matches past data or existing evaluation scores.

[0467] A "machine learning model" is a trained artificial intelligence model that improves the accuracy of information reliability evaluation based on feedback data.

[0468] The embodiments for carrying out the present invention will be described in detail below. The invention is a system for evaluating and classifying the reliability of information. This system is built around the interaction between terminals, servers, and users.

[0469] Information gathering

[0470] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. The terminal temporarily stores this information and sends it to the server. In this process, the input data is converted to JSON format and an HTTP POST request is used. For example, if a market analysis report is entered, the terminal sends it to the server.

[0471] Initial assessment of information

[0472] The server analyzes the information received from the terminal. First, it extracts metadata from the information source (such as the source and creation date), and then calculates an initial reliability score using an initial evaluation algorithm. In this process, for example, if the source is a government database, the initial reliability score is set higher. The server can use a central processing unit (CPU) or cloud-based analysis software.

[0473] Graded evaluation of reliability

[0474] The server then compares the received information with historical database data to assess its accuracy and consistency. Machine learning models (e.g., the BERT model) are used in this process. Evaluation may also be conducted through an expert review system. The server evaluates the agreement rate and consistency and collects the evaluation results.

[0475] Level Classification

[0476] The server calculates a final reliability score based on the results of a step-by-step evaluation and classifies the information into reliability levels. For example, if the agreement rate is 90% and the consistency is 85%, the reliability score will be 90 points, classifying it as A level (very reliable). The generated report is formatted in PDF format.

[0477] User Feedback

[0478] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. For example, the assessment results might be displayed on the user's screen as "Reliability: A level, 90% reliability." HTTP GET requests are used for communication between the server and the terminal.

[0479] Information provision

[0480] Users can utilize information that has been evaluated for reliability in business and public settings. For example, a user might use this information as part of a presentation at an internal company meeting. They can download the generated PDF report and incorporate it into their presentation.

[0481] Feedback collection and updating machine learning models

[0482] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server, which analyzes the feedback and updates the machine learning model. In this process, user-entered feedback is sent in JSON format and analyzed by the server. This improves the accuracy of information reliability evaluations in subsequent instances.

[0483] Examples of specific cases and prompt statements

[0484] For example, a user of a market analysis report:

[0485] Market Analysis Report:

[0486] Company A's sales increased by 20% compared to last year.

[0487] New product B's market share reached 15% in 2022.

[0488] If you input this information, the following prompt may be entered into the generating AI model:

[0489] Please evaluate the reliability of the following market analysis report.

[0490] Company A's sales increased by 20% compared to last year.

[0491] New product B's market share reached 15% in 2022.

[0492] Please classify the information from A (very reliable) to C (average reliability), taking into account the reliability of the source, the consistency of the information, and its accuracy.

[0493] By inputting this prompt into the generating AI model, its reliability is evaluated, and the results are fed back to the user.

[0494] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0495] Step 1: Enter information

[0496] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. When the user presses the "Submit" button, the terminal converts the input data into JSON format and sends an HTTP POST request to the server. The input is text information provided by the user, and the output is data converted into JSON format.

[0497] Specific actions:

[0498] When a user enters a market analysis report and clicks the "Submit" button, the terminal internally serializes the entered text information into JSON format and sends it to the server via an HTTP POST request.

[0499] Step 2: Initial information reception and analysis

[0500] The server parses the JSON data received from the terminal and extracts metadata of the information source (source, creation date, etc.). The server then uses an initial evaluation algorithm to calculate an initial confidence score. The input is the JSON data sent from the terminal, and the output is the analysis results, including the initial confidence score.

[0501] Specific actions:

[0502] The server receives an HTTP POST request, deserializes the JSON data to extract text information, then extracts metadata (e.g., source and creation date) and calculates an initial confidence score.

[0503] Step 3: Comparison with historical data

[0504] The server compares the information against historical databases to assess its accuracy and consistency. This is done using machine learning models (e.g., the BERT model). Inputs are metadata and text information, and outputs are detailed evaluation results (e.g., agreement rate, consistency score).

[0505] Specific actions:

[0506] The server uses the BERT model to compare the received text information with historical data in the database. As a result, it calculates the match rate and consistency score.

[0507] Step 4: Calculation and leveling of reliability scores

[0508] The server calculates a final reliability score based on the detailed evaluation results and classifies the information into reliability levels. It then generates a report based on the evaluation results. The input is the detailed evaluation results, and the output is the final reliability score, reliability level, and the generated report.

[0509] Specific actions:

[0510] The server integrates the detailed evaluation results and calculates a final reliability score, for example, "90 points." Based on this, the information is classified as A-level (highly reliable), and a report is generated in PDF format.

[0511] Step 5: Submit and view the report

[0512] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. The input is the generated report, and the output is the reliability assessment results displayed on the user's screen.

[0513] Specific actions:

[0514] The server sends a PDF report to the terminal via an HTTP GET request, and the terminal displays it on the user's screen. For example, a result such as "Confidence level: A, 90% reliability" might be displayed.

[0515] Step 6: Gathering Feedback

[0516] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The input is the feedback provided by the user, and the output is the feedback data sent to the server.

[0517] Specific actions:

[0518] When a user enters "Very helpful, highly accurate" into the feedback form and clicks the submit button, the device converts the feedback data into JSON format and sends it to the server.

[0519] Step 7: Analyze feedback and update the machine learning model

[0520] The server analyzes the received feedback and updates the machine learning model. The input is the feedback data sent from the terminal, and the output is the updated machine learning model.

[0521] Specific actions:

[0522] The server analyzes the feedback data and saves the results to a database. This data is then used to retrain the machine learning model during the next model update, improving the accuracy of the evaluation.

[0523] (Application Example 1)

[0524] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0525] Currently, many users access a wide variety of information through the internet, but there is a lack of means to evaluate its reliability. Especially in today's world, where fake news and misinformation spread easily, there is a need for technology to quickly and accurately identify reliable information. Conventional systems are limited to initial and incremental evaluations of information, making real-time evaluation difficult as users browse web pages and digital content. Therefore, a system is needed to evaluate the reliability of information in real time and provide this information to users.

[0526] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0527] In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, and server means for comparing the received information with past data and performing a step-by-step detailed evaluation. This makes it possible to easily and quickly evaluate the reliability of information. Furthermore, the server includes server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to a terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, client means for evaluating the reliability of web pages and digital content in real time and displaying it on a browser application, metadata extraction means for extracting metadata of web content accessed by the user and calculating an initial reliability score, detailed evaluation means for evaluating the reliability score in detail using a machine learning algorithm based on the input information, and client means for providing feedback on the reliability results of the web content to the user and collecting feedback on the reliability of the information. This makes it possible to evaluate the reliability of information in real time when a user views web pages and digital content and to provide reliable information quickly and accurately.

[0528] "Terminal means" refers to a device that includes hardware and software for a user to input information or to display information received from a server.

[0529] A "server system" is a system that analyzes information, calculates reliability scores, performs step-by-step detailed evaluations, and determines the final reliability level.

[0530] A "metadata extraction method" is a system for analyzing and extracting metadata such as the source and creation date of input information.

[0531] A "machine learning model" is an algorithmic model that learns patterns based on a vast dataset and evaluates the reliability of information.

[0532] A "browser application" is software used by users to access web pages and digital content, and it includes a function for evaluating the reliability of information.

[0533] A "client device" is a mechanism that acts as a terminal device, sending user input information to a server and displaying feedback from the server.

[0534] The "detailed evaluation method" is a system that, after the initial evaluation, compares the information entered with historical data in detail and uses a machine learning model to evaluate reliability with high accuracy.

[0535] A "report generation method" is a system that determines the reliability level based on a reliability score, formats the result, and generates a report.

[0536] A "feedback collection mechanism" is a system for receiving feedback from users, sending it to a server, and analyzing it.

[0537] This invention provides a system that evaluates the reliability of information in real time and provides it to the user. This system evaluates the reliability of web pages and digital content viewed by the user and presents the user with a reliability score and evaluation report.

[0538] Specifically, this system will be implemented using the following hardware and software.

[0539] Hardware:

[0540] Smartphone (iOS or Android)

[0541] Server (a cloud server equipped with a high-performance processor and a large-capacity database)

[0542] software:

[0543] Client applications: iOS (Swift), Android (Kotlin)

[0544] Server-side: Python, Flask / Django (Web frameworks)

[0545] Databases: PostgreSQL, Elasticsearch

[0546] Machine learning models: Scikit-learn, TensorFlow, PyTorch

[0547] Program processing

[0548] The terminal device (smartphone) inputs information about the web page accessed by the user and retrieves the URL and text content. The retrieved information is then sent to the server.

[0549] The server analyzes the received information and extracts metadata. In particular, it calculates an initial reliability score based on important metadata such as the source and creation date. At this stage, information from highly reliable sources such as government agencies and major news organizations receives a high score.

[0550] Next, the server compares the information with past database data and performs a step-by-step detailed evaluation. Specifically, it uses a machine learning model to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that was previously evaluated as highly reliable, it will receive a high score.

[0551] Based on these evaluation results, the server calculates a final reliability score and classifies the information into reliability levels (e.g., A level, B level, C level). The reliability level and evaluation results are generated in a report format and sent to the client application.

[0552] The terminal device (a smartphone browser application) displays the provided report to the user. The user can view the reliability assessment results in real time and make decisions based on the reliability of the information.

[0553] Furthermore, the terminal device sends user feedback to the server. The server device analyzes this feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations improves.

[0554] As a concrete example, the user can use the following prompt statement.

[0555] Please analyze the content of the provided URL and calculate a reliability score.

[0556] URL: "https: / / example.com / news / some-article"

[0557] In this way, the system of the present invention can evaluate the reliability of information from multiple perspectives and provide users with highly reliable information quickly and accurately. This enables users to make decisions based on highly reliable information.

[0558] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0559] Step 1:

[0560] The device (smartphone) obtains information about the web pages accessed by the user. When the user enters a URL into the browser application, the device accesses that URL and collects the text content and metadata (source, creation date, etc.) of the web page. The collected information is sent to the server.

[0561] Input: URL entered by the user

[0562] Output: Text content and metadata of the webpage

[0563] Step 2:

[0564] The server analyzes the information from the received web page and extracts metadata. In particular, it calculates an initial reliability score based on metadata such as the source and creation date. For example, information from government sources or highly reliable sources is assigned a high score.

[0565] Input: Text content and metadata of the webpage sent from the device.

[0566] Output: Initial reliability score

[0567] Step 3:

[0568] The server compares the input information with past database data and performs a step-by-step detailed evaluation. A machine learning model is used to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that has been previously evaluated as highly reliable, it will receive a high score.

[0569] Input: Initial reliability score and historical database

[0570] Output: Detailed evaluation score

[0571] Step 4:

[0572] The server calculates a final reliability score based on the detailed evaluation score. It then classifies the results into reliability levels (e.g., A level, B level, C level) and generates a report.

[0573] Input: Detailed evaluation score

[0574] Output: Report including reliability score and reliability level

[0575] Step 5:

[0576] The report generated by the server is sent to the terminal. The terminal displays the provided report to the user. The user can view the reliability evaluation results in real time and make decisions based on the reliability of the information.

[0577] Input: Report sent from server

[0578] Output: Display of reliability evaluation results to the user.

[0579] Step 6:

[0580] The terminal device collects user feedback and sends it to the server.

[0581] Input: User feedback

[0582] Output: Sending feedback data to the server

[0583] Step 7:

[0584] The server analyzes the received feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations is improved.

[0585] Input: User feedback data

[0586] Output: Updated machine learning model

[0587] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0588] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information, and by combining it with an emotion engine that recognizes the user's emotions, it improves the accuracy and reliability of information evaluation.

[0589] Information gathering

[0590] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions.

[0591] Initial assessment of information

[0592] The device temporarily stores the entered information and emotional data, and then sends it to the server using a secure communication protocol.

[0593] The server stores the information and sentiment data received from the terminal in a database. The server extracts metadata from the received information (source, creation date, etc.) and calculates an initial confidence score using an initial evaluation algorithm. Here, sentiment data obtained from the sentiment engine is also reflected in the calculation of the initial confidence score. For example, if the user shows very positive sentiment, the initial confidence score is adjusted.

[0594] Graded evaluation of reliability

[0595] The server compares the input information with historical databases. It evaluates the accuracy, consistency, and historical reliability scores of the information. Machine learning models and expert reviews are used for a step-by-step, detailed evaluation. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes.

[0596] Level Classification

[0597] The server calculates an overall reliability score based on its step-by-step evaluation. Sentimental data is also included in this score calculation. Based on the reliability score, the information is categorized into levels according to its reliability. For example, it may be classified into A level (very reliable), B level (quite reliable), etc. The classification results are then formatted into a report.

[0598] User Feedback

[0599] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[0600] Information provision

[0601] Users use information that has been evaluated for reliability in business and public settings. For example, this information might be used as presentation material in an internal company meeting.

[0602] Feedback collection and updating machine learning models

[0603] Users input feedback on the provided information via their device. Here too, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the emotions (positive or negative) the user feels when giving feedback.

[0604] The device sends emotional data along with feedback to the server. The server analyzes the received feedback and saves the results. The emotional data is also included in the analysis and is used to interpret the feedback. The server incorporates the feedback data into a machine learning model and updates the model. This improves the accuracy of reliability evaluations in subsequent attempts.

[0605] In this way, the system of the present invention, by combining an emotion engine, evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information. Furthermore, by utilizing emotion data, it achieves further improvements in the accuracy of the evaluation process and enhances the user experience.

[0606] The following describes the processing flow.

[0607] Step 1:

[0608] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine is activated and begins to recognize the user's emotions based on their facial expressions and tone of voice during input.

[0609] Step 2:

[0610] The terminal temporarily stores the emotional data recognized by the emotion engine along with the input information, and then sends it to the server using a secure communication protocol.

[0611] Step 3:

[0612] The server stores the information and sentiment data received from the terminal in a database.

[0613] Step 4:

[0614] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[0615] Step 5:

[0616] The server uses an initial assessment algorithm to calculate an initial confidence score based on metadata and sentiment data. For example, if a user expresses positive sentiment, the initial confidence score is adjusted accordingly.

[0617] Step 6:

[0618] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[0619] Step 7:

[0620] The server conducts a step-by-step detailed evaluation. In this step, machine learning models and expert reviews are used to assess the accuracy and absence of bias in the information. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes and used as complementary material for the assessment.

[0621] Step 8:

[0622] The server aggregates the evaluation results in stages and calculates an overall reliability score. Sentimental data is also reflected in this score calculation.

[0623] Step 9:

[0624] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[0625] Step 10:

[0626] The server formats the categorized information into a report format and sends it to the terminal.

[0627] Step 11:

[0628] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[0629] Step 12:

[0630] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[0631] Step 13:

[0632] When a user provides feedback on the information provided via their device, the emotion engine restarts to recognize and record the user's emotions at the time of feedback.

[0633] Step 14:

[0634] The device sends emotional data to the server along with feedback.

[0635] Step 15:

[0636] The server analyzes the feedback it receives and saves the results. Sentimental data is also included in the analysis and can be used to interpret the feedback.

[0637] Step 16:

[0638] The server incorporates feedback and sentiment data into the machine learning model, updating it. This improves the accuracy of reliability evaluations in subsequent attempts.

[0639] The above outlines the specific processing steps of a system that evaluates and classifies the reliability of information by combining it with an emotion engine. By using this system, users can obtain reliable information with greater accuracy and use it with confidence in business and public settings.

[0640] (Example 2)

[0641] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0642] In recent years, numerous systems have emerged to evaluate the reliability of information. However, these systems often lack consideration for the emotions users feel when providing information, resulting in limitations in their accuracy. Furthermore, there are challenges in the speed at which evaluation results are delivered to users. Against this backdrop, there is a growing demand for information reliability evaluation systems that incorporate user sentiment data.

[0643] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, server means for comparing the received information with past data and performing a stepwise detailed evaluation, server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to the terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, terminal means for recognizing the user's emotional data in real time and recording it, and server means for incorporating the recorded emotional data into the reliability evaluation process. This enables highly accurate reliability evaluation that reflects the user's emotions and rapid information provision.

[0644] "Terminal means" refers to a device that includes hardware and software for a user to input information and receive the processing results.

[0645] A "server system" is a central computer system designed to receive, store, and process information, and to generate and transmit evaluation results and reports.

[0646] An "initial reliability score" is a numerical index calculated to evaluate the reliability of received information at an initial stage.

[0647] "Historical data" refers to a collection of data that includes information collected and evaluated prior to being compared and referenced in the evaluation process.

[0648] A "step-by-step detailed evaluation" is a process of evaluating the accuracy, consistency, and reliability of information in multiple stages.

[0649] A "reliability score" is a numerical indicator that shows the result of a quantitative evaluation of the overall reliability of received information.

[0650] "Confidence level" refers to a category used to classify the reliability of information based on a reliability score.

[0651] A "report" is a document-format output that includes the results of the evaluation process, reliability scores, and reliability levels.

[0652] "Feedback" refers to input data that represents a user's opinion or impression of the information provided.

[0653] A "machine learning model" is an algorithm and structure that learns from data and makes future predictions and classifications.

[0654] "Emotional data" refers to data that recognizes and records the user's emotional state in real time during input and feedback.

[0655] This invention is a system for evaluating and categorizing the reliability of information, and by combining it with an emotion engine that recognizes user emotions, it improves the accuracy and reliability of information evaluation. Specific embodiments of this system are described below.

[0656] Information gathering

[0657] Users access a terminal and input text information, such as market analysis reports and news articles, into a dedicated information input form. Here, an emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions. Emotional data is a crucial element in reliability evaluation.

[0658] Initial assessment of information

[0659] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (e.g., HTTPS). The server stores the received information and sentiment data in a database, which may be an RDBMS such as MySQL or PostgreSQL. Furthermore, the server extracts metadata from the received information (source, creation date, etc.) and calculates an initial confidence score using an initial evaluation algorithm, which also takes sentiment data into consideration.

[0660] Graded evaluation of reliability

[0661] The server compares the input information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability scores of the information. For a stepwise, detailed evaluation, machine learning models and expert reviews are used. For example, machine learning libraries such as scikit-learn and TensorFlow are used to build models and further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation.

[0662] Level Classification

[0663] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This reliability score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Based on this, the server classifies the information and formats it into a report format (e.g., PDF or HTML). The Python ReportLab library is used to generate the report.

[0664] User Feedback

[0665] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability."

[0666] Information provision

[0667] Users use information that has been evaluated for reliability in business and public settings. For example, they might use this information as presentation material in an internal company meeting.

[0668] Feedback collection and updating machine learning models

[0669] Users input feedback on the provided information through their device. Here, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. The device sends the emotion data along with the feedback to the server. The server analyzes the received feedback and saves the results to a database. The emotion data is also included in the analysis and is used to interpret the content of the feedback. Finally, the server updates its machine learning model using the feedback data to improve the accuracy of reliability evaluations in the future.

[0670] Specific example

[0671] 1. When a user enters information into the input form for the "New Product Market Analysis Report," the emotion engine detects the user's positive emotions.

[0672] 2. The device sends this information and emotion data to the server.

[0673] 3. The server analyzes the received data and calculates an initial reliability score.

[0674] 4. The server performs a detailed evaluation and calculates a reliability score.

[0675] 5. The server categorizes the information based on its reliability score (e.g., Level A) and generates a report.

[0676] 6. The server sends the generated report to the terminal, and the user checks the evaluation results.

[0677] 7. Users utilize the information in their business and provide feedback on the results.

[0678] 8. The server receives feedback and updates the machine learning model.

[0679] Example of a prompt

[0680] "Please rate the reliability of the new product market analysis report. The sentiment during data entry was very positive."

[0681] This system allows users to quickly obtain reliable information and efficiently utilize it in business and public settings. By leveraging sentiment data, even more accurate evaluations can be achieved.

[0682] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0683] Step 1:

[0684] The user accesses the terminal and enters text information, such as market analysis reports and news articles, into a dedicated information input form. Along with the text information, the terminal uses an emotion engine to recognize the user's emotions at the time of input in real time and record them as emotion data. Input data: market analysis reports, news articles, and emotion data. Output data: recorded text information and emotion data.

[0685] Step 2:

[0686] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (HTTPS). Data processing involves formatting the input information and sentiment data and converting them into transmittable data packets. Input data: text information and sentiment data. Output data: data packets sent to the server.

[0687] Step 3:

[0688] The server stores the information and sentiment data received from the terminal in a database. An RDBMS such as MySQL or PostgreSQL is used as the database. Input data: Received text information and sentiment data. Output data: Information and sentiment data stored in the database.

[0689] Step 4:

[0690] The server extracts metadata (source, creation date, etc.) from the received information. This process uses libraries such as Python's BeautifulSoup. As a data processing step, metadata is parsed from the text information. Input data: Stored text information. Output data: Extracted metadata.

[0691] Step 5:

[0692] The server calculates an initial confidence score using an initial assessment algorithm. Sentiment data is also considered. The score calculation uses basic statistical processing and rule-based algorithms. Input data: Stored text information, sentiment data, metadata. Output data: Initial confidence score.

[0693] Step 6:

[0694] The server compares the entered information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability score of the information. Input data: Initial reliability score, stored text information. Output data: Matching results.

[0695] Step 7:

[0696] The server performs a detailed evaluation using machine learning models and expert reviews. For example, it builds evaluation models using machine learning libraries such as scikit-learn and TensorFlow to further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation. Input data: Matching results, sentiment data. Output data: Detailed evaluation results.

[0697] Step 8:

[0698] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Input data: Detailed evaluation results. Output data: Overall reliability score.

[0699] Step 9:

[0700] The server classifies information based on reliability scores and formats it into a report format (e.g., PDF or HTML). The report is generated using a Python library such as ReportLab. Input data: Overall reliability score. Output data: Generated report.

[0701] Step 10:

[0702] The server sends the generated report to the terminal. HTTPS is used again for this communication. Input data: Generated report. Output data: Sent report.

[0703] Step 11:

[0704] The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability." Input data: The submitted report. Output data: The displayed evaluation results.

[0705] Step 12:

[0706] Users use information that has been evaluated for reliability in business and public settings. For example, this information is used as presentation material in an internal company meeting. Input data: Information that has been evaluated for reliability. Output data: Information that was used.

[0707] Step 13:

[0708] The user inputs feedback on the provided information via a terminal. The emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. Input data: Feedback content, emotion data. Output data: Recorded feedback and emotion data.

[0709] Step 14:

[0710] The device sends emotional data along with feedback to the server. Input data: Recorded feedback and emotional data. Output data: Feedback and emotional data sent to the server.

[0711] Step 15:

[0712] The server analyzes the feedback it receives and stores the results in a database. Sentimental data is also included in the analysis and used to interpret the feedback. Input data: Feedback and sentimental data sent to the server. Output data: Analysis results.

[0713] Step 16:

[0714] The server updates the machine learning model using feedback data. This improves the accuracy of subsequent reliability evaluations. Input data: Analysis results. Output data: Updated machine learning model.

[0715] (Application Example 2)

[0716] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0717] Conventional advertising information reliability evaluation systems failed to fully utilize user sentiment data when assessing the accuracy and reliability of information. Therefore, improving the accuracy of sentiment-based evaluations was difficult, resulting in insufficient reliability evaluation results. Furthermore, the lack of mechanisms to effectively incorporate user feedback made it difficult to improve the quality of information provided.

[0718] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading advertising creatives, terminal means for collecting sentiment data, and server means for calculating a reliability score using the collected sentiment data. This makes it possible to collect and analyze user sentiment data in real time and reflect it in the reliability evaluation. Furthermore, by displaying the reliability evaluation results immediately, it becomes possible to quickly adjust the marketing strategy. In addition, by updating the machine learning model using user feedback data, it becomes possible to continuously improve the accuracy and reliability of the evaluation.

[0719] "Terminal means for inputting information" refers to electronic devices used by users to input information, including, for example, smartphones and tablets.

[0720] "Server means for calculating initial reliability score" refers to a server and related software that evaluates the initial reliability and calculates a score based on the input information.

[0721] "Server means for conducting step-by-step detailed evaluations" refers to a server and related software that compares input information with past data and conducts detailed evaluations step by step.

[0722] "Server means for determining reliability levels based on reliability scores and generating reports" refers to a server and related software that categorizes the reliability of information based on calculated reliability scores and generates the results as a report.

[0723] "Terminal means for sending generated reports to a terminal and displaying them to the user" refers to electronic devices and related software for sending reports generated by the server to the user's terminal and displaying those reports on that terminal.

[0724] "Terminal means for receiving user feedback and sending it to the server" refers to electronic devices and related software for users to input feedback and send that information to the server.

[0725] "Server means for analyzing feedback and updating machine learning models" refers to a server and related software that analyzes feedback sent by users and updates machine learning models based on the results.

[0726] "Terminal means for uploading advertising creatives" refers to electronic devices and related software used by users to upload advertising creatives (images and videos).

[0727] "Terminal means for collecting emotional data" refers to electronic devices and related software used to collect user emotions in real time using video feeds or cameras.

[0728] "Server means for calculating reliability scores using collected sentiment data" refers to a server and related software for calculating reliability scores based on collected sentiment data.

[0729] "Terminal means for displaying reliability evaluation results" refers to electronic equipment and related software for displaying calculated reliability scores and evaluation results to the user.

[0730] The system that realizes this application example integrates various means for performing information reliability evaluation and sentiment feedback analysis. The specific system configuration and processing flow are described below.

[0731] The server provides a system that includes "terminal means for uploading advertising creatives," "terminal means for collecting sentiment data," "server means for calculating a reliability score using the collected sentiment data," and "terminal means for displaying the reliability evaluation results." This configuration allows for the collection and analysis of user sentiment data in real time and the evaluation of information reliability from multiple perspectives.

[0732] First, users upload advertising creatives (images and videos) from their devices to the server. These devices are common electronic devices such as smartphones and tablets. Next, video feeds and cameras are used to collect emotional data. For example, the camera captures the face of a user viewing the advertising creative, and their emotions are recognized and recorded in real time. OpenCV or specific emotion recognition engines are used for this process.

[0733] The collected sentiment data is sent to the server and used as part of the initial confidence score calculation. The server uses this data to evaluate the accuracy and consistency of the information and calculates an overall confidence score. This includes cross-referencing with existing metadata and historical evaluation data. Furthermore, the server updates the confidence score using the sentiment data as a corrective factor and generates the results in a report format.

[0734] The generated report is sent to the device and displayed to the user. The user can review it and adjust specific advertising strategies based on the reliability assessment results. The report includes specific evaluations, such as "Reliability: High" and "Reliability Score: 95%".

[0735] After the advertisement is actually delivered, user feedback is collected again. The sentiment engine collects sentiment data during the feedback process and sends it to the server along with the feedback. The server analyzes this data and updates the parameters of the machine learning model to improve the accuracy of future reliability evaluations.

[0736] A concrete example would be uploading a new ad video and conducting a test session where several consumers watch the ad. The application would collect the consumers' emotional responses in real time and calculate the ad's credibility score. Alternatively, a prompt message such as, "A new ad video has been uploaded. Please analyze the emotional responses of consumers watching this video and calculate its credibility score," could be used.

[0737] In this way, the system of the present invention can utilize emotional data from multiple perspectives and evaluate the reliability of information with high accuracy. It can also contribute to the rapid adjustment of marketing strategies and the improvement of the user experience.

[0738] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0739] Step 1:

[0740] Users upload advertising creatives (images and videos) from their devices to the server. The input data is the advertising creative file, and the output data is the file stored on the server. The device also collects the file's metadata (creation date and time, uploader information, etc.) and sends it to the server. This process allows the advertising creatives to be managed within the system.

[0741] Step 2:

[0742] The device activates a video feed or camera to collect user emotion data. The input data is real-time video footage, and the output data is recognized emotion information (e.g., positive, negative, neutral). This process uses image analysis and emotion recognition engines based on OpenCV. The video data is analyzed frame by frame, and emotion information for each frame is collected and sent to the server.

[0743] Step 3:

[0744] The server receives the collected sentiment data and calculates an initial confidence score. The input data consists of sentiment information and ad creative metadata, and the output data is the initial confidence score. The server analyzes the collected sentiment data and calculates a normalized score (e.g., in the range of 0 to 100). A sentiment data correction algorithm is used for this process.

[0745] Step 4:

[0746] The server compares the received information with historical data and performs a step-by-step detailed evaluation. Input data includes advertising creatives, initial confidence scores, and historical evaluation data, while output data is a detailed confidence score. The server evaluates the degree of information consistency and historical confidence scores to calculate an overall confidence score. Cross-referencing with existing databases is performed at this stage.

[0747] Step 5:

[0748] The server determines the confidence level based on the confidence score and generates a report. The input data is a detailed confidence score, and the output data is a report that includes the confidence level and evaluation results. The server categorizes the confidence level of the advertisements based on the confidence score and formats the evaluation results into a report format. This makes the evaluation results easier to understand visually.

[0749] Step 6:

[0750] The server sends the generated report to the terminal and displays it to the user. The input data is the report, and the output data is the evaluation result displayed on the terminal. The terminal displays the report received from the server, allowing the user to view the reliability evaluation results. This enables the user to adjust their advertising strategy based on the evaluation results.

[0751] Step 7:

[0752] Users input feedback after ad delivery via their device and send it to the server. Input data consists of user feedback and sentiment data, while output data is the feedback information stored on the server. The device receives the user feedback and sends it to the server for analysis.

[0753] Step 8:

[0754] The server analyzes the feedback and updates the machine learning model based on the results. The input data consists of user feedback and sentiment data, while the output data is the updated machine learning model. The server analyzes the specific feedback content and sentiment data and updates the parameters of the machine learning model. This update improves the accuracy of reliability assessments in subsequent uses.

[0755] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0756] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0757] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0758] [Third Embodiment]

[0759] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0760] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0761] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0762] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0763] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0764] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0765] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0766] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0767] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0768] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0769] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0770] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0771] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information. The system of the present invention is built around the interaction between a terminal, a server, and a user.

[0772] Information gathering

[0773] The user accesses the device and enters information. Specifically, the user enters text information such as market analysis reports or news articles into an input form. Based on this, the device temporarily stores the entered information and sends it to the server.

[0774] Initial assessment of information

[0775] The server analyzes the information received from the terminal. The server extracts metadata from the information source (such as the source and creation date) and calculates an initial reliability score using an initial assessment algorithm. For example, if the information comes from a government database, the initial reliability score will be set higher.

[0776] Graded evaluation of reliability

[0777] The server then compares the input information with historical databases, evaluating its accuracy, consistency, and historical reliability scores. For a more detailed, step-by-step evaluation, machine learning models and expert reviews are used to determine the accuracy and any bias in the information.

[0778] Level Classification

[0779] The server calculates a reliability score based on the results of a step-by-step evaluation. This reliability score is used to classify the information into reliability levels. For example, it might be divided into A-level (very reliable), B-level (quite reliable), C-level (average reliability), etc. The classification results are then formatted into a report.

[0780] User Feedback

[0781] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[0782] Information provision

[0783] Users can utilize information that has been evaluated for reliability in business and public settings. A specific example would be a user using this information as presentation material in an internal company meeting.

[0784] Feedback collection and updating machine learning models

[0785] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The server analyzes the received feedback and updates the machine learning model based on the results. This improves the accuracy of information reliability assessments in subsequent instances.

[0786] In this way, the system of the present invention evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information.

[0787] The following describes the processing flow.

[0788] Step 1:

[0789] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form.

[0790] Step 2:

[0791] The terminal temporarily stores the entered information and sends it to the server using a secure communication protocol.

[0792] Step 3:

[0793] The server saves the information received from the terminal to the database.

[0794] Step 4:

[0795] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[0796] Step 5:

[0797] The server uses an initial assessment algorithm to calculate an initial reliability score based on metadata. For example, if the information source is a highly reliable government database, the initial reliability score will be set higher.

[0798] Step 6:

[0799] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[0800] Step 7:

[0801] The server conducts a step-by-step detailed assessment. In this step, machine learning models and expert reviews are used to evaluate the accuracy and absence of bias in the information.

[0802] Step 8:

[0803] The server aggregates the evaluation results in stages and calculates an overall reliability score.

[0804] Step 9:

[0805] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[0806] Step 10:

[0807] The server formats the categorized information into a report format and sends it to the terminal.

[0808] Step 11:

[0809] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[0810] Step 12:

[0811] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[0812] Step 13:

[0813] Users provide feedback on the information they receive via their device. Specifically, they write comments regarding the usefulness and accuracy of the information.

[0814] Step 14:

[0815] The device sends feedback to the server.

[0816] Step 15:

[0817] The server analyzes the received feedback and saves the results.

[0818] Step 16:

[0819] The server incorporates the feedback data into the machine learning model and updates it. This improves the accuracy of reliability evaluations in subsequent tests.

[0820] The above outlines the specific processing steps for a system that evaluates and classifies the reliability of information. By using this system, users can easily obtain reliable information and use it with confidence in business and public settings.

[0821] (Example 1)

[0822] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0823] Conventional information evaluation systems have challenges such as the susceptibility to subjectivity in assessing the reliability of information, and the difficulty in evaluating consistency and accuracy. In particular, there is a need to provide users with rapid and accurate feedback on evaluation results, but this is not being adequately achieved. Furthermore, improving the accuracy of evaluations after receiving feedback is also a practical problem.

[0824] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0825] In this invention, the server includes means for receiving information and calculating an initial reliability score, means for comparing the received information with past data and performing a step-by-step detailed evaluation, and means for determining a reliability level based on the reliability score and generating a report. This enables a multifaceted evaluation of the reliability of information, allowing users to quickly and accurately obtain reliable information.

[0826] "Device means for inputting information" refers to devices or terminals used by users to input information.

[0827] "Processor means for calculating initial reliability score" refers to a central processing unit (CPU) or microprocessor used to evaluate the initial reliability of received information and calculate a score.

[0828] "Processor means for performing step-by-step detailed evaluation" refers to a central processing unit (CPU) or microprocessor that compares received information with a past database and performs a detailed evaluation based on multiple evaluation criteria.

[0829] "Processor means for determining reliability levels based on reliability scores and generating reports" refers to a central processing unit (CPU) or microprocessor that calculates a final reliability score based on evaluation results, classifies information into reliability levels, and generates reports.

[0830] "Device means for sending generated reports to a device and displaying them to the user" refers to a device or terminal that sends reports generated by a server to a terminal so that the user can view and display them.

[0831] "Device means for receiving user feedback and sending it to a processor" refers to a device or terminal that receives feedback information provided by the user and sends it to a server.

[0832] "Processor means for analyzing feedback and updating machine learning models" refers to a central processing unit (CPU) or microprocessor that analyzes feedback information received from users and updates machine learning models based on the results obtained.

[0833] An "algorithm for evaluating the degree of agreement" is a computational procedure for evaluating how well received information matches past data or existing evaluation scores.

[0834] A "machine learning model" is a trained artificial intelligence model that improves the accuracy of information reliability evaluation based on feedback data.

[0835] The embodiments for carrying out the present invention will be described in detail below. The invention is a system for evaluating and classifying the reliability of information. This system is built around the interaction between terminals, servers, and users.

[0836] Information gathering

[0837] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. The terminal temporarily stores this information and sends it to the server. In this process, the input data is converted to JSON format and an HTTP POST request is used. For example, if a market analysis report is entered, the terminal sends it to the server.

[0838] Initial assessment of information

[0839] The server analyzes the information received from the terminal. First, it extracts metadata from the information source (such as the source and creation date), and then calculates an initial reliability score using an initial evaluation algorithm. In this process, for example, if the source is a government database, the initial reliability score is set higher. The server can use a central processing unit (CPU) or cloud-based analysis software.

[0840] Graded evaluation of reliability

[0841] The server then compares the received information with historical database data to assess its accuracy and consistency. Machine learning models (e.g., the BERT model) are used in this process. Evaluation may also be conducted through an expert review system. The server evaluates the agreement rate and consistency and collects the evaluation results.

[0842] Level Classification

[0843] The server calculates a final reliability score based on the results of a step-by-step evaluation and classifies the information into reliability levels. For example, if the agreement rate is 90% and the consistency is 85%, the reliability score will be 90 points, classifying it as A level (very reliable). The generated report is formatted in PDF format.

[0844] User Feedback

[0845] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. For example, the assessment results might be displayed on the user's screen as "Reliability: A level, 90% reliability." HTTP GET requests are used for communication between the server and the terminal.

[0846] Information provision

[0847] Users can utilize information that has been evaluated for reliability in business and public settings. For example, a user might use this information as part of a presentation at an internal company meeting. They can download the generated PDF report and incorporate it into their presentation.

[0848] Feedback collection and updating machine learning models

[0849] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server, which analyzes the feedback and updates the machine learning model. In this process, user-entered feedback is sent in JSON format and analyzed by the server. This improves the accuracy of information reliability evaluations in subsequent instances.

[0850] Examples of specific cases and prompt statements

[0851] For example, a user of a market analysis report:

[0852] Market Analysis Report:

[0853] Company A's sales increased by 20% compared to last year.

[0854] New product B's market share reached 15% in 2022.

[0855] If you input this information, the following prompt may be entered into the generating AI model:

[0856] Please evaluate the reliability of the following market analysis report.

[0857] Company A's sales increased by 20% compared to last year.

[0858] New product B's market share reached 15% in 2022.

[0859] Please classify the information from A (very reliable) to C (average reliability), taking into account the reliability of the source, the consistency of the information, and its accuracy.

[0860] By inputting this prompt into the generating AI model, its reliability is evaluated, and the results are fed back to the user.

[0861] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0862] Step 1: Enter information

[0863] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. When the user presses the "Submit" button, the terminal converts the input data into JSON format and sends an HTTP POST request to the server. The input is text information provided by the user, and the output is data converted into JSON format.

[0864] Specific actions:

[0865] When a user enters a market analysis report and clicks the "Submit" button, the terminal internally serializes the entered text information into JSON format and sends it to the server via an HTTP POST request.

[0866] Step 2: Initial information reception and analysis

[0867] The server parses the JSON data received from the terminal and extracts metadata of the information source (source, creation date, etc.). The server then uses an initial evaluation algorithm to calculate an initial confidence score. The input is the JSON data sent from the terminal, and the output is the analysis results, including the initial confidence score.

[0868] Specific actions:

[0869] The server receives an HTTP POST request, deserializes the JSON data to extract text information, then extracts metadata (e.g., source and creation date) and calculates an initial confidence score.

[0870] Step 3: Comparison with historical data

[0871] The server compares the information against historical databases to assess its accuracy and consistency. This is done using machine learning models (e.g., the BERT model). Inputs are metadata and text information, and outputs are detailed evaluation results (e.g., agreement rate, consistency score).

[0872] Specific actions:

[0873] The server uses the BERT model to compare the received text information with historical data in the database. As a result, it calculates the match rate and consistency score.

[0874] Step 4: Calculation and leveling of reliability scores

[0875] The server calculates a final reliability score based on the detailed evaluation results and classifies the information into reliability levels. It then generates a report based on the evaluation results. The input is the detailed evaluation results, and the output is the final reliability score, reliability level, and the generated report.

[0876] Specific actions:

[0877] The server integrates the detailed evaluation results and calculates a final reliability score, for example, "90 points." Based on this, the information is classified as A-level (highly reliable), and a report is generated in PDF format.

[0878] Step 5: Submit and view the report

[0879] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. The input is the generated report, and the output is the reliability assessment results displayed on the user's screen.

[0880] Specific actions:

[0881] The server sends a PDF report to the terminal via an HTTP GET request, and the terminal displays it on the user's screen. For example, a result such as "Confidence level: A, 90% reliability" might be displayed.

[0882] Step 6: Gathering Feedback

[0883] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The input is the feedback provided by the user, and the output is the feedback data sent to the server.

[0884] Specific actions:

[0885] When a user enters "Very helpful, highly accurate" into the feedback form and clicks the submit button, the device converts the feedback data into JSON format and sends it to the server.

[0886] Step 7: Analyze feedback and update the machine learning model

[0887] The server analyzes the received feedback and updates the machine learning model. The input is the feedback data sent from the terminal, and the output is the updated machine learning model.

[0888] Specific actions:

[0889] The server analyzes the feedback data and saves the results to a database. This data is then used to retrain the machine learning model during the next model update, improving the accuracy of the evaluation.

[0890] (Application Example 1)

[0891] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0892] Currently, many users access a wide variety of information through the internet, but there is a lack of means to evaluate its reliability. Especially in today's world, where fake news and misinformation spread easily, there is a need for technology to quickly and accurately identify reliable information. Conventional systems are limited to initial and incremental evaluations of information, making real-time evaluation difficult as users browse web pages and digital content. Therefore, a system is needed to evaluate the reliability of information in real time and provide this information to users.

[0893] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0894] In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, and server means for comparing the received information with past data and performing a step-by-step detailed evaluation. This makes it possible to easily and quickly evaluate the reliability of information. Furthermore, the server includes server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to a terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, client means for evaluating the reliability of web pages and digital content in real time and displaying it on a browser application, metadata extraction means for extracting metadata of web content accessed by the user and calculating an initial reliability score, detailed evaluation means for evaluating the reliability score in detail using a machine learning algorithm based on the input information, and client means for providing feedback on the reliability results of the web content to the user and collecting feedback on the reliability of the information. This makes it possible to evaluate the reliability of information in real time when a user views web pages and digital content and to provide reliable information quickly and accurately.

[0895] "Terminal means" refers to a device that includes hardware and software for a user to input information or to display information received from a server.

[0896] A "server system" is a system that analyzes information, calculates reliability scores, performs step-by-step detailed evaluations, and determines the final reliability level.

[0897] A "metadata extraction method" is a system for analyzing and extracting metadata such as the source and creation date of input information.

[0898] A "machine learning model" is an algorithmic model that learns patterns based on a vast dataset and evaluates the reliability of information.

[0899] A "browser application" is software used by users to access web pages and digital content, and it includes a function for evaluating the reliability of information.

[0900] A "client device" is a mechanism that acts as a terminal device, sending user input information to a server and displaying feedback from the server.

[0901] The "detailed evaluation method" is a system that, after the initial evaluation, compares the information entered with historical data in detail and uses a machine learning model to evaluate reliability with high accuracy.

[0902] A "report generation method" is a system that determines the reliability level based on a reliability score, formats the result, and generates a report.

[0903] A "feedback collection mechanism" is a system for receiving feedback from users, sending it to a server, and analyzing it.

[0904] This invention provides a system that evaluates the reliability of information in real time and provides it to the user. This system evaluates the reliability of web pages and digital content viewed by the user and presents the user with a reliability score and evaluation report.

[0905] Specifically, this system will be implemented using the following hardware and software.

[0906] Hardware:

[0907] Smartphone (iOS or Android)

[0908] Server (a cloud server equipped with a high-performance processor and a large-capacity database)

[0909] software:

[0910] Client applications: iOS (Swift), Android (Kotlin)

[0911] Server-side: Python, Flask / Django (Web frameworks)

[0912] Databases: PostgreSQL, Elasticsearch

[0913] Machine learning models: Scikit-learn, TensorFlow, PyTorch

[0914] Program processing

[0915] The terminal device (smartphone) inputs information about the web page accessed by the user and retrieves the URL and text content. The retrieved information is then sent to the server.

[0916] The server analyzes the received information and extracts metadata. In particular, it calculates an initial reliability score based on important metadata such as the source and creation date. At this stage, information from highly reliable sources such as government agencies and major news organizations receives a high score.

[0917] Next, the server compares the information with past database data and performs a step-by-step detailed evaluation. Specifically, it uses a machine learning model to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that was previously evaluated as highly reliable, it will receive a high score.

[0918] Based on these evaluation results, the server calculates a final reliability score and classifies the information into reliability levels (e.g., A level, B level, C level). The reliability level and evaluation results are generated in a report format and sent to the client application.

[0919] The terminal device (a smartphone browser application) displays the provided report to the user. The user can view the reliability assessment results in real time and make decisions based on the reliability of the information.

[0920] Furthermore, the terminal device sends user feedback to the server. The server device analyzes this feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations improves.

[0921] As a concrete example, the user can use the following prompt statement.

[0922] Please analyze the content of the provided URL and calculate a reliability score.

[0923] URL: "https: / / example.com / news / some-article"

[0924] In this way, the system of the present invention can evaluate the reliability of information from multiple perspectives and provide users with highly reliable information quickly and accurately. This enables users to make decisions based on highly reliable information.

[0925] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0926] Step 1:

[0927] The device (smartphone) obtains information about the web pages accessed by the user. When the user enters a URL into the browser application, the device accesses that URL and collects the text content and metadata (source, creation date, etc.) of the web page. The collected information is sent to the server.

[0928] Input: URL entered by the user

[0929] Output: Text content and metadata of the webpage

[0930] Step 2:

[0931] The server analyzes the information from the received web page and extracts metadata. In particular, it calculates an initial reliability score based on metadata such as the source and creation date. For example, information from government sources or highly reliable sources is assigned a high score.

[0932] Input: Text content and metadata of the webpage sent from the device.

[0933] Output: Initial reliability score

[0934] Step 3:

[0935] The server compares the input information with past database data and performs a step-by-step detailed evaluation. A machine learning model is used to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that has been previously evaluated as highly reliable, it will receive a high score.

[0936] Input: Initial reliability score and historical database

[0937] Output: Detailed evaluation score

[0938] Step 4:

[0939] The server calculates a final reliability score based on the detailed evaluation score. It then classifies the results into reliability levels (e.g., A level, B level, C level) and generates a report.

[0940] Input: Detailed evaluation score

[0941] Output: Report including reliability score and reliability level

[0942] Step 5:

[0943] The report generated by the server is sent to the terminal. The terminal displays the provided report to the user. The user can view the reliability evaluation results in real time and make decisions based on the reliability of the information.

[0944] Input: Report sent from server

[0945] Output: Display of reliability evaluation results to the user.

[0946] Step 6:

[0947] The terminal device collects user feedback and sends it to the server.

[0948] Input: User feedback

[0949] Output: Sending feedback data to the server

[0950] Step 7:

[0951] The server analyzes the received feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations is improved.

[0952] Input: User feedback data

[0953] Output: Updated machine learning model

[0954] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0955] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information, and by combining it with an emotion engine that recognizes the user's emotions, it improves the accuracy and reliability of information evaluation.

[0956] Information gathering

[0957] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions.

[0958] Initial assessment of information

[0959] The device temporarily stores the entered information and emotional data, and then sends it to the server using a secure communication protocol.

[0960] The server stores the information and sentiment data received from the terminal in a database. The server extracts metadata from the received information (source, creation date, etc.) and calculates an initial confidence score using an initial evaluation algorithm. Here, sentiment data obtained from the sentiment engine is also reflected in the calculation of the initial confidence score. For example, if the user shows very positive sentiment, the initial confidence score is adjusted.

[0961] Graded evaluation of reliability

[0962] The server compares the input information with historical databases. It evaluates the accuracy, consistency, and historical reliability scores of the information. Machine learning models and expert reviews are used for a step-by-step, detailed evaluation. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes.

[0963] Level Classification

[0964] The server calculates an overall reliability score based on its step-by-step evaluation. Sentimental data is also included in this score calculation. Based on the reliability score, the information is categorized into levels according to its reliability. For example, it may be classified into A level (very reliable), B level (quite reliable), etc. The classification results are then formatted into a report.

[0965] User Feedback

[0966] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[0967] Information provision

[0968] Users use information that has been evaluated for reliability in business and public settings. For example, this information might be used as presentation material in an internal company meeting.

[0969] Feedback collection and updating machine learning models

[0970] Users input feedback on the provided information via their device. Here too, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the emotions (positive or negative) the user feels when giving feedback.

[0971] The device sends emotional data along with feedback to the server. The server analyzes the received feedback and saves the results. The emotional data is also included in the analysis and is used to interpret the feedback. The server incorporates the feedback data into a machine learning model and updates the model. This improves the accuracy of reliability evaluations in subsequent attempts.

[0972] In this way, the system of the present invention, by combining an emotion engine, evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information. Furthermore, by utilizing emotion data, it achieves further improvements in the accuracy of the evaluation process and enhances the user experience.

[0973] The following describes the processing flow.

[0974] Step 1:

[0975] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine is activated and begins to recognize the user's emotions based on their facial expressions and tone of voice during input.

[0976] Step 2:

[0977] The terminal temporarily stores the emotional data recognized by the emotion engine along with the input information, and then sends it to the server using a secure communication protocol.

[0978] Step 3:

[0979] The server stores the information and sentiment data received from the terminal in a database.

[0980] Step 4:

[0981] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[0982] Step 5:

[0983] The server uses an initial assessment algorithm to calculate an initial confidence score based on metadata and sentiment data. For example, if a user expresses positive sentiment, the initial confidence score is adjusted accordingly.

[0984] Step 6:

[0985] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[0986] Step 7:

[0987] The server conducts a step-by-step detailed evaluation. In this step, machine learning models and expert reviews are used to assess the accuracy and absence of bias in the information. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes and used as complementary material for the assessment.

[0988] Step 8:

[0989] The server aggregates the evaluation results in stages and calculates an overall reliability score. Sentimental data is also reflected in this score calculation.

[0990] Step 9:

[0991] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[0992] Step 10:

[0993] The server formats the categorized information into a report format and sends it to the terminal.

[0994] Step 11:

[0995] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[0996] Step 12:

[0997] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[0998] Step 13:

[0999] When a user provides feedback on the information provided via their device, the emotion engine restarts to recognize and record the user's emotions at the time of feedback.

[1000] Step 14:

[1001] The device sends emotional data to the server along with feedback.

[1002] Step 15:

[1003] The server analyzes the feedback it receives and saves the results. Sentimental data is also included in the analysis and can be used to interpret the feedback.

[1004] Step 16:

[1005] The server incorporates feedback and sentiment data into the machine learning model, updating it. This improves the accuracy of reliability evaluations in subsequent attempts.

[1006] The above outlines the specific processing steps of a system that evaluates and classifies the reliability of information by combining it with an emotion engine. By using this system, users can obtain reliable information with greater accuracy and use it with confidence in business and public settings.

[1007] (Example 2)

[1008] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1009] In recent years, numerous systems have emerged to evaluate the reliability of information. However, these systems often lack consideration for the emotions users feel when providing information, resulting in limitations in their accuracy. Furthermore, there are challenges in the speed at which evaluation results are delivered to users. Against this backdrop, there is a growing demand for information reliability evaluation systems that incorporate user sentiment data.

[1010] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, server means for comparing the received information with past data and performing a stepwise detailed evaluation, server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to the terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, terminal means for recognizing the user's emotional data in real time and recording it, and server means for incorporating the recorded emotional data into the reliability evaluation process. This enables highly accurate reliability evaluation that reflects the user's emotions and rapid information provision.

[1011] "Terminal means" refers to a device that includes hardware and software for a user to input information and receive the processing results.

[1012] A "server system" is a central computer system designed to receive, store, and process information, and to generate and transmit evaluation results and reports.

[1013] An "initial reliability score" is a numerical index calculated to evaluate the reliability of received information at an initial stage.

[1014] "Historical data" refers to a collection of data that includes information collected and evaluated prior to being compared and referenced in the evaluation process.

[1015] A "step-by-step detailed evaluation" is a process of evaluating the accuracy, consistency, and reliability of information in multiple stages.

[1016] A "reliability score" is a numerical indicator that shows the result of a quantitative evaluation of the overall reliability of received information.

[1017] "Confidence level" refers to a category used to classify the reliability of information based on a reliability score.

[1018] A "report" is a document-format output that includes the results of the evaluation process, reliability scores, and reliability levels.

[1019] "Feedback" refers to input data that represents a user's opinion or impression of the information provided.

[1020] A "machine learning model" is an algorithm and structure that learns from data and makes future predictions and classifications.

[1021] "Emotional data" refers to data that recognizes and records the user's emotional state in real time during input and feedback.

[1022] This invention is a system for evaluating and categorizing the reliability of information, and by combining it with an emotion engine that recognizes user emotions, it improves the accuracy and reliability of information evaluation. Specific embodiments of this system are described below.

[1023] Information gathering

[1024] Users access a terminal and input text information, such as market analysis reports and news articles, into a dedicated information input form. Here, an emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions. Emotional data is a crucial element in reliability evaluation.

[1025] Initial assessment of information

[1026] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (e.g., HTTPS). The server stores the received information and sentiment data in a database, which may be an RDBMS such as MySQL or PostgreSQL. Furthermore, the server extracts metadata from the received information (source, creation date, etc.) and calculates an initial confidence score using an initial evaluation algorithm, which also takes sentiment data into consideration.

[1027] Graded evaluation of reliability

[1028] The server compares the input information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability scores of the information. For a stepwise, detailed evaluation, machine learning models and expert reviews are used. For example, machine learning libraries such as scikit-learn and TensorFlow are used to build models and further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation.

[1029] Level Classification

[1030] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This reliability score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Based on this, the server classifies the information and formats it into a report format (e.g., PDF or HTML). The Python ReportLab library is used to generate the report.

[1031] User Feedback

[1032] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability."

[1033] Information provision

[1034] Users use information that has been evaluated for reliability in business and public settings. For example, they might use this information as presentation material in an internal company meeting.

[1035] Feedback collection and updating machine learning models

[1036] Users input feedback on the provided information through their device. Here, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. The device sends the emotion data along with the feedback to the server. The server analyzes the received feedback and saves the results to a database. The emotion data is also included in the analysis and is used to interpret the content of the feedback. Finally, the server updates its machine learning model using the feedback data to improve the accuracy of reliability evaluations in the future.

[1037] Specific example

[1038] 1. When a user enters information into the input form for the "New Product Market Analysis Report," the emotion engine detects the user's positive emotions.

[1039] 2. The device sends this information and emotion data to the server.

[1040] 3. The server analyzes the received data and calculates an initial reliability score.

[1041] 4. The server performs a detailed evaluation and calculates a reliability score.

[1042] 5. The server categorizes the information based on its reliability score (e.g., Level A) and generates a report.

[1043] 6. The server sends the generated report to the terminal, and the user checks the evaluation results.

[1044] 7. Users utilize the information in their business and provide feedback on the results.

[1045] 8. The server receives feedback and updates the machine learning model.

[1046] Example of a prompt

[1047] "Please rate the reliability of the new product market analysis report. The sentiment during data entry was very positive."

[1048] This system allows users to quickly obtain reliable information and efficiently utilize it in business and public settings. By leveraging sentiment data, even more accurate evaluations can be achieved.

[1049] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1050] Step 1:

[1051] The user accesses the terminal and enters text information, such as market analysis reports and news articles, into a dedicated information input form. Along with the text information, the terminal uses an emotion engine to recognize the user's emotions at the time of input in real time and record them as emotion data. Input data: market analysis reports, news articles, and emotion data. Output data: recorded text information and emotion data.

[1052] Step 2:

[1053] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (HTTPS). Data processing involves formatting the input information and sentiment data and converting them into transmittable data packets. Input data: text information and sentiment data. Output data: data packets sent to the server.

[1054] Step 3:

[1055] The server stores the information and sentiment data received from the terminal in a database. An RDBMS such as MySQL or PostgreSQL is used as the database. Input data: Received text information and sentiment data. Output data: Information and sentiment data stored in the database.

[1056] Step 4:

[1057] The server extracts metadata (source, creation date, etc.) from the received information. This process uses libraries such as Python's BeautifulSoup. As a data processing step, metadata is parsed from the text information. Input data: Stored text information. Output data: Extracted metadata.

[1058] Step 5:

[1059] The server calculates an initial confidence score using an initial assessment algorithm. Sentiment data is also considered. The score calculation uses basic statistical processing and rule-based algorithms. Input data: Stored text information, sentiment data, metadata. Output data: Initial confidence score.

[1060] Step 6:

[1061] The server compares the entered information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability score of the information. Input data: Initial reliability score, stored text information. Output data: Matching results.

[1062] Step 7:

[1063] The server performs a detailed evaluation using machine learning models and expert reviews. For example, it builds evaluation models using machine learning libraries such as scikit-learn and TensorFlow to further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation. Input data: Matching results, sentiment data. Output data: Detailed evaluation results.

[1064] Step 8:

[1065] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Input data: Detailed evaluation results. Output data: Overall reliability score.

[1066] Step 9:

[1067] The server classifies information based on reliability scores and formats it into a report format (e.g., PDF or HTML). The report is generated using a Python library such as ReportLab. Input data: Overall reliability score. Output data: Generated report.

[1068] Step 10:

[1069] The server sends the generated report to the terminal. HTTPS is used again for this communication. Input data: Generated report. Output data: Sent report.

[1070] Step 11:

[1071] The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability." Input data: The submitted report. Output data: The displayed evaluation results.

[1072] Step 12:

[1073] Users use information that has been evaluated for reliability in business and public settings. For example, this information is used as presentation material in an internal company meeting. Input data: Information that has been evaluated for reliability. Output data: Information that was used.

[1074] Step 13:

[1075] The user inputs feedback on the provided information via a terminal. The emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. Input data: Feedback content, emotion data. Output data: Recorded feedback and emotion data.

[1076] Step 14:

[1077] The device sends emotional data along with feedback to the server. Input data: Recorded feedback and emotional data. Output data: Feedback and emotional data sent to the server.

[1078] Step 15:

[1079] The server analyzes the feedback it receives and stores the results in a database. Sentimental data is also included in the analysis and used to interpret the feedback. Input data: Feedback and sentimental data sent to the server. Output data: Analysis results.

[1080] Step 16:

[1081] The server updates the machine learning model using feedback data. This improves the accuracy of subsequent reliability evaluations. Input data: Analysis results. Output data: Updated machine learning model.

[1082] (Application Example 2)

[1083] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1084] Conventional advertising information reliability evaluation systems failed to fully utilize user sentiment data when assessing the accuracy and reliability of information. Therefore, improving the accuracy of sentiment-based evaluations was difficult, resulting in insufficient reliability evaluation results. Furthermore, the lack of mechanisms to effectively incorporate user feedback made it difficult to improve the quality of information provided.

[1085] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading advertising creatives, terminal means for collecting sentiment data, and server means for calculating a reliability score using the collected sentiment data. This makes it possible to collect and analyze user sentiment data in real time and reflect it in the reliability evaluation. Furthermore, by displaying the reliability evaluation results immediately, it becomes possible to quickly adjust the marketing strategy. In addition, by updating the machine learning model using user feedback data, it becomes possible to continuously improve the accuracy and reliability of the evaluation.

[1086] "Terminal means for inputting information" refers to electronic devices used by users to input information, including, for example, smartphones and tablets.

[1087] "Server means for calculating initial reliability score" refers to a server and related software that evaluates the initial reliability and calculates a score based on the input information.

[1088] "Server means for conducting step-by-step detailed evaluations" refers to a server and related software that compares input information with past data and conducts detailed evaluations step by step.

[1089] "Server means for determining reliability levels based on reliability scores and generating reports" refers to a server and related software that categorizes the reliability of information based on calculated reliability scores and generates the results as a report.

[1090] "Terminal means for sending generated reports to a terminal and displaying them to the user" refers to electronic devices and related software for sending reports generated by the server to the user's terminal and displaying those reports on that terminal.

[1091] "Terminal means for receiving user feedback and sending it to the server" refers to electronic devices and related software for users to input feedback and send that information to the server.

[1092] "Server means for analyzing feedback and updating machine learning models" refers to a server and related software that analyzes feedback sent by users and updates machine learning models based on the results.

[1093] "Terminal means for uploading advertising creatives" refers to electronic devices and related software used by users to upload advertising creatives (images and videos).

[1094] "Terminal means for collecting emotional data" refers to electronic devices and related software used to collect user emotions in real time using video feeds or cameras.

[1095] "Server means for calculating reliability scores using collected sentiment data" refers to a server and related software for calculating reliability scores based on collected sentiment data.

[1096] "Terminal means for displaying reliability evaluation results" refers to electronic equipment and related software for displaying calculated reliability scores and evaluation results to the user.

[1097] The system that realizes this application example integrates various means for performing information reliability evaluation and sentiment feedback analysis. The specific system configuration and processing flow are described below.

[1098] The server provides a system that includes "terminal means for uploading advertising creatives," "terminal means for collecting sentiment data," "server means for calculating a reliability score using the collected sentiment data," and "terminal means for displaying the reliability evaluation results." This configuration allows for the collection and analysis of user sentiment data in real time and the evaluation of information reliability from multiple perspectives.

[1099] First, users upload advertising creatives (images and videos) from their devices to the server. These devices are common electronic devices such as smartphones and tablets. Next, video feeds and cameras are used to collect emotional data. For example, the camera captures the face of a user viewing the advertising creative, and their emotions are recognized and recorded in real time. OpenCV or specific emotion recognition engines are used for this process.

[1100] The collected sentiment data is sent to the server and used as part of the initial confidence score calculation. The server uses this data to evaluate the accuracy and consistency of the information and calculates an overall confidence score. This includes cross-referencing with existing metadata and historical evaluation data. Furthermore, the server updates the confidence score using the sentiment data as a corrective factor and generates the results in a report format.

[1101] The generated report is sent to the device and displayed to the user. The user can review it and adjust specific advertising strategies based on the reliability assessment results. The report includes specific evaluations, such as "Reliability: High" and "Reliability Score: 95%".

[1102] After the advertisement is actually delivered, user feedback is collected again. The sentiment engine collects sentiment data during the feedback process and sends it to the server along with the feedback. The server analyzes this data and updates the parameters of the machine learning model to improve the accuracy of future reliability evaluations.

[1103] A concrete example would be uploading a new ad video and conducting a test session where several consumers watch the ad. The application would collect the consumers' emotional responses in real time and calculate the ad's credibility score. Alternatively, a prompt message such as, "A new ad video has been uploaded. Please analyze the emotional responses of consumers watching this video and calculate its credibility score," could be used.

[1104] In this way, the system of the present invention can utilize emotional data from multiple perspectives and evaluate the reliability of information with high accuracy. It can also contribute to the rapid adjustment of marketing strategies and the improvement of the user experience.

[1105] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1106] Step 1:

[1107] Users upload advertising creatives (images and videos) from their devices to the server. The input data is the advertising creative file, and the output data is the file stored on the server. The device also collects the file's metadata (creation date and time, uploader information, etc.) and sends it to the server. This process allows the advertising creatives to be managed within the system.

[1108] Step 2:

[1109] The device activates a video feed or camera to collect user emotion data. The input data is real-time video footage, and the output data is recognized emotion information (e.g., positive, negative, neutral). This process uses image analysis and emotion recognition engines based on OpenCV. The video data is analyzed frame by frame, and emotion information for each frame is collected and sent to the server.

[1110] Step 3:

[1111] The server receives the collected sentiment data and calculates an initial confidence score. The input data consists of sentiment information and ad creative metadata, and the output data is the initial confidence score. The server analyzes the collected sentiment data and calculates a normalized score (e.g., in the range of 0 to 100). A sentiment data correction algorithm is used for this process.

[1112] Step 4:

[1113] The server compares the received information with historical data and performs a step-by-step detailed evaluation. Input data includes advertising creatives, initial confidence scores, and historical evaluation data, while output data is a detailed confidence score. The server evaluates the degree of information consistency and historical confidence scores to calculate an overall confidence score. Cross-referencing with existing databases is performed at this stage.

[1114] Step 5:

[1115] The server determines the confidence level based on the confidence score and generates a report. The input data is a detailed confidence score, and the output data is a report that includes the confidence level and evaluation results. The server categorizes the confidence level of the advertisements based on the confidence score and formats the evaluation results into a report format. This makes the evaluation results easier to understand visually.

[1116] Step 6:

[1117] The server sends the generated report to the terminal and displays it to the user. The input data is the report, and the output data is the evaluation result displayed on the terminal. The terminal displays the report received from the server, allowing the user to view the reliability evaluation results. This enables the user to adjust their advertising strategy based on the evaluation results.

[1118] Step 7:

[1119] Users input feedback after ad delivery via their device and send it to the server. Input data consists of user feedback and sentiment data, while output data is the feedback information stored on the server. The device receives the user feedback and sends it to the server for analysis.

[1120] Step 8:

[1121] The server analyzes the feedback and updates the machine learning model based on the results. The input data consists of user feedback and sentiment data, while the output data is the updated machine learning model. The server analyzes the specific feedback content and sentiment data and updates the parameters of the machine learning model. This update improves the accuracy of reliability assessments in subsequent uses.

[1122] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1123] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1124] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1125] [Fourth Embodiment]

[1126] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1127] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1128] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1129] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1130] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1131] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1132] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1133] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1134] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1135] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1136] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1137] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1138] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1139] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information. The system of the present invention is built around the interaction between a terminal, a server, and a user.

[1140] Information gathering

[1141] The user accesses the device and enters information. Specifically, the user enters text information such as market analysis reports or news articles into an input form. Based on this, the device temporarily stores the entered information and sends it to the server.

[1142] Initial assessment of information

[1143] The server analyzes the information received from the terminal. The server extracts metadata from the information source (such as the source and creation date) and calculates an initial reliability score using an initial assessment algorithm. For example, if the information comes from a government database, the initial reliability score will be set higher.

[1144] Graded evaluation of reliability

[1145] The server then compares the input information with historical databases, evaluating its accuracy, consistency, and historical reliability scores. For a more detailed, step-by-step evaluation, machine learning models and expert reviews are used to determine the accuracy and any bias in the information.

[1146] Level Classification

[1147] The server calculates a reliability score based on the results of a step-by-step evaluation. This reliability score is used to classify the information into reliability levels. For example, it might be divided into A-level (very reliable), B-level (quite reliable), C-level (average reliability), etc. The classification results are then formatted into a report.

[1148] User Feedback

[1149] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[1150] Information provision

[1151] Users can utilize information that has been evaluated for reliability in business and public settings. A specific example would be a user using this information as presentation material in an internal company meeting.

[1152] Feedback collection and updating machine learning models

[1153] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The server analyzes the received feedback and updates the machine learning model based on the results. This improves the accuracy of information reliability assessments in subsequent instances.

[1154] In this way, the system of the present invention evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information.

[1155] The following describes the processing flow.

[1156] Step 1:

[1157] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form.

[1158] Step 2:

[1159] The terminal temporarily stores the entered information and sends it to the server using a secure communication protocol.

[1160] Step 3:

[1161] The server saves the information received from the terminal to the database.

[1162] Step 4:

[1163] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[1164] Step 5:

[1165] The server uses an initial assessment algorithm to calculate an initial reliability score based on metadata. For example, if the information source is a highly reliable government database, the initial reliability score will be set higher.

[1166] Step 6:

[1167] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[1168] Step 7:

[1169] The server conducts a step-by-step detailed assessment. In this step, machine learning models and expert reviews are used to evaluate the accuracy and absence of bias in the information.

[1170] Step 8:

[1171] The server aggregates the evaluation results in stages and calculates an overall reliability score.

[1172] Step 9:

[1173] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[1174] Step 10:

[1175] The server formats the categorized information into a report format and sends it to the terminal.

[1176] Step 11:

[1177] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[1178] Step 12:

[1179] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[1180] Step 13:

[1181] Users provide feedback on the information they receive via their device. Specifically, they write comments regarding the usefulness and accuracy of the information.

[1182] Step 14:

[1183] The device sends feedback to the server.

[1184] Step 15:

[1185] The server analyzes the received feedback and saves the results.

[1186] Step 16:

[1187] The server incorporates the feedback data into the machine learning model and updates it. This improves the accuracy of reliability evaluations in subsequent tests.

[1188] The above outlines the specific processing steps for a system that evaluates and classifies the reliability of information. By using this system, users can easily obtain reliable information and use it with confidence in business and public settings.

[1189] (Example 1)

[1190] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1191] Conventional information evaluation systems have challenges such as the susceptibility to subjectivity in assessing the reliability of information, and the difficulty in evaluating consistency and accuracy. In particular, there is a need to provide users with rapid and accurate feedback on evaluation results, but this is not being adequately achieved. Furthermore, improving the accuracy of evaluations after receiving feedback is also a practical problem.

[1192] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1193] In this invention, the server includes means for receiving information and calculating an initial reliability score, means for comparing the received information with past data and performing a step-by-step detailed evaluation, and means for determining a reliability level based on the reliability score and generating a report. This enables a multifaceted evaluation of the reliability of information, allowing users to quickly and accurately obtain reliable information.

[1194] "Device means for inputting information" refers to devices or terminals used by users to input information.

[1195] "Processor means for calculating initial reliability score" refers to a central processing unit (CPU) or microprocessor used to evaluate the initial reliability of received information and calculate a score.

[1196] "Processor means for performing step-by-step detailed evaluation" refers to a central processing unit (CPU) or microprocessor that compares received information with a past database and performs a detailed evaluation based on multiple evaluation criteria.

[1197] "Processor means for determining reliability levels based on reliability scores and generating reports" refers to a central processing unit (CPU) or microprocessor that calculates a final reliability score based on evaluation results, classifies information into reliability levels, and generates reports.

[1198] "Device means for sending generated reports to a device and displaying them to the user" refers to a device or terminal that sends reports generated by a server to a terminal so that the user can view and display them.

[1199] "Device means for receiving user feedback and sending it to a processor" refers to a device or terminal that receives feedback information provided by the user and sends it to a server.

[1200] "Processor means for analyzing feedback and updating machine learning models" refers to a central processing unit (CPU) or microprocessor that analyzes feedback information received from users and updates machine learning models based on the results obtained.

[1201] An "algorithm for evaluating the degree of agreement" is a computational procedure for evaluating how well received information matches past data or existing evaluation scores.

[1202] A "machine learning model" is a trained artificial intelligence model that improves the accuracy of information reliability evaluation based on feedback data.

[1203] The embodiments for carrying out the present invention will be described in detail below. The invention is a system for evaluating and classifying the reliability of information. This system is built around the interaction between terminals, servers, and users.

[1204] Information gathering

[1205] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. The terminal temporarily stores this information and sends it to the server. In this process, the input data is converted to JSON format and an HTTP POST request is used. For example, if a market analysis report is entered, the terminal sends it to the server.

[1206] Initial assessment of information

[1207] The server analyzes the information received from the terminal. First, it extracts metadata from the information source (such as the source and creation date), and then calculates an initial reliability score using an initial evaluation algorithm. In this process, for example, if the source is a government database, the initial reliability score is set higher. The server can use a central processing unit (CPU) or cloud-based analysis software.

[1208] Graded evaluation of reliability

[1209] The server then compares the received information with historical database data to assess its accuracy and consistency. Machine learning models (e.g., the BERT model) are used in this process. Evaluation may also be conducted through an expert review system. The server evaluates the agreement rate and consistency and collects the evaluation results.

[1210] Level Classification

[1211] The server calculates a final reliability score based on the results of a step-by-step evaluation and classifies the information into reliability levels. For example, if the agreement rate is 90% and the consistency is 85%, the reliability score will be 90 points, classifying it as A level (very reliable). The generated report is formatted in PDF format.

[1212] User Feedback

[1213] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. For example, the assessment results might be displayed on the user's screen as "Reliability: A level, 90% reliability." HTTP GET requests are used for communication between the server and the terminal.

[1214] Information provision

[1215] Users can utilize information that has been evaluated for reliability in business and public settings. For example, a user might use this information as part of a presentation at an internal company meeting. They can download the generated PDF report and incorporate it into their presentation.

[1216] Feedback collection and updating machine learning models

[1217] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server, which analyzes the feedback and updates the machine learning model. In this process, user-entered feedback is sent in JSON format and analyzed by the server. This improves the accuracy of information reliability evaluations in subsequent instances.

[1218] Examples of specific cases and prompt statements

[1219] For example, a user of a market analysis report:

[1220] Market Analysis Report:

[1221] Company A's sales increased by 20% compared to last year.

[1222] New product B's market share reached 15% in 2022.

[1223] If you input this information, the following prompt may be entered into the generating AI model:

[1224] Please evaluate the reliability of the following market analysis report.

[1225] Company A's sales increased by 20% compared to last year.

[1226] New product B's market share reached 15% in 2022.

[1227] Please classify the information from A (very reliable) to C (average reliability), taking into account the reliability of the source, the consistency of the information, and its accuracy.

[1228] By inputting this prompt into the generating AI model, its reliability is evaluated, and the results are fed back to the user.

[1229] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1230] Step 1: Enter information

[1231] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into an input form. When the user presses the "Submit" button, the terminal converts the input data into JSON format and sends an HTTP POST request to the server. The input is text information provided by the user, and the output is data converted into JSON format.

[1232] Specific actions:

[1233] When a user enters a market analysis report and clicks the "Submit" button, the terminal internally serializes the entered text information into JSON format and sends it to the server via an HTTP POST request.

[1234] Step 2: Initial information reception and analysis

[1235] The server parses the JSON data received from the terminal and extracts metadata of the information source (source, creation date, etc.). The server then uses an initial evaluation algorithm to calculate an initial confidence score. The input is the JSON data sent from the terminal, and the output is the analysis results, including the initial confidence score.

[1236] Specific actions:

[1237] The server receives an HTTP POST request, deserializes the JSON data to extract text information, then extracts metadata (e.g., source and creation date) and calculates an initial confidence score.

[1238] Step 3: Comparison with historical data

[1239] The server compares the information against historical databases to assess its accuracy and consistency. This is done using machine learning models (e.g., the BERT model). Inputs are metadata and text information, and outputs are detailed evaluation results (e.g., agreement rate, consistency score).

[1240] Specific actions:

[1241] The server uses the BERT model to compare the received text information with historical data in the database. As a result, it calculates the match rate and consistency score.

[1242] Step 4: Calculation and leveling of reliability scores

[1243] The server calculates a final reliability score based on the detailed evaluation results and classifies the information into reliability levels. It then generates a report based on the evaluation results. The input is the detailed evaluation results, and the output is the final reliability score, reliability level, and the generated report.

[1244] Specific actions:

[1245] The server integrates the detailed evaluation results and calculates a final reliability score, for example, "90 points." Based on this, the information is classified as A-level (highly reliable), and a report is generated in PDF format.

[1246] Step 5: Submit and view the report

[1247] The server sends the generated report to the terminal, and the terminal displays the reliability assessment results to the user. The input is the generated report, and the output is the reliability assessment results displayed on the user's screen.

[1248] Specific actions:

[1249] The server sends a PDF report to the terminal via an HTTP GET request, and the terminal displays it on the user's screen. For example, a result such as "Confidence level: A, 90% reliability" might be displayed.

[1250] Step 6: Gathering Feedback

[1251] Users provide feedback on deliverables and presentation results based on the information provided. The terminal sends this feedback to the server. The input is the feedback provided by the user, and the output is the feedback data sent to the server.

[1252] Specific actions:

[1253] When a user enters "Very helpful, highly accurate" into the feedback form and clicks the submit button, the device converts the feedback data into JSON format and sends it to the server.

[1254] Step 7: Analyze feedback and update the machine learning model

[1255] The server analyzes the received feedback and updates the machine learning model. The input is the feedback data sent from the terminal, and the output is the updated machine learning model.

[1256] Specific actions:

[1257] The server analyzes the feedback data and saves the results to a database. This data is then used to retrain the machine learning model during the next model update, improving the accuracy of the evaluation.

[1258] (Application Example 1)

[1259] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1260] Currently, many users access a wide variety of information through the internet, but there is a lack of means to evaluate its reliability. Especially in today's world, where fake news and misinformation spread easily, there is a need for technology to quickly and accurately identify reliable information. Conventional systems are limited to initial and incremental evaluations of information, making real-time evaluation difficult as users browse web pages and digital content. Therefore, a system is needed to evaluate the reliability of information in real time and provide this information to users.

[1261] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1262] In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, and server means for comparing the received information with past data and performing a step-by-step detailed evaluation. This makes it possible to easily and quickly evaluate the reliability of information. Furthermore, the server includes server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to a terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, client means for evaluating the reliability of web pages and digital content in real time and displaying it on a browser application, metadata extraction means for extracting metadata of web content accessed by the user and calculating an initial reliability score, detailed evaluation means for evaluating the reliability score in detail using a machine learning algorithm based on the input information, and client means for providing feedback on the reliability results of the web content to the user and collecting feedback on the reliability of the information. This makes it possible to evaluate the reliability of information in real time when a user views web pages and digital content and to provide reliable information quickly and accurately.

[1263] "Terminal means" refers to a device that includes hardware and software for a user to input information or to display information received from a server.

[1264] A "server system" is a system that analyzes information, calculates reliability scores, performs step-by-step detailed evaluations, and determines the final reliability level.

[1265] A "metadata extraction method" is a system for analyzing and extracting metadata such as the source and creation date of input information.

[1266] A "machine learning model" is an algorithmic model that learns patterns based on a vast dataset and evaluates the reliability of information.

[1267] A "browser application" is software used by users to access web pages and digital content, and it includes a function for evaluating the reliability of information.

[1268] A "client device" is a mechanism that acts as a terminal device, sending user input information to a server and displaying feedback from the server.

[1269] The "detailed evaluation method" is a system that, after the initial evaluation, compares the information entered with historical data in detail and uses a machine learning model to evaluate reliability with high accuracy.

[1270] A "report generation method" is a system that determines the reliability level based on a reliability score, formats the result, and generates a report.

[1271] A "feedback collection mechanism" is a system for receiving feedback from users, sending it to a server, and analyzing it.

[1272] This invention provides a system that evaluates the reliability of information in real time and provides it to the user. This system evaluates the reliability of web pages and digital content viewed by the user and presents the user with a reliability score and evaluation report.

[1273] Specifically, this system will be implemented using the following hardware and software.

[1274] Hardware:

[1275] Smartphone (iOS or Android)

[1276] Server (a cloud server equipped with a high-performance processor and a large-capacity database)

[1277] software:

[1278] Client applications: iOS (Swift), Android (Kotlin)

[1279] Server-side: Python, Flask / Django (Web frameworks)

[1280] Databases: PostgreSQL, Elasticsearch

[1281] Machine learning models: Scikit-learn, TensorFlow, PyTorch

[1282] Program processing

[1283] The terminal device (smartphone) inputs information about the web page accessed by the user and retrieves the URL and text content. The retrieved information is then sent to the server.

[1284] The server analyzes the received information and extracts metadata. In particular, it calculates an initial reliability score based on important metadata such as the source and creation date. At this stage, information from highly reliable sources such as government agencies and major news organizations receives a high score.

[1285] Next, the server compares the information with past database data and performs a step-by-step detailed evaluation. Specifically, it uses a machine learning model to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that was previously evaluated as highly reliable, it will receive a high score.

[1286] Based on these evaluation results, the server calculates a final reliability score and classifies the information into reliability levels (e.g., A level, B level, C level). The reliability level and evaluation results are generated in a report format and sent to the client application.

[1287] The terminal device (a smartphone browser application) displays the provided report to the user. The user can view the reliability assessment results in real time and make decisions based on the reliability of the information.

[1288] Furthermore, the terminal device sends user feedback to the server. The server device analyzes this feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations improves.

[1289] As a concrete example, the user can use the following prompt statement.

[1290] Please analyze the content of the provided URL and calculate a reliability score.

[1291] URL: "https: / / example.com / news / some-article"

[1292] In this way, the system of the present invention can evaluate the reliability of information from multiple perspectives and provide users with highly reliable information quickly and accurately. This enables users to make decisions based on highly reliable information.

[1293] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1294] Step 1:

[1295] The device (smartphone) obtains information about the web pages accessed by the user. When the user enters a URL into the browser application, the device accesses that URL and collects the text content and metadata (source, creation date, etc.) of the web page. The collected information is sent to the server.

[1296] Input: URL entered by the user

[1297] Output: Text content and metadata of the webpage

[1298] Step 2:

[1299] The server analyzes the information from the received web page and extracts metadata. In particular, it calculates an initial reliability score based on metadata such as the source and creation date. For example, information from government sources or highly reliable sources is assigned a high score.

[1300] Input: Text content and metadata of the webpage sent from the device.

[1301] Output: Initial reliability score

[1302] Step 3:

[1303] The server compares the input information with past database data and performs a step-by-step detailed evaluation. A machine learning model is used to assess the accuracy, consistency, and historical reliability score of the information. For example, if the content of an article matches content that has been previously evaluated as highly reliable, it will receive a high score.

[1304] Input: Initial reliability score and historical database

[1305] Output: Detailed evaluation score

[1306] Step 4:

[1307] The server calculates a final reliability score based on the detailed evaluation score. It then classifies the results into reliability levels (e.g., A level, B level, C level) and generates a report.

[1308] Input: Detailed evaluation score

[1309] Output: Report including reliability score and reliability level

[1310] Step 5:

[1311] The report generated by the server is sent to the terminal. The terminal displays the provided report to the user. The user can view the reliability evaluation results in real time and make decisions based on the reliability of the information.

[1312] Input: Report sent from server

[1313] Output: Display of reliability evaluation results to the user.

[1314] Step 6:

[1315] The terminal device collects user feedback and sends it to the server.

[1316] Input: User feedback

[1317] Output: Sending feedback data to the server

[1318] Step 7:

[1319] The server analyzes the received feedback and updates the machine learning model. By repeating this process, the accuracy of subsequent information reliability evaluations is improved.

[1320] Input: User feedback data

[1321] Output: Updated machine learning model

[1322] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1323] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system for evaluating and classifying the reliability of information, and by combining it with an emotion engine that recognizes the user's emotions, it improves the accuracy and reliability of information evaluation.

[1324] Information gathering

[1325] The user accesses the terminal and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions.

[1326] Initial assessment of information

[1327] The device temporarily stores the entered information and emotional data, and then sends it to the server using a secure communication protocol.

[1328] The server stores the information and sentiment data received from the terminal in a database. The server extracts metadata from the received information (source, creation date, etc.) and calculates an initial confidence score using an initial evaluation algorithm. Here, sentiment data obtained from the sentiment engine is also reflected in the calculation of the initial confidence score. For example, if the user shows very positive sentiment, the initial confidence score is adjusted.

[1329] Graded evaluation of reliability

[1330] The server compares the input information with historical databases. It evaluates the accuracy, consistency, and historical reliability scores of the information. Machine learning models and expert reviews are used for a step-by-step, detailed evaluation. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes.

[1331] Level Classification

[1332] The server calculates an overall reliability score based on its step-by-step evaluation. Sentimental data is also included in this score calculation. Based on the reliability score, the information is categorized into levels according to its reliability. For example, it may be classified into A level (very reliable), B level (quite reliable), etc. The classification results are then formatted into a report.

[1333] User Feedback

[1334] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can view evaluation results such as "Reliability: A level, 95% reliability."

[1335] Information provision

[1336] Users use information that has been evaluated for reliability in business and public settings. For example, this information might be used as presentation material in an internal company meeting.

[1337] Feedback collection and updating machine learning models

[1338] Users input feedback on the provided information via their device. Here too, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the emotions (positive or negative) the user feels when giving feedback.

[1339] The device sends emotional data along with feedback to the server. The server analyzes the received feedback and saves the results. The emotional data is also included in the analysis and is used to interpret the feedback. The server incorporates the feedback data into a machine learning model and updates the model. This improves the accuracy of reliability evaluations in subsequent attempts.

[1340] In this way, the system of the present invention, by combining an emotion engine, evaluates the reliability of information from multiple perspectives, enabling users to quickly and accurately obtain reliable information. Furthermore, by utilizing emotion data, it achieves further improvements in the accuracy of the evaluation process and enhances the user experience.

[1341] The following describes the processing flow.

[1342] Step 1:

[1343] The user opens their device and enters text information, such as market analysis reports or news articles, into a dedicated information input form. At this point, the emotion engine is activated and begins to recognize the user's emotions based on their facial expressions and tone of voice during input.

[1344] Step 2:

[1345] The terminal temporarily stores the emotional data recognized by the emotion engine along with the input information, and then sends it to the server using a secure communication protocol.

[1346] Step 3:

[1347] The server stores the information and sentiment data received from the terminal in a database.

[1348] Step 4:

[1349] The metadata (source, creation date, etc.) of the information received by the server is extracted.

[1350] Step 5:

[1351] The server uses an initial assessment algorithm to calculate an initial confidence score based on metadata and sentiment data. For example, if a user expresses positive sentiment, the initial confidence score is adjusted accordingly.

[1352] Step 6:

[1353] The server analyzes the text of the information and compares it with data collected in the past. Specifically, it checks the degree of accuracy of similar information in the past.

[1354] Step 7:

[1355] The server conducts a step-by-step detailed evaluation. In this step, machine learning models and expert reviews are used to assess the accuracy and absence of bias in the information. Sentimental data provided by the sentiment engine is also incorporated into these evaluation processes and used as complementary material for the assessment.

[1356] Step 8:

[1357] The server aggregates the evaluation results in stages and calculates an overall reliability score. Sentimental data is also reflected in this score calculation.

[1358] Step 9:

[1359] The server categorizes information into levels of reliability based on its reliability score. For example, it might classify information as A-level (very reliable), B-level (quite reliable), etc.

[1360] Step 10:

[1361] The server formats the categorized information into a report format and sends it to the terminal.

[1362] Step 11:

[1363] The terminal receives the report and displays the reliability assessment results to the user. For example, it might display "Reliability: A level, 95% reliability."

[1364] Step 12:

[1365] Users will use this information, which has been evaluated for reliability, in business and public settings. For example, it can be used as presentation material in internal company meetings.

[1366] Step 13:

[1367] When a user provides feedback on the information provided via their device, the emotion engine restarts to recognize and record the user's emotions at the time of feedback.

[1368] Step 14:

[1369] The device sends emotional data to the server along with feedback.

[1370] Step 15:

[1371] The server analyzes the feedback it receives and saves the results. Sentimental data is also included in the analysis and can be used to interpret the feedback.

[1372] Step 16:

[1373] The server incorporates feedback and sentiment data into the machine learning model, updating it. This improves the accuracy of reliability evaluations in subsequent attempts.

[1374] The above outlines the specific processing steps of a system that evaluates and classifies the reliability of information by combining it with an emotion engine. By using this system, users can obtain reliable information with greater accuracy and use it with confidence in business and public settings.

[1375] (Example 2)

[1376] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1377] In recent years, numerous systems have emerged to evaluate the reliability of information. However, these systems often lack consideration for the emotions users feel when providing information, resulting in limitations in their accuracy. Furthermore, there are challenges in the speed at which evaluation results are delivered to users. Against this backdrop, there is a growing demand for information reliability evaluation systems that incorporate user sentiment data.

[1378] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for inputting information, server means for receiving the input information and calculating an initial reliability score, server means for comparing the received information with past data and performing a stepwise detailed evaluation, server means for determining the reliability level based on the reliability score and generating a report, terminal means for sending the generated report to the terminal and displaying it to the user, terminal means for receiving feedback from the user and sending it to the server, server means for analyzing the feedback and updating the machine learning model, terminal means for recognizing the user's emotional data in real time and recording it, and server means for incorporating the recorded emotional data into the reliability evaluation process. This enables highly accurate reliability evaluation that reflects the user's emotions and rapid information provision.

[1379] "Terminal means" refers to a device that includes hardware and software for a user to input information and receive the processing results.

[1380] A "server system" is a central computer system designed to receive, store, and process information, and to generate and transmit evaluation results and reports.

[1381] An "initial reliability score" is a numerical index calculated to evaluate the reliability of received information at an initial stage.

[1382] "Historical data" refers to a collection of data that includes information collected and evaluated prior to being compared and referenced in the evaluation process.

[1383] A "step-by-step detailed evaluation" is a process of evaluating the accuracy, consistency, and reliability of information in multiple stages.

[1384] A "reliability score" is a numerical indicator that shows the result of a quantitative evaluation of the overall reliability of received information.

[1385] "Confidence level" refers to a category used to classify the reliability of information based on a reliability score.

[1386] A "report" is a document-format output that includes the results of the evaluation process, reliability scores, and reliability levels.

[1387] "Feedback" refers to input data that represents a user's opinion or impression of the information provided.

[1388] A "machine learning model" is an algorithm and structure that learns from data and makes future predictions and classifications.

[1389] "Emotional data" refers to data that recognizes and records the user's emotional state in real time during input and feedback.

[1390] This invention is a system for evaluating and categorizing the reliability of information, and by combining it with an emotion engine that recognizes user emotions, it improves the accuracy and reliability of information evaluation. Specific embodiments of this system are described below.

[1391] Information gathering

[1392] Users access a terminal and input text information, such as market analysis reports and news articles, into a dedicated information input form. Here, an emotion engine recognizes and records the user's emotions in real time as they input the information. For example, it can detect whether the user is expressing positive or negative emotions. Emotional data is a crucial element in reliability evaluation.

[1393] Initial assessment of information

[1394] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (e.g., HTTPS). The server stores the received information and sentiment data in a database, which may be an RDBMS such as MySQL or PostgreSQL. Furthermore, the server extracts metadata from the received information (source, creation date, etc.) and calculates an initial confidence score using an initial evaluation algorithm, which also takes sentiment data into consideration.

[1395] Graded evaluation of reliability

[1396] The server compares the input information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability scores of the information. For a stepwise, detailed evaluation, machine learning models and expert reviews are used. For example, machine learning libraries such as scikit-learn and TensorFlow are used to build models and further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation.

[1397] Level Classification

[1398] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This reliability score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Based on this, the server classifies the information and formats it into a report format (e.g., PDF or HTML). The Python ReportLab library is used to generate the report.

[1399] User Feedback

[1400] The server sends the generated report to the terminal. The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability."

[1401] Information provision

[1402] Users use information that has been evaluated for reliability in business and public settings. For example, they might use this information as presentation material in an internal company meeting.

[1403] Feedback collection and updating machine learning models

[1404] Users input feedback on the provided information through their device. Here, the emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. The device sends the emotion data along with the feedback to the server. The server analyzes the received feedback and saves the results to a database. The emotion data is also included in the analysis and is used to interpret the content of the feedback. Finally, the server updates its machine learning model using the feedback data to improve the accuracy of reliability evaluations in the future.

[1405] Specific example

[1406] 1. When a user enters information into the input form for the "New Product Market Analysis Report," the emotion engine detects the user's positive emotions.

[1407] 2. The device sends this information and emotion data to the server.

[1408] 3. The server analyzes the received data and calculates an initial reliability score.

[1409] 4. The server performs a detailed evaluation and calculates a reliability score.

[1410] 5. The server categorizes the information based on its reliability score (e.g., Level A) and generates a report.

[1411] 6. The server sends the generated report to the terminal, and the user checks the evaluation results.

[1412] 7. Users utilize the information in their business and provide feedback on the results.

[1413] 8. The server receives feedback and updates the machine learning model.

[1414] Example of a prompt

[1415] "Please rate the reliability of the new product market analysis report. The sentiment during data entry was very positive."

[1416] This system allows users to quickly obtain reliable information and efficiently utilize it in business and public settings. By leveraging sentiment data, even more accurate evaluations can be achieved.

[1417] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1418] Step 1:

[1419] The user accesses the terminal and enters text information, such as market analysis reports and news articles, into a dedicated information input form. Along with the text information, the terminal uses an emotion engine to recognize the user's emotions at the time of input in real time and record them as emotion data. Input data: market analysis reports, news articles, and emotion data. Output data: recorded text information and emotion data.

[1420] Step 2:

[1421] The terminal temporarily stores the entered information and sentiment data and sends it to the server using a secure communication protocol (HTTPS). Data processing involves formatting the input information and sentiment data and converting them into transmittable data packets. Input data: text information and sentiment data. Output data: data packets sent to the server.

[1422] Step 3:

[1423] The server stores the information and sentiment data received from the terminal in a database. An RDBMS such as MySQL or PostgreSQL is used as the database. Input data: Received text information and sentiment data. Output data: Information and sentiment data stored in the database.

[1424] Step 4:

[1425] The server extracts metadata (source, creation date, etc.) from the received information. This process uses libraries such as Python's BeautifulSoup. As a data processing step, metadata is parsed from the text information. Input data: Stored text information. Output data: Extracted metadata.

[1426] Step 5:

[1427] The server calculates an initial confidence score using an initial assessment algorithm. Sentiment data is also considered. The score calculation uses basic statistical processing and rule-based algorithms. Input data: Stored text information, sentiment data, metadata. Output data: Initial confidence score.

[1428] Step 6:

[1429] The server compares the entered information with historical data in a database. SQL queries are used to retrieve historical data and evaluate the accuracy, consistency, and historical reliability score of the information. Input data: Initial reliability score, stored text information. Output data: Matching results.

[1430] Step 7:

[1431] The server performs a detailed evaluation using machine learning models and expert reviews. For example, it builds evaluation models using machine learning libraries such as scikit-learn and TensorFlow to further verify the reliability of the information. Sentiment data provided by the sentiment engine is also incorporated into this evaluation. Input data: Matching results, sentiment data. Output data: Detailed evaluation results.

[1432] Step 8:

[1433] The server calculates an overall reliability score based on the results of its step-by-step evaluation. This score serves as a criterion for classifying the reliability of information into levels such as A (very reliable) and B (quite reliable). Input data: Detailed evaluation results. Output data: Overall reliability score.

[1434] Step 9:

[1435] The server classifies information based on reliability scores and formats it into a report format (e.g., PDF or HTML). The report is generated using a Python library such as ReportLab. Input data: Overall reliability score. Output data: Generated report.

[1436] Step 10:

[1437] The server sends the generated report to the terminal. HTTPS is used again for this communication. Input data: Generated report. Output data: Sent report.

[1438] Step 11:

[1439] The terminal displays the reliability evaluation results of the report to the user. The user can check the evaluation results in detail, such as "Reliability: A level, 95% reliability." Input data: The submitted report. Output data: The displayed evaluation results.

[1440] Step 12:

[1441] Users use information that has been evaluated for reliability in business and public settings. For example, this information is used as presentation material in an internal company meeting. Input data: Information that has been evaluated for reliability. Output data: Information that was used.

[1442] Step 13:

[1443] The user inputs feedback on the provided information via a terminal. The emotion engine recognizes and records the user's emotions in real time when they provide feedback. Specifically, it detects the user's emotions (positive or negative) when they give feedback. Input data: Feedback content, emotion data. Output data: Recorded feedback and emotion data.

[1444] Step 14:

[1445] The device sends emotional data along with feedback to the server. Input data: Recorded feedback and emotional data. Output data: Feedback and emotional data sent to the server.

[1446] Step 15:

[1447] The server analyzes the feedback it receives and stores the results in a database. Sentimental data is also included in the analysis and used to interpret the feedback. Input data: Feedback and sentimental data sent to the server. Output data: Analysis results.

[1448] Step 16:

[1449] The server updates the machine learning model using feedback data. This improves the accuracy of subsequent reliability evaluations. Input data: Analysis results. Output data: Updated machine learning model.

[1450] (Application Example 2)

[1451] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1452] Conventional advertising information reliability evaluation systems failed to fully utilize user sentiment data when assessing the accuracy and reliability of information. Therefore, improving the accuracy of sentiment-based evaluations was difficult, resulting in insufficient reliability evaluation results. Furthermore, the lack of mechanisms to effectively incorporate user feedback made it difficult to improve the quality of information provided.

[1453] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading advertising creatives, terminal means for collecting sentiment data, and server means for calculating a reliability score using the collected sentiment data. This makes it possible to collect and analyze user sentiment data in real time and reflect it in the reliability evaluation. Furthermore, by displaying the reliability evaluation results immediately, it becomes possible to quickly adjust the marketing strategy. In addition, by updating the machine learning model using user feedback data, it becomes possible to continuously improve the accuracy and reliability of the evaluation.

[1454] "Terminal means for inputting information" refers to electronic devices used by users to input information, including, for example, smartphones and tablets.

[1455] "Server means for calculating initial reliability score" refers to a server and related software that evaluates the initial reliability and calculates a score based on the input information.

[1456] "Server means for conducting step-by-step detailed evaluations" refers to a server and related software that compares input information with past data and conducts detailed evaluations step by step.

[1457] "Server means for determining reliability levels based on reliability scores and generating reports" refers to a server and related software that categorizes the reliability of information based on calculated reliability scores and generates the results as a report.

[1458] "Terminal means for sending generated reports to a terminal and displaying them to the user" refers to electronic devices and related software for sending reports generated by the server to the user's terminal and displaying those reports on that terminal.

[1459] "Terminal means for receiving user feedback and sending it to the server" refers to electronic devices and related software for users to input feedback and send that information to the server.

[1460] "Server means for analyzing feedback and updating machine learning models" refers to a server and related software that analyzes feedback sent by users and updates machine learning models based on the results.

[1461] "Terminal means for uploading advertising creatives" refers to electronic devices and related software used by users to upload advertising creatives (images and videos).

[1462] "Terminal means for collecting emotional data" refers to electronic devices and related software used to collect user emotions in real time using video feeds or cameras.

[1463] "Server means for calculating reliability scores using collected sentiment data" refers to a server and related software for calculating reliability scores based on collected sentiment data.

[1464] "Terminal means for displaying reliability evaluation results" refers to electronic equipment and related software for displaying calculated reliability scores and evaluation results to the user.

[1465] The system that realizes this application example integrates various means for performing information reliability evaluation and sentiment feedback analysis. The specific system configuration and processing flow are described below.

[1466] The server provides a system that includes "terminal means for uploading advertising creatives," "terminal means for collecting sentiment data," "server means for calculating a reliability score using the collected sentiment data," and "terminal means for displaying the reliability evaluation results." This configuration allows for the collection and analysis of user sentiment data in real time and the evaluation of information reliability from multiple perspectives.

[1467] First, users upload advertising creatives (images and videos) from their devices to the server. These devices are common electronic devices such as smartphones and tablets. Next, video feeds and cameras are used to collect emotional data. For example, the camera captures the face of a user viewing the advertising creative, and their emotions are recognized and recorded in real time. OpenCV or specific emotion recognition engines are used for this process.

[1468] The collected sentiment data is sent to the server and used as part of the initial confidence score calculation. The server uses this data to evaluate the accuracy and consistency of the information and calculates an overall confidence score. This includes cross-referencing with existing metadata and historical evaluation data. Furthermore, the server updates the confidence score using the sentiment data as a corrective factor and generates the results in a report format.

[1469] The generated report is sent to the device and displayed to the user. The user can review it and adjust specific advertising strategies based on the reliability assessment results. The report includes specific evaluations, such as "Reliability: High" and "Reliability Score: 95%".

[1470] After the advertisement is actually delivered, user feedback is collected again. The sentiment engine collects sentiment data during the feedback process and sends it to the server along with the feedback. The server analyzes this data and updates the parameters of the machine learning model to improve the accuracy of future reliability evaluations.

[1471] A concrete example would be uploading a new ad video and conducting a test session where several consumers watch the ad. The application would collect the consumers' emotional responses in real time and calculate the ad's credibility score. Alternatively, a prompt message such as, "A new ad video has been uploaded. Please analyze the emotional responses of consumers watching this video and calculate its credibility score," could be used.

[1472] In this way, the system of the present invention can utilize emotional data from multiple perspectives and evaluate the reliability of information with high accuracy. It can also contribute to the rapid adjustment of marketing strategies and the improvement of the user experience.

[1473] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1474] Step 1:

[1475] Users upload advertising creatives (images and videos) from their devices to the server. The input data is the advertising creative file, and the output data is the file stored on the server. The device also collects the file's metadata (creation date and time, uploader information, etc.) and sends it to the server. This process allows the advertising creatives to be managed within the system.

[1476] Step 2:

[1477] The device activates a video feed or camera to collect user emotion data. The input data is real-time video footage, and the output data is recognized emotion information (e.g., positive, negative, neutral). This process uses image analysis and emotion recognition engines based on OpenCV. The video data is analyzed frame by frame, and emotion information for each frame is collected and sent to the server.

[1478] Step 3:

[1479] The server receives the collected sentiment data and calculates an initial confidence score. The input data consists of sentiment information and ad creative metadata, and the output data is the initial confidence score. The server analyzes the collected sentiment data and calculates a normalized score (e.g., in the range of 0 to 100). A sentiment data correction algorithm is used for this process.

[1480] Step 4:

[1481] The server compares the received information with historical data and performs a step-by-step detailed evaluation. Input data includes advertising creatives, initial confidence scores, and historical evaluation data, while output data is a detailed confidence score. The server evaluates the degree of information consistency and historical confidence scores to calculate an overall confidence score. Cross-referencing with existing databases is performed at this stage.

[1482] Step 5:

[1483] The server determines the confidence level based on the confidence score and generates a report. The input data is a detailed confidence score, and the output data is a report that includes the confidence level and evaluation results. The server categorizes the confidence level of the advertisements based on the confidence score and formats the evaluation results into a report format. This makes the evaluation results easier to understand visually.

[1484] Step 6:

[1485] The server sends the generated report to the terminal and displays it to the user. The input data is the report, and the output data is the evaluation result displayed on the terminal. The terminal displays the report received from the server, allowing the user to view the reliability evaluation results. This enables the user to adjust their advertising strategy based on the evaluation results.

[1486] Step 7:

[1487] Users input feedback after ad delivery via their device and send it to the server. Input data consists of user feedback and sentiment data, while output data is the feedback information stored on the server. The device receives the user feedback and sends it to the server for analysis.

[1488] Step 8:

[1489] The server analyzes the feedback and updates the machine learning model based on the results. The input data consists of user feedback and sentiment data, while the output data is the updated machine learning model. The server analyzes the specific feedback content and sentiment data and updates the parameters of the machine learning model. This update improves the accuracy of reliability assessments in subsequent uses.

[1490] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1491] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1492] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1493] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1494] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1495] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1496] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1497] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1498] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1499] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1500] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1501] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1502] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1503] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1504] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1505] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1506] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1507] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1508] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1509] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1510] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1511] The following is further disclosed regarding the embodiments described above.

[1512] (Claim 1)

[1513] A terminal device for inputting information,

[1514] A server means for receiving input information and calculating an initial reliability score,

[1515] A server means for comparing received information with past data and performing a step-by-step detailed evaluation,

[1516] A server means for determining the confidence level based on the confidence score and generating a report,

[1517] A terminal means for sending the generated report to a terminal and displaying it to the user,

[1518] A terminal device for receiving user feedback and sending it to a server,

[1519] A server for analyzing feedback and updating machine learning models,

[1520] A system that includes this.

[1521] (Claim 2)

[1522] The system according to claim 1, wherein the stepwise detailed evaluation means uses an algorithm that evaluates the degree of agreement with past evaluation scores and existing data.

[1523] (Claim 3)

[1524] The system according to claim 1, wherein the machine learning model is updated based on user feedback data.

[1525] "Example 1"

[1526] (Claim 1)

[1527] A device for inputting information,

[1528] A processor means for receiving input information and calculating an initial reliability score,

[1529] A processor means for comparing received information with past data and performing a step-by-step detailed evaluation,

[1530] A processor means for determining the confidence level based on the confidence score and generating a report,

[1531] A device means for sending the generated report to a device and displaying it to the user,

[1532] A device means for receiving user feedback and transmitting it to a processor,

[1533] A processor for analyzing feedback and updating machine learning models,

[1534] A system that includes this.

[1535] (Claim 2)

[1536] The system according to claim 1, wherein the stepwise detailed evaluation means uses an algorithm that evaluates the degree of agreement with past evaluation scores and existing data.

[1537] (Claim 3)

[1538] The system according to claim 1, wherein the machine learning model is updated based on feedback data from users.

[1539] "Application Example 1"

[1540] (Claim 1)

[1541] A terminal device for inputting information,

[1542] A server means for receiving input information and calculating an initial reliability score,

[1543] A server means for comparing received information with past data and performing a step-by-step detailed evaluation,

[1544] A server means for determining the confidence level based on the confidence score and generating a report,

[1545] A terminal means for sending the generated report to a terminal and displaying it to the user,

[1546] A terminal device for receiving user feedback and sending it to a server,

[1547] A server for analyzing feedback and updating machine learning models,

[1548] A client-side means for evaluating the reliability of web pages and digital content in real time and displaying it on a browser application,

[1549] A metadata extraction means for extracting metadata from web content accessed by a user and calculating an initial reliability score,

[1550] A detailed evaluation method for evaluating reliability scores in detail using a machine learning algorithm based on the input information,

[1551] A client means for providing feedback to users on the reliability of web content and collecting feedback on the reliability of information,

[1552] A system that includes this.

[1553] (Claim 2)

[1554] The system according to claim 1, wherein the stepwise detailed evaluation means uses an algorithm that evaluates the degree of agreement with past evaluation scores and existing data.

[1555] (Claim 3)

[1556] The system according to claim 1, wherein the machine learning model is updated based on user feedback data.

[1557] "Example 2 of combining an emotion engine"

[1558] (Claim 1)

[1559] A terminal device for inputting information,

[1560] A server means for receiving input information and calculating an initial reliability score,

[1561] A server means for comparing received information with past data and performing a step-by-step detailed evaluation,

[1562] A server means for determining the confidence level based on the confidence score and generating a report,

[1563] A terminal means for sending the generated report to a terminal and displaying it to the user,

[1564] A terminal device for receiving user feedback and sending it to a server,

[1565] A server for analyzing feedback and updating machine learning models,

[1566] A terminal device for recognizing and recording user emotion data in real time,

[1567] A server means for incorporating recorded emotional data into a reliability evaluation process,

[1568] A system that includes this.

[1569] (Claim 2)

[1570] The system according to claim 1, wherein the stepwise detailed evaluation means uses an algorithm that evaluates the degree of agreement with past evaluation scores and existing data.

[1571] (Claim 3)

[1572] The system according to claim 1, wherein the machine learning model is updated based on user feedback data and sentiment data.

[1573] "Application example 2 when combining with an emotional engine"

[1574] (Claim 1)

[1575] A terminal device for inputting information,

[1576] A server means for receiving input information and calculating an initial reliability score,

[1577] A server means for comparing received information with past data and performing a step-by-step detailed evaluation,

[1578] A server means for determining the confidence level based on the confidence score and generating a report,

[1579] A terminal means for sending the generated report to a terminal and displaying it to the user,

[1580] A terminal device for receiving user feedback and sending it to a server,

[1581] A server for analyzing feedback and updating machine learning models,

[1582] A terminal device for uploading advertising creatives,

[1583] A terminal device for collecting emotional data,

[1584] A server means for calculating a reliability score using collected sentiment data,

[1585] A terminal means for displaying reliability evaluation results,

[1586] A system that includes this.

[1587] (Claim 2)

[1588] The system according to claim 1, wherein the stepwise detailed evaluation means uses an algorithm that evaluates the degree of agreement with past evaluation scores and existing data.

[1589] (Claim 3)

[1590] The system according to claim 1, wherein the machine learning model is updated based on user feedback data. [Explanation of symbols]

[1591] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A terminal device for inputting information, A server means for receiving input information and calculating an initial reliability score, A server means for comparing received information with past data and performing a step-by-step detailed evaluation, A server means for determining the confidence level based on the confidence score and generating a report, A terminal means for sending the generated report to a terminal and displaying it to the user, A terminal device for receiving user feedback and sending it to a server, A server for analyzing feedback and updating machine learning models, A system that includes this.

2. The system according to claim 1, wherein the stepwise detailed evaluation means uses an algorithm that evaluates the degree of agreement with past evaluation scores and existing data.

3. The system according to claim 1, wherein the machine learning model is updated based on user feedback data.

Images