Generative AI evaluation system
The generative AI evaluation system addresses the risk of misinformation by evaluating responses from multiple perspectives, ensuring reliable educational content through integrated scoring and teacher feedback.
JP7756468B1Active Publication Date: 2025-10-20小澤 暢吾
5 Cites 3 Cited by
Patent Information
- Application Number
- JP2025108325
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2025-06-26
- Publication Date
- 2025-10-20
- Estimated Expiration
- 2045-06-26
AI Technical Summary
Technical Problem
Generative AI responses in educational settings may contain misinformation, posing a risk of providing incorrect knowledge to students.
Method used
A generative AI evaluation system that includes a subject consistency evaluation unit, semantic matching evaluation unit, factuality verification unit, and integrated evaluation unit to assess response reliability, with teacher and administrator controls for final confirmation and management.
Benefits of technology
Ensures the provision of highly reliable information by objectively evaluating responses from multiple perspectives, preventing misinformation and supporting educational instruction through teacher feedback and visualization.
✦ Generated by Eureka AI based on patent content.
Abstract
The goal is to objectively and multifacetedly evaluate the reliability of the response sentences output by the generative AI. [Solution] The generative AI evaluation system 10 is a generative AI evaluation system that evaluates the reliability of response sentences output by a generative AI, and includes: a subject consistency evaluation unit 12 that calculates a Z-score based on the relationship between the response sentence and subject consistency; a semantic matching evaluation unit 14 that evaluates the semantic proximity between the response sentence and a knowledge base and calculates an RAG score; a factuality verification unit 16 that divides the response sentence into sentences and verifies its factuality by matching it with external information; an integrated evaluation unit 18 that calculates a reliability integrated score by weighted averaging based on the Z-score, RAG score, and the results of the factuality verification; and an output control unit 20 that controls whether to output the response sentence and the display format depending on the reliability integrated score.
Need to check novelty before this filing date? Find Prior Art