An artificial intelligence information education system and method based on a RISC-V processor architecture

Abstract

This application discloses an artificial intelligence information education system and method based on a RISC-V processor architecture, relating to the field of artificial intelligence information education. The system includes: a central processing unit integrating multiple RISC-V CPU cores and an AI-accelerated neural network processing unit; a multimodal emotion coprocessor performing multi-label emotion classification and emotion intensity regression based on multimodal input data to obtain the user's emotion analysis results; the central processing unit assigning different AI task subgraphs to corresponding computing units; a RAG retrieval acceleration module retrieving relevant text fragments from an educational knowledge base to obtain enhanced cue context; and a neural network processor generating natural language responses to the multimodal input data using a large language model inference engine based on the enhanced cue context and the user's emotion analysis results. This application can provide real-time, secure, and human-like personalized AI tutoring and emotional interaction for teaching scenarios in completely offline or weak network environments.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence information education, and in particular to an artificial intelligence information education system and method based on the RISC-V processor architecture. Background Technology

[0002] With the deep integration of "digital education" and "artificial intelligence + education," intelligent education terminals are evolving from auxiliary tools to core teaching carriers. However, the underlying hardware system supporting such applications faces significant challenges. Current mainstream solutions suffer from three main defects: First, the use of general-purpose mobile chips, such as System on Chip (SoC), results in high power consumption and short battery life during local inference, making it difficult to achieve affordable costs. Second, cloud-based collaborative solutions upload core artificial intelligence (AI) tasks for processing, heavily relying on the network, experiencing high latency, and posing a privacy risk to sensitive data such as student voice, images, and behavior during transmission, violating regulations on local data processing. Third, the use of single-function dedicated chips that only support voice wake-up or simple optical character recognition (OCR) cannot understand complex contexts and student emotions, resulting in mechanical and rigid interactions that hinder personalized learning.

[0003] Therefore, there is an urgent need for a dedicated AI education system that is low-power, highly secure, and highly interactive, in order to achieve real-time multimodal perception, personalized knowledge services, and anthropomorphic emotional interaction in offline environments. Summary of the Invention

[0004] The purpose of this application is to provide an artificial intelligence information education system and method based on the RISC-V processor architecture, which can provide real-time, secure, and human-like personalized AI tutoring and emotional interaction for teaching scenarios in completely offline or weak network environments.

[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides an artificial intelligence information education system based on a RISC-V processor architecture, comprising: a central processing unit and an AI-accelerated neural network processing unit; the central processing unit is connected to the AI-accelerated neural network processing unit; The central processing unit integrates multiple RISC-V CPU cores; the AI-accelerated neural network processing unit includes: a neural network processor, a RAG retrieval acceleration module, and a multimodal emotion coprocessor; the neural network processor has a built-in large language model inference engine; multimodal input data is used to input the central processing unit and the AI-accelerated neural network processing unit; the multimodal input data includes: interactive tasks issued by the user in different ways; The multimodal emotion coprocessor is used to extract and fuse features from the multimodal input data to obtain a fused feature vector, and to perform multi-label emotion classification and emotion intensity regression based on the fused features to obtain the user's emotion analysis results. The central processing unit is used to perform computation graph analysis on the multimodal input data. Based on the task characteristics obtained from the analysis, combined with the historical performance database and the set scheduling rules, it allocates different AI task subgraphs in the multimodal input data to the corresponding computing units. The computing units are RISC-V CPU cores and / or neural network processors. The neural network processor is used to convert the text in the multimodal input data into a query vector for an AI task subgraph that requires knowledge retrieval, and send the query vector to the RAG retrieval acceleration module through the central processing unit; The RAG retrieval acceleration module is used to retrieve text fragments related to the multimodal input data from the educational knowledge base based on the query vector; The central processing unit is also used to combine the text fragment with the text in the multimodal input data to obtain an enhanced prompt context, and send the enhanced prompt context to the neural network processor; The neural network processor is also used to generate a natural language response to the multimodal input data based on the enhanced cue context and the user's sentiment analysis results, using the large language model inference engine.

[0006] In one embodiment, the multimodal input data specifically includes: interactive tasks issued by the user in the form of voice, images, and text; The multimodal emotion coprocessor is specifically used for: The text in the multimodal input data is subjected to sentiment dictionary matching and semantic sentiment encoding to obtain text features; the images in the multimodal input data are subjected to face detection, pixel alignment and facial action extraction to obtain image features; and the speech in the multimodal input data is subjected to prosodic feature extraction to obtain speech features. A lightweight controller is used to evaluate the quality of the text features, image features, and speech features, and a fusion strategy is determined based on the evaluation results. The fusion strategy is then used to fuse the text features, image features, and speech features to obtain a fused feature vector. The fused feature vector is input into a multilayer perceptron for multi-label sentiment classification and sentiment intensity regression to obtain the user's sentiment analysis results.

[0007] In one embodiment, in performing computational graph analysis on the multimodal input data, and allocating different AI task subgraphs in the multimodal input data to corresponding computing units based on the analyzed task characteristics, combined with a historical performance database and set scheduling rules, the central processing unit is specifically used for: The multimodal input data is split into multiple AI task subgraphs; A dynamic perception scheduler is used to perform three-dimensional feature analysis on the AI ​​task subgraph to obtain task features; the task features include: operator type, data size and computational precision. Based on the task characteristics and historical performance database, predict the overhead data for the execution of the AI ​​task subgraph on the neural network processor and each of the RISC-V CPU cores; the overhead data includes: estimated time and estimated energy consumption; The AI ​​task subgraphs are dynamically scheduled according to the overhead data and the set scheduling rules, and different AI task subgraphs are assigned to the corresponding computing units. The scheduling rules include: if the regularity of the AI ​​task subgraph is greater than a set regularity value and the computational density is greater than a set density value, then the AI ​​task subgraph is assigned to the neural network processor; if the control complexity of the AI ​​task subgraph is greater than a set complexity value or the data volume is less than a set value, then the AI ​​task subgraph is assigned to multiple RISC-V CPU cores for parallel execution; if the AI ​​task subgraph is a hybrid task subgraph or the expected efficiency of a single hardware component is lower than a set efficiency value, then the AI ​​task subgraph is segmented and pipelined, with the regular computational subgraph assigned to the neural network processor and the dynamic control subgraph assigned to the RISC-V CPU cores.

[0008] In one embodiment, the RAG retrieval acceleration module is specifically used for: Calculate the cosine similarity between the query vector and the vector corresponding to the text block in the educational knowledge base, and determine the text segments related to the multimodal input data from the set number of text blocks with the highest cosine similarity.

[0009] In one embodiment, in generating a natural language response to the multimodal input data using the large language model inference engine based on the enhanced cue context and the user's sentiment analysis results, the neural network processor is specifically configured to: The enhanced prompt context is input into the large language model inference engine, and the generation style of the large language model inference engine is adjusted according to the user's sentiment analysis results to obtain a natural language response to the multimodal input data.

[0010] In one embodiment, the AI-based information education system based on the RISC-V processor architecture further includes: a graphics processing module; the graphics processing module is used to display the user's sentiment analysis results and the natural language response.

[0011] In one embodiment, the AI ​​information education system based on the RISC-V processor architecture further includes: a peripheral interface; the peripheral interface is used to collect the multimodal input data.

[0012] In one embodiment, the AI ​​information education system based on the RISC-V processor architecture further includes: an on-chip interconnect bus; the central processing unit and the AI ​​accelerated neural network processing unit are connected through the on-chip interconnect bus.

[0013] Secondly, this application provides an artificial intelligence information education method based on a RISC-V processor architecture. This method is used in the aforementioned artificial intelligence information education system based on a RISC-V processor architecture. The method includes: Acquire multimodal input data; the multimodal input data includes: interactive tasks issued by the user in different ways; Feature extraction and feature fusion are performed on the multimodal input data to obtain a fused feature vector. Multi-label sentiment classification and sentiment intensity regression are then performed based on the fused features to obtain the user's sentiment analysis results. The multimodal input data is subjected to computational graph analysis. Based on the task characteristics obtained from the analysis, combined with the historical performance database and the set scheduling rules, different AI task subgraphs in the multimodal input data are assigned to corresponding computing units. The computing units are RISC-V CPU cores and / or neural network processors. For AI task subgraphs that require knowledge retrieval, the text in the multimodal input data is converted into a query vector. Based on the query vector, text fragments related to the multimodal input data are retrieved from the educational knowledge base. The text fragments are then combined with the text in the multimodal input data to obtain an enhanced prompt context. Based on the enhanced prompt context and the user's sentiment analysis results, the large language model inference engine is used to generate a natural language response to the multimodal input data.

[0014] In one embodiment, feature extraction and feature fusion are performed on the multimodal input data to obtain a fused feature vector. Multi-label sentiment classification and sentiment intensity regression are then performed based on the fused features to obtain the user's sentiment analysis results. Specifically, this includes: The text in the multimodal input data is subjected to sentiment dictionary matching and semantic sentiment encoding to obtain text features; the images in the multimodal input data are subjected to face detection, pixel alignment and facial action extraction to obtain image features; and the speech in the multimodal input data is subjected to prosodic feature extraction to obtain speech features. A lightweight controller is used to evaluate the quality of the text features, image features, and speech features, and a fusion strategy is determined based on the evaluation results. The fusion strategy is then used to fuse the text features, image features, and speech features to obtain a fused feature vector. The fused feature vector is input into a multilayer perceptron for multi-label sentiment classification and sentiment intensity regression to obtain the user's sentiment analysis results.

[0015] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides an AI-powered information education system and method based on a RISC-V processor architecture. Through a central processing unit and an AI-accelerated neural network processing unit (including a neural network processor, a RAG retrieval acceleration module, and a multimodal emotion coprocessor), it achieves end-to-end localized processing of core AI tasks (perception, retrieval, reasoning, and sentiment analysis), eliminating network dependence and transmission latency, and ensuring real-time teaching interaction. The modal emotion coprocessor can understand students' complex emotions and intentions in real time, upgrading AI interaction from mechanical responses to a more human-like, intuitive interaction, significantly enhancing the sense of companionship and adaptability in teaching. The RAG retrieval acceleration module retrieves text fragments related to multimodal input data from the local educational knowledge base to obtain enhanced contextual prompts, enabling the neural network processor to generate natural language responses to the multimodal input data. This allows for accurate, in-depth, and contextualized knowledge question-and-answer sessions and explanations even in offline environments, overcoming the limitations of traditional educational hardware's rigid knowledge structure. This application provides real-time, secure, and human-like personalized AI tutoring and emotional interaction for teaching scenarios in completely offline or weak network environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of the structure of an artificial intelligence information education system based on a RISC-V processor architecture, provided for an embodiment of this application; Figure 2 A schematic diagram illustrating the parallel multimodal sentiment analysis and feature extraction process provided in this application embodiment; Figure 3 A schematic diagram illustrating the process of making intelligent scheduling decisions based on task characteristics and system status, provided in an embodiment of this application. Figure 4 A schematic diagram illustrating the complete process of performing local knowledge retrieval and context enhancement provided in this application embodiment; Figure 5 This is a schematic diagram illustrating the core process of NPU performing large language model generation calculations, as provided in the embodiments of this application.

[0018] Figure reference numerals: Central Processing Unit—1, AI Accelerated Neural Network Processing Unit—2, Graphics Processing Module—3, Low-Speed ​​Peripheral Communication Interface—4, Basic Storage Controller—5, Audio DSP and its External Interface—6, General-Purpose High-Speed ​​Peripheral Interface—7, Encryption Core—8, On-Chip Interconnect Bus—9. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] Current information and education devices often employ general-purpose mobile chips, such as smartphone SoCs, whose architecture is not optimized for lightweight AI models in education. This results in high power consumption and short battery life during local inference, making cost-effectiveness difficult to achieve. Cloud-based collaborative solutions, which upload core AI tasks for processing, heavily rely on the network, suffer from high latency, and pose privacy risks during transmission of sensitive data. Dedicated chips with limited functionality only support voice wake-up or simple OCR, lacking multimodal fusion and affective computing capabilities. They cannot understand complex contexts and student emotions, resulting in mechanical and rigid interactions that hinder personalized learning. Furthermore, the intelligent knowledge services of existing devices are generally "fixed," with their preset knowledge bases lacking context-based dynamic retrieval and generation capabilities, making it impossible to provide contextualized answers based on specific courses and student history.

[0022] To address the aforementioned issues, this application provides a domestically developed, low-power, highly secure, and highly interactive dedicated AI education system and method to achieve real-time multimodal perception, personalized knowledge services, and anthropomorphic emotional interaction in offline environments.

[0023] In one exemplary embodiment, such as Figure 1 As shown, an artificial intelligence information education system based on the RISC-V (Reduced Instruction Set Computer-Five) processor architecture is provided, including: a central processing unit (CPU) 1 and an AI-accelerated neural network processing unit 2; the central processing unit 1 is connected to the AI-accelerated neural network processing unit 2.

[0024] The central processing unit 1 integrates multiple RISC-V CPU cores; the AI-accelerated neural network processing unit 2 includes: a neural network processing unit (NPU), a RAG (Retrieval-Augmented Generation) retrieval acceleration module, and a multimodal emotion coprocessor; the neural network processing unit has a built-in large language model (LLM) inference engine; multimodal input data is used to input the central processing unit 1 and the AI-accelerated neural network processing unit 2; the multimodal input data includes: interactive tasks issued by the user in different ways.

[0025] The multimodal emotion coprocessor is used to extract and fuse features from the multimodal input data to obtain a fused feature vector, and to perform multi-label emotion classification and emotion intensity regression based on the fused features to obtain the user's emotion analysis results.

[0026] The central processing unit 1 is used to perform computation graph analysis on the multimodal input data. Based on the task characteristics obtained from the analysis, combined with the historical performance database and the set scheduling rules, it allocates different AI task subgraphs in the multimodal input data to the corresponding computing units. The computing units are RISC-V CPU cores and / or neural network processors.

[0027] The neural network processor is used to convert the text in the multimodal input data into a query vector for an AI task subgraph that requires knowledge retrieval, and then send the query vector to the RAG retrieval acceleration module through the central processing unit 1.

[0028] The RAG retrieval acceleration module is used to retrieve text fragments related to the multimodal input data from the educational knowledge base based on the query vector.

[0029] The central processing unit 1 is further configured to combine the text fragment with the text in the multimodal input data to obtain an enhanced prompt context, and send the enhanced prompt context to the neural network processor.

[0030] The neural network processor is also used to generate a natural language response to the multimodal input data based on the enhanced cue context and the user's sentiment analysis results, using the large language model inference engine.

[0031] In another exemplary embodiment of this application, the central processing unit 1 is a quad-core 64-bit RISC-V CPU cluster, serving as the system's control and general-purpose computing center. Each CPU core is equipped with a 32KB L1 instruction cache and a data cache, and the four cores share a 4MB L2 cache. The four cores can perform heterogeneous division of labor, respectively undertaking tasks such as system management and scheduling, AI computing assistance, I / O control, and performance monitoring. In addition, the central processing unit 1 also includes a 32KB BootROM for storing and executing the chip boot firmware.

[0032] In another exemplary embodiment of this application, the AI ​​accelerated neural network processing unit 2 is a dedicated hardware accelerator designed specifically for AI tasks in educational scenarios. It is the core computing unit of the system, and its three main sub-functional units are as follows.

[0033] Reconfigurable Neural Processor (rNPU): Adopting a "static operator hardware + dynamic configuration management" architecture, it integrates a mixed-precision systolic array, a nonlinear function pipeline, and a zero-overhead data reconstruction unit, providing high-efficiency inference for models such as Transformer.

[0034] RAG retrieval acceleration module: As an independent hardware IP (Intellectual Property), it integrates vector similarity calculation circuit, on-chip index cache and hardware sorting network, specifically to accelerate the vector retrieval process of local knowledge base.

[0035] Multimodal Emotion Coprocessor: Integrates text, visual (and optional speech) feature extraction and fusion hardware to achieve low-power emotion recognition and analysis.

[0036] In another exemplary embodiment of this application, such as Figure 1 As shown, the AI ​​information education system based on the RISC-V processor architecture further includes: a graphics processing module 3; the graphics processing module 3 is used to display the user's sentiment analysis results and the natural language response.

[0037] The graphics processing module 3 mainly includes a graphics processing unit (GPU) and its image input / output interfaces. Specifically, the graphics processing module 3 includes a GPU core, a video memory (VRAM) controller, and interfaces such as MIPI-CSI (MIPI Camera Serial Interface), DVP (Digital Video Port), MIPI-DSI (MIPI Display Serial Interface), and HDMI (High-Definition Multimedia Interface). This graphics processing module 3 is responsible for accelerating the rendering and display output of the graphical user interface (GUI), and assisting in the parallel computing required for visual AI tasks. Image data is input through the CSI or DVP interface, and after processing, can be output through the DSI or HDMI interface.

[0038] In another exemplary embodiment of this application, such as Figure 1 As shown, the artificial intelligence information education system based on the RISC-V processor architecture further includes: a peripheral interface; the peripheral interface is used to collect the multimodal input data.

[0039] The peripheral interfaces include: a low-speed peripheral communication interface 4, an audio DSP and its external interface 6, and a general-purpose high-speed peripheral interface 7. The low-speed peripheral communication interface 4 integrates general-purpose embedded interfaces such as UART (Universal Asynchronous Receiver / Transmitter), SPI (Serial Peripheral Interface), I2C (Inter-Integrated Circuit), PWM (Pulse Width Modulation), GPIO (General-Purpose Input / Output), Timer, WDT (Watchdog Timer), and CAN (Controller Area Network), used to connect to the various low-speed external devices such as sensors, actuators, and control boards in the teaching equipment. The audio DSP (Digital Signal Processor) and its external interface 6 include a digital signal processor (DSP) and corresponding audio codec interfaces, responsible for acquiring, denoising, encoding, decoding, and synthesizing voice signals during interactive teaching, and can drive external speakers or headphones. The Universal High-Speed ​​Peripheral Interface 7 primarily integrates a PCIe (Peripheral Component Interconnect Express) interface, which is used to expand the connection of high-speed external devices such as Gigabit Ethernet cards and NVMe (Non-Volatile Memory Express) solid-state drives to enhance the system's network and storage capabilities.

[0040] In another exemplary embodiment of this application, such as Figure 1 As shown, the AI-based information education system based on the RISC-V processor architecture also includes a basic storage controller 5. The basic storage controller 5 provides core storage control functions for the system and comprises the following components.

[0041] DDR (Double Data Rate) controller: Used to connect to external high-capacity DDR memory, serving as the main working memory for the system and AI models. QSPI (Quad Serial Peripheral Interface) flash memory controller: Used to connect to external Nor Flash memory to store boot code and firmware. eMMC (embedded MultiMediaCard) controller: Supports the eMMC 5.0 protocol and is used to connect to high-capacity, erasable and rewritable storage chips to store the operating system, applications, and local educational knowledge base data.

[0042] In another exemplary embodiment of this application, such as Figure 1As shown, the AI ​​information education system based on the RISC-V processor architecture also includes an encryption core 8. The encryption core 8 is an independent security subsystem that integrates hardware encryption engines such as AES (Advanced Encryption Standard), DES (Data Encryption Standard), HASH, and PKA (Public Key Algorithm) to encrypt and verify the integrity of chip firmware, AI model parameters, user data, and communication processes, ensuring system security and reliability.

[0043] In another exemplary embodiment of this application, such as Figure 1 As shown, the AI ​​information education system based on the RISC-V processor architecture further includes: an on-chip interconnect bus 9; the system architecture uses the NOC (Network on Chip) / AXI (Advanced eXtensible Interface) on-chip interconnect bus as the central interconnect skeleton, the central processing unit 1, the AI ​​accelerated neural network processing unit 2, the graphics processing module 3, the peripheral interface, the basic storage controller 5, and the encryption core 8 are all directly connected to the on-chip interconnect bus 9, and high-speed, orderly data communication and control signal exchange are performed through the on-chip interconnect bus 9.

[0044] The on-chip interconnect bus 9 is a NOC / AXI on-chip interconnect bus, which is the high-speed data communication backbone within the system. It adopts the Network On-Chip (NOC) architecture and the AXI protocol, and is responsible for coordinating all data transmission, memory access and interrupt requests between the central processing unit 1, AI accelerated neural network processing unit 2, graphics processing module 3, peripheral interfaces, basic storage controller 5 and encryption core 8, to ensure low-latency and high-bandwidth concurrent communication. It is a key infrastructure for realizing efficient collaboration of heterogeneous computing.

[0045] In another exemplary embodiment of this application, the multimodal input data specifically includes interactive tasks issued by users (such as students) in the form of voice, images, and text.

[0046] The multimodal emotion coprocessor is specifically used for: The text in the multimodal input data is subjected to sentiment dictionary matching and semantic sentiment encoding to obtain text features. The images in the multimodal input data are subjected to face detection, pixel alignment and facial motion extraction to obtain image features. The speech in the multimodal input data is subjected to prosodic feature extraction to obtain speech features.

[0047] A lightweight controller is used to evaluate the quality of the text features, image features, and speech features, and a fusion strategy is determined based on the evaluation results. The fusion strategy is then used to fuse the text features, image features, and speech features to obtain a fused feature vector.

[0048] The fused feature vector is input into a multilayer perceptron for multi-label sentiment classification and sentiment intensity regression to obtain the user's sentiment analysis results.

[0049] The multimodal emotion coprocessor in this embodiment simultaneously performs: emotion dictionary matching and lightweight semantic encoding of text; face detection, alignment and micro-expression action unit extraction of images; prosodic feature analysis of speech, and finally outputs the user's comprehensive emotional state (such as confusion, excitement) and signal quality report.

[0050] In another exemplary embodiment of this application, in performing computational graph analysis on the multimodal input data, and allocating different AI task subgraphs in the multimodal input data to corresponding computing units based on the analyzed task characteristics, combined with a historical performance database and set scheduling rules, the central processing unit 1 is specifically used for: The multimodal input data is split into multiple AI task subgraphs.

[0051] A dynamic perception scheduler is used to perform three-dimensional feature analysis on the AI ​​task subgraph to obtain task features; the task features include: operator type, data size and computational precision.

[0052] Based on the task characteristics and historical performance database, predict the overhead data for the execution of the AI ​​task subgraph on the neural network processor and each of the RISC-V CPU cores; the overhead data includes: estimated time and estimated energy consumption.

[0053] The AI ​​task subgraphs are dynamically scheduled based on the overhead data and the set scheduling rules, and different AI task subgraphs are assigned to the corresponding computing units.

[0054] The scheduling rules include: if the regularity of the AI ​​task subgraph is greater than a set regularity value and the computational density is greater than a set density value, then the AI ​​task subgraph is assigned to the neural network processor; if the control complexity of the AI ​​task subgraph is greater than a set complexity value or the data volume is less than a set value, then the AI ​​task subgraph is assigned to multiple RISC-V CPU cores for parallel execution; if the AI ​​task subgraph is a hybrid task subgraph or the expected efficiency of a single hardware component is lower than a set efficiency value, then the AI ​​task subgraph is segmented and pipelined, with the regular computational subgraph assigned to the neural network processor and the dynamic control subgraph assigned to the RISC-V CPU cores.

[0055] In this embodiment, the dynamic perception scheduler in the central processing unit 1 is activated, performs computational graph analysis on the multimodal input data, combines real-time monitoring of CPU / NPU load, cache heat, and power consumption data, and decides to allocate different subgraphs of the task to the neural network processor or specific CPU cores for execution based on preset rules and historical performance models, and plans data flow paths. The central processing unit 1 realizes intelligent scheduling decisions based on task characteristics and system status.

[0056] In another exemplary embodiment of this application, the RAG retrieval acceleration module is specifically used for: Calculate the cosine similarity between the query vector and the vector corresponding to the text block in the educational knowledge base, and determine the text segments related to the multimodal input data from the set number of text blocks with the highest cosine similarity.

[0057] In this embodiment, the RAG retrieval acceleration module, in conjunction with the CPU 1 and the neural network processor, enables local knowledge retrieval and context enhancement. The general process is as follows: when a task requiring knowledge retrieval exists, the dynamic perceptual scheduler triggers the RAG process. First, the embedded model in the neural network processor converts the text in the multimodal input data into query vectors. Next, the CPU 1 submits the query vectors to the RAG retrieval acceleration module, which quickly retrieves the most relevant text fragments from the local educational knowledge base stored in the external eMMC. Finally, the CPU 1 combines these retrieval results with the student's question to construct an enhanced prompt context.

[0058] In another exemplary embodiment of this application, in generating a natural language response to the multimodal input data using the large language model inference engine based on the enhanced cue context and the user's sentiment analysis results, the neural network processor is specifically configured to: The enhanced prompt context is input into the large language model inference engine, and the generation style of the large language model inference engine is adjusted according to the user's sentiment analysis results to obtain a natural language response to the multimodal input data.

[0059] In this embodiment, during the text generation process of the large language model inference engine, the generation style is fine-tuned by incorporating the user's sentiment analysis results (for example, using more detailed and encouraging language when a student appears confused). After the neural network processor completes the probability distribution calculation of the answer text, the central processing unit 1 performs decoding and sampling to obtain the final natural language response.

[0060] The large language model inference engine includes: a Transformer model input layer, an offline compilation configuration module, a runtime adaptive scheduling module, a hardware reconfigurable control module, and a fixed operator hardware layer.

[0061] This embodiment presents an AI-powered information education system based on a RISC-V processor architecture, which is a heterogeneous multi-core system-on-a-chip (SoC). It integrates a quad-core RISC-V CPU, a dedicated neural network processor, a multimodal emotion recognition coprocessor, a local educational knowledge base and RAG retrieval acceleration module, a graphics processing module, and rich peripheral interfaces. Through a Transformer processor architecture combining static operator hardware and dynamic configuration management, it achieves high-efficiency AI inference. Based on an intelligent task scheduling algorithm, it achieves optimal load distribution between the CPU and NPU. Utilizing hardware-accelerated local knowledge retrieval and multimodal emotion fusion technology, it solves the problems of high power consumption, large response latency, network dependence, privacy leaks, and awkward interaction found in existing educational AI terminal devices. This application can provide real-time, secure, and human-like personalized AI tutoring and emotional interaction for primary and secondary school teaching scenarios in completely offline or weak network environments, providing core hardware support for building an autonomous and controllable next-generation intelligent education infrastructure.

[0062] The following section uses primary and secondary school students as an example to describe in detail the workflow of the AI ​​information education system based on the RISC-V processor architecture in the above embodiment when performing a complete intelligent education interaction task.

[0063] Step 1: Receive and parse the student's multimodal input data.

[0064] The system receives questions or interactions from students in the form of voice, images, and text through peripherals such as microphones, cameras, and touchscreens, obtaining multimodal input data. Specifically, the microphone is connected to an audio DSP; the camera is connected to a MIPI-CSI interface; and the touchscreen is connected to an I2C interface.

[0065] Step 2: Perform parallel multimodal sentiment analysis and feature extraction.

[0066] Based on the multimodal input data obtained in step one, it is fed in parallel into a multimodal emotion recognition coprocessor. This coprocessor simultaneously performs: emotion dictionary matching and lightweight semantic encoding of the text; face detection, alignment, and micro-expression action unit extraction of the image; and prosodic feature analysis of the speech. Finally, it outputs the student's comprehensive emotional state (e.g., confusion, excitement) and a signal quality report.

[0067] This step involves parallel multimodal sentiment analysis and feature extraction, as follows: Figure 2 As shown, specifically: 2.1: Parallel input preprocessing and feature extraction.

[0068] The multimodal emotion recognition coprocessor simultaneously receives text, image, and (optionally) speech input. It operates on three parallel hardware paths: emotion dictionary matching and lightweight BiGRU semantic encoding for text; face detection, keypoint alignment, and intensity regression of 17 core facial action units for images; and prosodic features such as fundamental frequency and energy for speech.

[0069] 2.2: Adaptively select fusion strategy based on signal quality assessment.

[0070] A lightweight controller evaluates the feature quality (such as face clarity and text integrity) extracted from each modality in step one. Based on the evaluation results, a fusion strategy is dynamically selected: if all modalities have high quality, early fusion (such as feature splicing) is used; if there is strong correlation between modalities, mid-term fusion (such as hardware-based cross-attention computation) is used; and if any modality has poor quality, late-term fusion (such as decision-layer weighting) is used.

[0071] 2.3: Perform cross-modal feature fusion and sentiment decision-making in the hardware fusion core.

[0072] Under the selected fusion strategy, the hardware fusion core integrates multimodal features. For example, in mid-stage fusion, text features are used as the query to focus on visual features, and the correlation weights between modalities are dynamically calculated. The fused features are fed into a hardware multilayer perceptron, which simultaneously performs multi-label sentiment classification (e.g., happy, confused) and sentiment intensity regression.

[0073] 2.4: Perform conflict detection and generate an interpretability report.

[0074] The decision engine calculates the consistency (such as KL divergence) between the initial decision results of different modalities. If the consistency exceeds a threshold, it is marked as a "hard conflict". The final output of the multimodal emotion recognition coprocessor includes: discrete emotion labels, emotion dimension coordinates, emotion change curves and inflection points, multimodal consistency report, and an interpretability report describing the conflict situation (if any).

[0075] 2.5: Output the sentiment analysis results for upper-level system decision-making.

[0076] The coprocessor submits the multi-level sentiment analysis results generated in step 2.4 to the main CPU via the bus. This result will serve as key context to guide subsequent answer generation style adjustments (step 5) or interaction strategy selection, and can trigger a system call to the more powerful NPU for deep inference when complex conflicts are detected.

[0077] Step 3: Make intelligent scheduling decisions based on task characteristics and system status.

[0078] Based on the question content parsed in step one and the emotional context analyzed in step two, the dynamic perception scheduler in the CPU is activated. The scheduler performs computational graph analysis on the AI ​​question-answering task, combining real-time monitoring of CPU / NPU load, cache heat, and power consumption data. According to preset rules and historical performance models, it decides to allocate different subgraphs of the task to the NPU or specific CPU cores for execution and plans the data flow path.

[0079] The process of making intelligent scheduling decisions based on task characteristics and system state is as follows: Figure 3 As shown, specifically: 3.1: Conduct task feature analysis and real-time system status awareness.

[0080] The scheduler first performs a three-dimensional feature analysis on the input AI task subgraph, including operator type (computation / memory access / control intensive), data size, and computational precision. Simultaneously, it reads the real-time load, cache hit rate, and power consumption status of each CPU core and NPU from the hardware performance monitoring unit.

[0081] 3.2: Combine historical performance databases to predict the overhead of different execution paths.

[0082] Based on the task characteristics analyzed in 3.1, the scheduler queries a pre-built historical performance database to predict the estimated time and energy consumption for the execution of the task subgraph on the NPU or individual CPU cores.

[0083] 3.3: Make dynamic scheduling decisions based on the rules and prediction results.

[0084] The scheduler executes a multi-condition decision tree: if the task is highly regular and computationally intensive, it is directly assigned to the NPU; if the task is complex to control or has a very small amount of data, it is assigned to a multi-core CPU for parallel execution; if the task is hybrid or single hardware is expected to be inefficient, the "task splitting and pipeline" analysis is triggered.

[0085] 3.4: Perform computational graph partitioning and pipeline planning for hybrid tasks.

[0086] For tasks identified as requiring splitting in section 3.3, the scheduler uses a graph cutting algorithm to find the optimal splitting point in the computation graph, separating the regular computational subgraph from the dynamic control subgraph. Subsequently, a producer-consumer pipeline is planned, where the CPU performs preprocessing and dynamic branching, while the NPU performs core computations, with both exchanging data via zero-copy shared memory.

[0087] 3.5: Collaborate on task execution and collect feedback data to optimize the scheduling model.

[0088] The scheduler distributes tasks (or subtasks) to the corresponding computing units (NPU or CPU cores) for execution. After a task is completed, the system collects data such as actual execution time and energy consumption, feeds it back to update the historical performance database and predictive models, and enables the scheduling strategy to continuously optimize itself.

[0089] Step 4: Perform local knowledge retrieval and context enhancement.

[0090] For the task portion in step three that requires knowledge retrieval, the scheduler triggers the RAG process. First, the embedded model in the NPU converts the student question text into a query vector. Next, the CPU submits the query vector to the RAG retrieval acceleration module, which quickly retrieves the most relevant text fragments from the local educational knowledge base stored in the off-chip eMMC. Finally, the CPU combines these retrieval results with the student question to construct an enhanced prompt context.

[0091] The complete process of performing local knowledge retrieval and context enhancement is as follows: Figure 4 As shown, specifically: 4.1: Embedded vectorization of user queries.

[0092] The natural language question posed by the user is first preprocessed by the CPU, such as by word segmentation, and then submitted to a lightweight embedding model (such as BGE-lite) embedded in the NPU. The NPU executes the computation of this model, converting the text query into a high-dimensional semantic vector (e.g., a 768-dimensional FP16 vector).

[0093] 4.2: Submit the query vector to the hardware retrieval acceleration module for similarity search.

[0094] The CPU sends the query vector address generated in section 4.1, along with the vector partition information to be searched, to a dedicated hardware retrieval acceleration module via a configuration register. This module then reads the vector index data (built based on the HNSW graph) and the corresponding vector data from the local knowledge base in DDR memory via DMA.

[0095] 4.3: Parallel execution of approximate nearest neighbor retrieval within the hardware module.

[0096] The parallel distance calculation unit within the retrieval acceleration module simultaneously calculates the cosine similarity between the query vector and a batch of vectors in the database (equivalent to the inner product after L2 normalization). The hardware graph traversal engine navigates quickly based on the HNSW index, and the on-chip priority queue maintains and updates the Top-K candidate results with the highest similarity in real time.

[0097] 4.4: Building an enhanced prompt context.

[0098] The retrieval acceleration module writes the IDs and similarity scores of the Top-K relevant text blocks back to shared memory and notifies the CPU of the interrupt. The CPU retrieves the corresponding original text blocks from the knowledge base based on these IDs, sorts them by similarity, and intelligently truncates them to meet length limits. Finally, it combines this background information with the original question to form a structured prompt word template.

[0099] 4.5: Submit the enhanced context to the large language model to generate the answer.

[0100] The CPU submits the complete prompt context, constructed in section 4.4, to the large language model inference engine in the NPU, initiating the text generation process. At this point, the RAG retrieval enhancement process concludes, and the generation phase begins.

[0101] Step 5: Generate personalized and emotionally appropriate responses.

[0102] The enhanced contextual hints obtained in step four are sent to the large language model inference engine in the NPU for text generation. During the generation process, the generation style can be fine-tuned by incorporating the student's emotional state output in step two (for example, using more detailed and encouraging language when the student appears confused). After the NPU completes the probability distribution calculation of the answer text, the CPU performs decoding and sampling to obtain the final natural language response.

[0103] In generating personalized and emotion-adapted responses, the core process of NPU performing large language model generation computation is as follows: Figure 5 As shown, this process is completed under the joint guidance of enhanced prompts and the student's emotional state, as detailed below.

[0104] 5.1: Perform offline compilation and static configuration before model deployment.

[0105] When the AI ​​model is loaded onto the chip, the offline compilation and configuration module runs first. This module receives the structured parameters of the Transformer model (such as the number of layers, dimensions, and quantization precision), analyzes the data dependencies in the computation graph, and generates an optimal hardware configuration scheme, including the computation graph partitioning strategy, data tile size, memory layout planning, and precision mode configuration of the computation units. This stage lays the hardware foundation for subsequent dynamic scheduling and sentiment adaptation computation.

[0106] 5.2: Based on the static configuration generated in 5.1, dynamic adaptive scheduling is performed at runtime.

[0107] During model execution, the runtime adaptive scheduling module continuously monitors the actual hardware status, including the data throughput of each computing unit buffer, the chip's real-time power consumption and temperature, and the sparsity exhibited in the input data. Based on this, the module introduces the structural features of the enhanced prompt context and the student's emotional state as scheduling criteria to determine whether the pre-set execution plan needs to be fine-tuned, prioritizing the computational resources and latency requirements of the emotion adaptation generation task.

[0108] 5.3: Implement reconfigurable hardware control to dynamically adapt to the optimal computing mode.

[0109] Based on the decisions made by the scheduling module in section 5.2, the hardware reconfigurable control module sends real-time configuration signals to the underlying hardware circuitry. Its operations include: switching the core computing systolic array between modes such as "single large matrix multiplication," "multiple small matrix parallelism," or "sparse computing"; or reconfiguring data paths to adapt to special operators such as LayerNorm. In emotion-adaptation scenarios, the system can prioritize low-latency or high-throughput computing modes based on the student's current emotional state to match interactive response requirements.

[0110] 5.4: Based on the solidified operator hardware layer, it efficiently executes computational tasks.

[0111] In the hardware mode defined in 5.3, the fixed high-performance operator units begin to operate. The linear computation unit (pulsating array) performs dense matrix operations with static weights and pulsating data; the zero-overhead data reconstruction unit instantaneously completes tensor deformation and transpose through memory view transformation, eliminating data transport overhead; the nonlinear and reduction pipeline processes functions such as Softmax and GELU in parallel. This stage actually completes the fusion expression of enhanced prompt context and student emotional features in the model's forward computation.

[0112] 5.5: Collaborate to complete the forward propagation of the entire Transformer layer.

[0113] The format of the "model input data" (precision, chunk size) serves as the starting point and constraint for the entire process, driving the configuration and execution from 5.1 to 5.4. During this process, enhanced contextual prompts provide the foundation for content-level generation, while the student's emotional state acts as an implicit condition for style and tone control, jointly influencing the intermediate activation and final output distribution of each computational unit. Under the macro-level framework of "offline configuration" and the on-site command of "runtime scheduling," the entire system, with each hardware unit working collaboratively, ultimately completes the model inference computation with integrated emotional adaptation efficiently and adaptively.

[0114] Step 6: Output multimodal interactive response.

[0115] The natural language response generated in step five is converted into speech using a text-to-speech (TTS) module and / or displayed through a graphical interface. Simultaneously, based on the sentiment analysis results from step two, the system can drive the virtual avatar to display corresponding facial expressions or actions, completing a human-like, emotionally-adaptive closed-loop teaching interaction.

[0116] The artificial intelligence information education system based on the RISC-V processor architecture provided in this embodiment has the following advantages.

[0117] High performance and high energy efficiency: Through a reconfigurable NPU architecture optimized for lightweight AI models in educational scenarios, intelligent task scheduling, and hardware-level retrieval and affective computing acceleration, it provides AI inference performance far exceeding that of general-purpose chips per unit power consumption, meeting the requirements of educational terminals for long battery life and low cost.

[0118] Strong real-time performance and high security: The system achieves end-to-end localized processing of core AI tasks (perception, retrieval, reasoning, and sentiment analysis), eliminating network dependence and transmission latency, and ensuring the real-time nature of teaching interactions. All sensitive student data is processed on the device itself, complying with regulations such as the Personal Information Protection Law, fundamentally eliminating the risk of privacy leaks.

[0119] Natural multimodal interaction capabilities: The integrated multimodal emotion coprocessor can understand students' complex emotions and intentions in real time, upgrading AI interaction from mechanical responses to human-like interaction that "reads between the lines," significantly improving the sense of companionship and adaptability in teaching.

[0120] Powerful offline knowledge service capabilities: The hardware-accelerated local knowledge base and RAG module enable the device to conduct accurate, in-depth, and contextualized knowledge Q&A and explanations even in environments without a network, breaking through the limitations of traditional educational hardware's fixed knowledge.

[0121] Domestic production and independent control: Based on the RISC-V processor architecture with an open instruction set, it is independently designed and realizes full-stack control from the chip bottom layer to the application algorithm, laying the hardware foundation for building a safe and reliable new generation of educational information infrastructure.

[0122] Based on the same inventive concept, embodiments of this application also provide an artificial intelligence information education method based on a RISC-V processor architecture, including: (1) Acquire multimodal input data; the multimodal input data includes: interactive tasks issued by the user in different ways.

[0123] (2) Perform feature extraction and feature fusion on the multimodal input data to obtain a fused feature vector, and perform multi-label sentiment classification and sentiment intensity regression based on the fused features to obtain the user's sentiment analysis results.

[0124] (3) Perform computation graph analysis on the multimodal input data, and based on the task characteristics obtained from the analysis, combined with the historical performance database and the set scheduling rules, assign different AI task subgraphs in the multimodal input data to the corresponding computing units; the computing units are RISC-V CPU cores and / or neural network processors.

[0125] (4) For AI task subgraphs that require knowledge retrieval, the text in the multimodal input data is converted into a query vector. Based on the query vector, text fragments related to the multimodal input data are retrieved from the educational knowledge base. The text fragments are then combined with the text in the multimodal input data to obtain an enhanced prompt context.

[0126] (5) Based on the enhanced prompt context and the user's sentiment analysis results, the large language model inference engine is used to generate a natural language response to the multimodal input data.

[0127] As an optional implementation method, step (2) specifically includes: The text in the multimodal input data is subjected to sentiment dictionary matching and semantic sentiment encoding to obtain text features. The images in the multimodal input data are subjected to face detection, pixel alignment and facial motion extraction to obtain image features. The speech in the multimodal input data is subjected to prosodic feature extraction to obtain speech features.

[0128] A lightweight controller is used to evaluate the quality of the text features, image features, and speech features, and a fusion strategy is determined based on the evaluation results. The fusion strategy is then used to fuse the text features, image features, and speech features to obtain a fused feature vector.

[0129] The fused feature vector is input into a multilayer perceptron for multi-label sentiment classification and sentiment intensity regression to obtain the user's sentiment analysis results.

[0130] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An artificial intelligence information education system based on RISC-V processor architecture, characterized in that, The AI-based information education system based on the RISC-V processor architecture includes: a central processing unit and an AI-accelerated neural network processing unit; the central processing unit is connected to the AI-accelerated neural network processing unit. The central processing unit integrates multiple RISC-V CPU cores; the AI-accelerated neural network processing unit includes: a neural network processor, a RAG retrieval acceleration module, and a multimodal emotion coprocessor; the neural network processor has a built-in large language model inference engine; multimodal input data is used to input the central processing unit and the AI-accelerated neural network processing unit; the multimodal input data includes: interactive tasks issued by the user in different ways; The multimodal emotion coprocessor is used to extract and fuse features from the multimodal input data to obtain a fused feature vector, and to perform multi-label emotion classification and emotion intensity regression based on the fused features to obtain the user's emotion analysis results. The central processing unit is used to perform computation graph analysis on the multimodal input data. Based on the task characteristics obtained from the analysis, combined with the historical performance database and the set scheduling rules, it allocates different AI task subgraphs in the multimodal input data to the corresponding computing units. The computing units are RISC-V CPU cores and / or neural network processors. The neural network processor is used to convert the text in the multimodal input data into a query vector for an AI task subgraph that requires knowledge retrieval, and send the query vector to the RAG retrieval acceleration module through the central processing unit; The RAG retrieval acceleration module is used to retrieve text fragments related to the multimodal input data from the educational knowledge base based on the query vector; The central processing unit is also used to combine the text fragment with the text in the multimodal input data to obtain an enhanced prompt context, and send the enhanced prompt context to the neural network processor; The neural network processor is also used to generate a natural language response to the multimodal input data based on the enhanced cue context and the user's sentiment analysis results, using the large language model inference engine.

2. The artificial intelligence information education system based on RISC-V processor architecture according to claim 1, characterized in that, The multimodal input data specifically includes: interactive tasks issued by the user in the form of voice, images, and text; The multimodal emotion coprocessor is specifically used for: The text in the multimodal input data is subjected to sentiment dictionary matching and semantic sentiment encoding to obtain text features; the images in the multimodal input data are subjected to face detection, pixel alignment and facial action extraction to obtain image features; and the speech in the multimodal input data is subjected to prosodic feature extraction to obtain speech features. A lightweight controller is used to evaluate the quality of the text features, image features, and speech features, and a fusion strategy is determined based on the evaluation results. The fusion strategy is then used to fuse the text features, image features, and speech features to obtain a fused feature vector. The fused feature vector is input into a multilayer perceptron for multi-label sentiment classification and sentiment intensity regression to obtain the user's sentiment analysis results.

3. The artificial intelligence information education system based on the RISC-V processor architecture according to claim 1, characterized in that, In performing computational graph analysis on the multimodal input data, and based on the analyzed task characteristics, combined with a historical performance database and set scheduling rules, allocating different AI task subgraphs in the multimodal input data to corresponding computing units, the central processing unit is specifically used for: The multimodal input data is split into multiple AI task subgraphs; A dynamic perception scheduler is used to perform three-dimensional feature analysis on the AI ​​task subgraph to obtain task features; the task features include: operator type, data size and computational precision. Based on the task characteristics and historical performance database, predict the overhead data for the execution of the AI ​​task subgraph on the neural network processor and each of the RISC-V CPU cores; the overhead data includes: estimated time and estimated energy consumption; The AI ​​task subgraphs are dynamically scheduled according to the overhead data and the set scheduling rules, and different AI task subgraphs are assigned to the corresponding computing units. The scheduling rules include: if the regularity of the AI ​​task subgraph is greater than a set regularity value and the computational density is greater than a set density value, then the AI ​​task subgraph is assigned to the neural network processor; if the control complexity of the AI ​​task subgraph is greater than a set complexity value or the data volume is less than a set value, then the AI ​​task subgraph is assigned to multiple RISC-V CPU cores for parallel execution; if the AI ​​task subgraph is a hybrid task subgraph or the expected efficiency of a single hardware component is lower than a set efficiency value, then the AI ​​task subgraph is segmented and pipelined, with the regular computational subgraph assigned to the neural network processor and the dynamic control subgraph assigned to the RISC-V CPU cores.

4. The artificial intelligence information education system based on RISC-V processor architecture according to claim 1, characterized in that, The RAG retrieval acceleration module is specifically used for: Calculate the cosine similarity between the query vector and the vector corresponding to the text block in the educational knowledge base, and determine the text segments related to the multimodal input data from the set number of text blocks with the highest cosine similarity.

5. The artificial intelligence information education system based on RISC-V processor architecture according to claim 1, characterized in that, In generating a natural language response to the multimodal input data using the large language model inference engine based on the enhanced cue context and the user's sentiment analysis results, the neural network processor is specifically configured to: The enhanced prompt context is input into the large language model inference engine, and the generation style of the large language model inference engine is adjusted according to the user's sentiment analysis results to obtain a natural language response to the multimodal input data.

6. The artificial intelligence information education system based on RISC-V processor architecture according to claim 1, characterized in that, The AI-based information education system based on the RISC-V processor architecture further includes a graphics processing module; the graphics processing module is used to display the user's sentiment analysis results and the natural language response.

7. The artificial intelligence information education system based on RISC-V processor architecture according to claim 1, characterized in that, The AI ​​information education system based on the RISC-V processor architecture further includes: a peripheral interface; the peripheral interface is used to collect the multimodal input data.

8. The artificial intelligence information education system based on RISC-V processor architecture according to claim 1, characterized in that, The AI ​​information education system based on the RISC-V processor architecture further includes: an on-chip interconnect bus; the central processing unit and the AI ​​accelerated neural network processing unit are connected through the on-chip interconnect bus.

9. An artificial intelligence-based information education method based on RISC-V processor architecture, characterized in that, The AI ​​information education method based on the RISC-V processor architecture is used in any one of the AI ​​information education systems based on the RISC-V processor architecture according to claims 1-8, wherein the AI ​​information education method based on the RISC-V processor architecture includes: Acquire multimodal input data; the multimodal input data includes: interactive tasks issued by the user in different ways; Feature extraction and feature fusion are performed on the multimodal input data to obtain a fused feature vector. Multi-label sentiment classification and sentiment intensity regression are then performed based on the fused features to obtain the user's sentiment analysis results. The multimodal input data is subjected to computational graph analysis. Based on the task characteristics obtained from the analysis, combined with the historical performance database and the set scheduling rules, different AI task subgraphs in the multimodal input data are assigned to corresponding computing units. The computing units are RISC-V CPU cores and / or neural network processors. For AI task subgraphs that require knowledge retrieval, the text in the multimodal input data is converted into a query vector. Based on the query vector, text fragments related to the multimodal input data are retrieved from the educational knowledge base. The text fragments are then combined with the text in the multimodal input data to obtain an enhanced prompt context. Based on the enhanced prompt context and the user's sentiment analysis results, the large language model inference engine is used to generate a natural language response to the multimodal input data.

10. The artificial intelligence information education method based on RISC-V processor architecture according to claim 9, characterized in that, Feature extraction and feature fusion are performed on the multimodal input data to obtain a fused feature vector. Multi-label sentiment classification and sentiment intensity regression are then performed based on the fused features to obtain the user's sentiment analysis results, specifically including: The text in the multimodal input data is subjected to sentiment dictionary matching and semantic sentiment encoding to obtain text features; the images in the multimodal input data are subjected to face detection, pixel alignment and facial action extraction to obtain image features; and the speech in the multimodal input data is subjected to prosodic feature extraction to obtain speech features. A lightweight controller is used to evaluate the quality of the text features, image features, and speech features, and a fusion strategy is determined based on the evaluation results. The fusion strategy is then used to fuse the text features, image features, and speech features to obtain a fused feature vector. The fused feature vector is input into a multilayer perceptron for multi-label sentiment classification and sentiment intensity regression to obtain the user's sentiment analysis results.

Images