Method and system for constructing after-class service AI intelligent assistant based on neural network

Abstract

The invention discloses an after-class service AI intelligent assistant construction method and system based on a neural network, and relates to the crossing field of artificial intelligence and intelligent education. The AI intelligent assistant obtains to-be-answered information or to-be-corrected information in response to question input of a user, and answers meeting preset conditions are screened from a preset sample set through a trained multi-layer neural network model to serve as standard answers; predicting an answer corresponding to the to-be-answered information from a preset sample set through the neural network model trained by the sample set, comparing the consistency between the answer in the to-be-corrected information and a standard answer, scoring each answer score point, and calculating a total score of the question; data such as question and answer contents and homework correction conditions of the students are analyzed regularly, learning reports of the users are generated regularly, and subsequent learning promotion schemes are recommended to the users intelligently. The question and answer accuracy can be improved, and the learning efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the intersection of artificial intelligence and smart education, and in particular to a method and system for constructing an after-school service AI intelligent assistant based on a neural network. Background Art

[0002] Intelligent question answering is a system based on artificial intelligence technology that can answer natural language questions raised by users. It combines natural language processing, knowledge graphs, machine learning and other technologies to achieve accurate and efficient question answering. However, in actual use, for some questions raised by users, there will be a deviation between the understanding of artificial intelligence and the understanding of users, resulting in artificial intelligence being unable to give the answers users need. With the continuous intelligence of the education field, using computer technology to replace part of manual labor for teaching and managing students has huge advantages. Summary of the invention

[0003] In order to solve the technical problems existing in the prior art, the present application provides a method and system for constructing an after-school service AI intelligent assistant based on a neural network, which can obtain more accurate and timely question and answer results.

[0004] The present application provides a method for constructing an after-school service AI intelligent assistant based on a neural network, the method comprising the following steps:

[0005] S1: Acquire interactive information uploaded by a user, wherein the interactive information is information about questions to be answered uploaded by the user or information about questions and answers to be corrected;

[0006] S2: Preprocessing the user interaction information, including image processing, text processing and voice processing. The image processing unit is used to perform image recognition based on the image data to obtain the text content of the question to be answered or to be corrected. The voice processing unit is used to respond to the user's voice input, set the topic according to the scene, and generate a response according to the context and the big data model to obtain the text content of the question to be answered or to be corrected. The text processing unit is used to convert the text into a word vector sequence, convert the information to be answered uploaded by the user into a first word vector sequence, and convert the information to be corrected into a second word vector sequence.

[0007] S3: construct a multi-layer neural network model, wherein the multi-layer neural network model includes an input layer, a plurality of cascaded convolutional layers, an encoder, a normalization layer, and a fully connected output layer arranged in sequence from front to back;

[0008] S4: Obtain a sample set, and train the multi-layer neural network model based on the sample set, wherein the sample set is questions and corresponding answers obtained from a corpus database and historical question and answer information, wherein the historical question and answer information includes historical questions and historical answers corresponding to the historical questions;

[0009] The input layer is responsible for receiving the original data samples and converting them into a one-dimensional format for processing by subsequent network layers. The network structure contains three consecutive convolutional layers. The first convolutional layer is equipped with 32 3x3 convolution kernels, and the ReLU activation function is applied with a step size of 1. The second convolutional layer that follows has 64 3x3 convolution kernels, and the ReLU activation function is also used, with a step size of 1. The third convolutional layer is equipped with 128 3x3 convolution kernels, and the ReLU activation function is still used, with a step size of 1.

[0010] The encoder consists of four downsampling modules, each of which consists of two 3x3 convolutional layers, a ReLU activation function, and a 2x2 pooling layer. The purpose is to reduce the dimensionality of the input data and convert it into one-dimensional data.

[0011] After the normalization layer, the one-dimensional data is feature extracted and normalized, and then these features are activated by the LeakyRelu activation function;

[0012] The fully connected output layer receives the output of the third convolutional layer, converts it into a one-dimensional vector, and generates a discriminant value for classification or regression through the fully connected layer;

[0013] S5: input the word vector sequence into the multi-layer neural network model to obtain the multi-layer neural network model to select answers that meet the preset conditions from the preset sample set as the standard answer;

[0014] S6: Evaluate the uploaded information to be corrected, compare the students' responses with the preset standard answers, and make quantitative assessments for each scoring point. When it is found that there is a deviation between the user's answer and the standard answer, provide the problem-solving logic of the standard answer and record it in the wrong question collection. By regularly analyzing the user's question and answer performance and homework correction records, compile a learning progress report based on the time period, and make personalized learning improvement suggestions accordingly.

[0015] The present application also provides a system for constructing an after-school service AI intelligent assistant based on a neural network, which is used to execute the method for constructing an after-school service AI intelligent assistant based on a neural network. The system includes the following modules: a first acquisition unit, which is used to acquire interactive information uploaded by a user, wherein the interactive information is information about questions to be answered uploaded by the user or information to be corrected containing questions and answers;

[0016] A processing unit pre-processes the user interaction information, including image processing, text processing and voice processing. The image processing unit is used to perform image recognition based on the image data to obtain the text content of the question to be answered or to be corrected. The voice processing unit is used to respond to the user's voice input, and the AI ​​intelligent assistant interacts with the user. At the same time, various chat topics are set according to various chat scenarios, and responses are generated according to the context and big data model to obtain the text content of the question to be answered or to be corrected; the text processing unit is used to convert the text into a word vector sequence, convert the information to be answered uploaded by the user into a first word vector sequence, and convert the information to be corrected into a second word vector sequence;

[0017] A first construction unit constructs a multi-layer neural network model, wherein the multi-layer neural network model includes an input layer, a plurality of cascaded convolutional layers, an encoder, a normalization layer, and a fully connected output layer arranged in sequence from front to back;

[0018] A second acquisition unit acquires a sample set, and trains the multi-layer neural network model based on the sample set, wherein the sample set is questions and corresponding answers acquired from a corpus database and historical question-and-answer information, wherein the historical question-and-answer information includes historical questions and historical answers corresponding to the historical questions;

[0019] Determine the subunit, input the word vector sequence into the multi-layer neural network model, so as to obtain the answer that meets the preset conditions selected by the multi-layer neural network model from the preset sample set as the standard answer;

[0020] The feedback sub-unit evaluates the uploaded information to be marked, compares the students' responses with the preset standard answers, and makes quantitative assessments on each scoring point. When a deviation is found between the user's answer and the standard answer, the unit provides the problem-solving logic of the standard answer and records it in the wrong question collection. By regularly analyzing the user's question and answer performance and homework marking records, it compiles a learning progress report based on the time period and makes personalized learning improvement suggestions accordingly.

[0021] Beneficial effects:

[0022] The construction method of the after-school service AI smart assistant based on neural network proposed in this application can quickly process and respond to a large number of queries and provide instant feedback. At the same time, the feedback learning mechanism of the model is continuously optimized and adjusted to ensure the accuracy and efficiency of the AI ​​smart assistant's answers. The use of a neural network prediction large model to automatically answer user questions can quickly and accurately answer user questions, greatly improving the efficiency of information acquisition and user experience. At the same time, AI smart assistants can be used as auxiliary tools for teachers to help them manage classes and answer students' questions, learn each student's questioning habits and knowledge level, and thus provide more personalized answers and learning suggestions, which are widely used in the field of education. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 This is a flow chart of the method for constructing an after-school service AI intelligent assistant based on neural network proposed in this application.

[0025] Figure 2 This is a schematic diagram of the system structure for building an after-school service AI intelligent assistant based on neural network proposed in this application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] First, refer to Figure 1 As shown, the present application proposes a method for constructing an after-school service AI intelligent assistant based on a neural network, and the method comprises the following steps:

[0028] S1: Acquire interactive information uploaded by a user, wherein the interactive information is information about questions to be answered uploaded by the user or information about questions and answers to be corrected;

[0029] Specifically, users can upload unanswered information for students or teachers to be graded, such as some difficult questions that need to be answered or homework with solution ideas. Users can control the selection box displayed in the terminal interface to capture the questions to be searched for answers in the image. In this way, the terminal analyzes the question content through the image area captured in the selection box and uploads it, thereby completing the function of taking a photo to search for questions.

[0030] Specifically, the interactive information uploaded by the user can be input through voice, text or image. If the data format uploaded by the user is image or voice, it is necessary to first perform image recognition or voice recognition through the image processing unit or the voice processing unit to obtain the text content, and then perform text conversion to obtain the word vector sequence. If the uploaded data format is text, the text processing subunit performs text conversion to the word vector sequence.

[0031] The image format and text information are obtained. For the image format, the user can directly photograph the title through a collection device such as a camera, or photograph the image after certain image processing, or take a screenshot. There is no limitation here.

[0032] S2: Preprocessing the user interaction information, including image processing, text processing and voice processing. The image processing unit is used to perform image recognition based on the image data to obtain the text content of the question to be answered or to be corrected. The voice processing unit is used to respond to the user's voice input, set the topic according to the scene, and generate a response according to the context and the big data model to obtain the text content of the question to be answered or to be corrected. The text processing unit is used to convert the text into a word vector sequence, convert the information to be answered uploaded by the user into a first word vector sequence, and convert the information to be corrected into a second word vector sequence.

[0033] Specifically, the multimodal OCR in the present application recognizes and processes common text information, etc., identifies relevant text from the image to be imaged. Through multimodal recognition, not only the visible text information in the image can be recognized, but also the semantic features in the image can be recognized and converted into text information.

[0034] Image processing includes inputting the image into OCR recognition, using the pre-trained text classification FastText model to extract the feature information of the text in the image and the position coordinates of the bounding box of each OCR mark in the image, and obtaining M OCR marks for each image. For the mth mark (where l∈[1,...,M]), the amount of text content contained in the image depends on the size of the M value, and finally a 300-dimensional feature vector Xo is obtained, which represents the image text and obtains the position information.

[0035] The extraction process can be expressed as:

[0036] (token,Bo)=OCR

[0037] Xo=FastText(token)

[0038] Among them, Xo∈RM×300 and Bo∈RM×4 represent the image text features on the target image object and the coordinate position of the bounding box of each word in the image, respectively, and token represents the OCR tag.

[0039] Voice processing includes responding to the user's voice input, the AI ​​smart assistant interacting with the user, setting various chat topics according to various chat scenarios, and generating responses based on the context and big data models, converting them into text. During this process, users can continuously modify and correct their own problems, such as inaccuracies caused by pronunciation, noise, etc., until the user confirms the final version;

[0040] Text processing includes but is not limited to extracting text features through natural language processing technology, for example, using methods such as text encoders to convert text into vector representations, or using pre-trained word embedding models to convert text into dense vector representations to obtain the above-mentioned initial text features for describing the semantic and grammatical information of the target text information.

[0041] S3: construct a multi-layer neural network model, wherein the multi-layer neural network model includes an input layer, a plurality of cascaded convolutional layers, an encoder, a normalization layer, and a fully connected output layer arranged in sequence from front to back;

[0042] The input layer is responsible for receiving the original data samples and converting them into a one-dimensional format for processing by subsequent network layers. The network structure contains three consecutive convolutional layers. The first convolutional layer is equipped with 32 3x3 convolution kernels, and the ReLU activation function is applied with a step size of 1. The second convolutional layer that follows has 64 3x3 convolution kernels, and the ReLU activation function is also used, with a step size of 1. The third convolutional layer is equipped with 128 3x3 convolution kernels, and the ReLU activation function is still used, with a step size of 1.

[0043] The encoder consists of four downsampling modules, each of which consists of two 3x3 convolutional layers, a ReLU activation function, and a 2x2 pooling layer. The purpose is to reduce the dimensionality of the input data and convert it into one-dimensional data.

[0044] After the normalization layer, the one-dimensional data is feature extracted and normalized, and then these features are activated by the LeakyRelu activation function;

[0045] The fully connected output layer receives the output of the third convolutional layer, converts it into a one-dimensional vector, and generates a discriminant value for classification or regression through the fully connected layer.

[0046] S4: Obtain a sample set, and train the multi-layer neural network model based on the sample set, wherein the sample set is questions and corresponding answers obtained from a corpus database and historical question and answer information, wherein the historical question and answer information includes historical questions and historical answers corresponding to the historical questions;

[0047] Specifically, training the multi-layer neural network model based on the sample set specifically includes: forward propagation and back propagation parameter adjustment. First, in the forward propagation stage, the training data is input into the previously constructed multi-layer neural network, and the calculation is performed layer by layer from the input layer to the output layer in the order of the network layers, and finally the prediction result of the model is obtained; then, these prediction results are compared with the actual values, and the loss value Loss is determined by calculating the error; in the process of forward propagation, each network layer will calculate the activation value, and convert these values ​​into power domain fixed-point data so as to serve as the input of the next layer, and this process will continue until the activation value of the last network layer is calculated, and this value is the final result of the forward propagation;

[0048] In the back propagation phase, the loss value is sent to the neural network, starting from the output layer and calculating forward layer by layer until it reaches the input layer. In this process, the stochastic gradient descent method with momentum is used to optimize the network parameters. Through continuous iterations, until the network parameters converge, the optimal network structure is obtained.

[0049] In the calculation of loss value, one is the output value of the standard fully connected layer, and the first loss value Loss1 is calculated using the flexible maximum loss function; the other is the output value of the fully connected layer with regularization, and the second loss value Loss2 is calculated using the flexible maximum loss function with additional intervals. These two loss values ​​will be weighted and summed according to the pre-set weighting coefficients α and β to obtain the total loss value Total Loss = αLoss1 + βLoss2;

[0050] Finally, by setting the number of iterations, the total loss value is repeatedly calculated and the parameters of the network structure model are optimized until the optimal multi-layer neural network model is reached, completing the training process.

[0051] S5: input the word vector sequence into the multi-layer neural network model to obtain the multi-layer neural network model to select answers that meet the preset conditions from the preset sample set as the standard answer;

[0052] Specifically, the verification of the multi-layer neural network model through questions and answers in the sample set includes the following: Verify the performance of the trained neural network: select a single question from the test set, and pair it with all possible answers in the test set to form multiple sets of question-answer pairs (A, Q), input these pairs into the trained neural network to calculate the degree of match between the question and the answer, and sort these pairs according to the degree of match, so as to evaluate the accuracy of the neural network in matching questions and answers. If the pairing of the question and the correct answer is in the top 5 list, it is considered that the correct answer has been identified; if not, it is considered that the identification has failed. Repeat this verification process for each question in the test set, and count the ratio of correctly matched questions to total questions. If this ratio meets or exceeds the set threshold, the effectiveness of the neural network is confirmed; if it is lower than the threshold, the neural network needs to be retrained and tested until it reaches or exceeds the predetermined matching success rate;

[0053] Apply a tested and verified neural network to calculate the matching degree between questions and answers: combine the question to be matched with multiple candidate answers into multiple question-answer pairs (A, Q), and input these combinations into the verified neural network to evaluate the matching degree between the question and each answer, and select the answer with the highest matching degree as the best answer to the question;

[0054] If the recognition fails, this usually indicates that the sample data source is insufficient. For example, the preset sample library may not contain information about the question and its answer. In order to improve the generalization ability of the intelligent AI assistant, once the AI ​​assistant confirms that the current question is a new question that has not been encountered before, it will conduct online learning, update the parameters of the neural network model, and effectively predict the corresponding answer, and save the answer in the preset sample library.

[0055] S6: Evaluate the uploaded information to be corrected, compare the students' responses with the preset standard answers, and make quantitative assessments for each scoring point. When it is found that there is a deviation between the user's answer and the standard answer, provide the problem-solving logic of the standard answer and record it in the wrong question collection. By regularly analyzing the user's question and answer performance and homework correction records, compile a learning progress report based on the time period, and make personalized learning improvement suggestions accordingly.

[0056] Specifically, the method further includes the following steps: obtaining a standard answer, and preprocessing the standard answer to extract keywords from the standard answer;

[0057] Determine the text information entropy of the answer based on the probability of each answer keyword appearing in the answer;

[0058] Compare the answers in the information to be corrected and pre-process them in the same way as the standard answers to obtain keywords of the answers to be corrected;

[0059] Perform vector conversion on the standard answer keywords and the answer keywords to be corrected to obtain the standard answer word vectors corresponding to the multiple standard answer keywords and the target word vectors corresponding to the answer keywords to be corrected, and calculate the cosine similarity between the two;

[0060]

[0061] Where, s is the cosine similarity between each standard answer keyword and the keyword of the answer to be corrected; v1,i is the value of the standard answer keyword in the i-th dimension; v2,i is the value of the answer keyword to be corrected in the i-th dimension;

[0062] According to the following formula, the cosine similarity is weighted averaged according to the text information entropy to obtain the processed similarity:

[0063]

[0064] Wherein, sg is the similarity after processing; k is the number of elements in the similarity probability array; si is the value of the i-th element in the similarity probability array; h(x) is the text information entropy of the text to be processed;

[0065] According to the probability of each answer keyword appearing in the answer, the information entropy of the answer is determined and calculated according to the following formula: h(x) = -∑P(x)×log2(P(key));

[0066] Wherein, h(x) is the information entropy of the answer; p(x) is the probability of each answer keyword appearing in the answer; X is the answer text, and key is the answer keyword.

[0067] Specifically, the AI ​​smart assistant can be set up to regularly summarize and analyze wrong questions every week or month to find out which knowledge points the user has problems with. Wrong questions encountered during the learning process can be entered into the system, and the system automatically records information such as question type, error cause, and problem-solving ideas. Based on the data analysis results in the wrong question book, special training questions are generated for user personalized training to strengthen weak links, and algorithms are used to intelligently recommend similar questions or variant questions to increase the diversity of training. Based on the user's learning data, weekly, monthly and semester reports are generated to visualize the data, and charts are used to display error frequency, error type, etc., to help users understand the learning status more intuitively. At the same time, learning suggestions and the next learning plan can be generated based on the report, and suitable learning resources and methods can be recommended to help set short-term and long-term learning goals.

[0068] In a second aspect, the present application provides a system for constructing an after-school service AI intelligent assistant based on a neural network, which is used to execute the method for constructing an after-school service AI intelligent assistant based on a neural network, including: a first acquisition unit, which is used to acquire interactive information uploaded by a user, wherein the interactive information is information about questions to be answered uploaded by the user or information to be corrected containing questions and answers;

[0069] A processing unit pre-processes the user interaction information, including image processing, text processing and voice processing. The image processing unit is used to perform image recognition based on the image data to obtain the text content of the question to be answered or to be corrected. The voice processing unit is used to respond to the user's voice input, and the AI ​​intelligent assistant interacts with the user. At the same time, various chat topics are set according to various chat scenarios, and responses are generated according to the context and big data model to obtain the text content of the question to be answered or to be corrected; the text processing unit is used to convert the text into a word vector sequence, convert the information to be answered uploaded by the user into a first word vector sequence, and convert the information to be corrected into a second word vector sequence;

[0070] A first construction unit constructs a multi-layer neural network model, wherein the multi-layer neural network model includes an input layer, a plurality of cascaded convolutional layers, an encoder, a normalization layer, and a fully connected output layer arranged in sequence from front to back;

[0071] A second acquisition unit acquires a sample set, and trains the multi-layer neural network model based on the sample set, wherein the sample set is questions and corresponding answers acquired from a corpus database and historical question-and-answer information, wherein the historical question-and-answer information includes historical questions and historical answers corresponding to the historical questions;

[0072] Determine the subunit, input the word vector sequence into the multi-layer neural network model, so as to obtain the answer that meets the preset conditions selected by the multi-layer neural network model from the preset sample set as the standard answer;

[0073] The feedback sub-unit evaluates the uploaded information to be marked, compares the students' responses with the preset standard answers, and makes quantitative assessments on each scoring point. When a deviation is found between the user's answer and the standard answer, the unit provides the problem-solving logic of the standard answer and records it in the wrong question collection. By regularly analyzing the user's question and answer performance and homework marking records, it compiles a learning progress report based on the time period and makes personalized learning improvement suggestions accordingly.

[0074] Specifically, the multi-layer neural network model framework includes multiple cascaded convolutional layers including three convolutional layers. The first convolutional layer, the second convolutional layer, and the third convolutional layer all use the output of the previous network layer connected to them as input data, wherein the first convolutional layer includes 32 3x3 convolutional kernels, ReLU activation function, and a step size of 1; the second convolutional layer contains 64 3x3 convolutional kernels, a step size of 1, and also uses the ReLU activation function; the third convolutional layer contains 128 3x3 convolutional kernels, a step size of 1, and also uses the ReLU activation function;

[0075] The encoder contains 4 downsampling modules, each of which contains two 3×3 convolutional layers, a ReLU activation function, and a 2×2 pooling layer to downsample the input data and convert it into one-dimensional data;

[0076] The normalization layer extracts features from the one-dimensional data, and then normalizes the extracted features. The processed features are activated by the LeakyRelu function. The activated features are used as output data, and thus as input for the next network layer.

[0077] The fully connected output layer takes the output data of the third convolutional layer as input, flattens it into a one-dimensional vector, and then outputs a discriminant value through full connection.

[0078] The present application provides a computer storage medium, wherein the computer storage medium stores at least one instruction, wherein the instruction is suitable for being loaded by a processor and executing the method steps of one or more embodiments of the present specification.

[0079] The present application provides a computer program product, which stores at least one instruction, and the instruction is suitable for being loaded by a processor and executing the method steps of one or more embodiments of the present specification.

[0080] The present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps of one or more embodiments of the present specification.

[0081] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method for constructing an after-school service AI intelligent assistant based on a neural network, characterized in that: The following steps are involved: S1: Acquire interactive information uploaded by a user, wherein the interactive information is information about questions to be answered uploaded by the user or information about questions and answers to be corrected; S2: Preprocessing the user interaction information, including image processing, text processing and voice processing. The image processing unit is used to perform image recognition based on the image data uploaded by the user to obtain the text content of the question to be answered or to be corrected. The voice processing unit is used to respond to the user's voice input, set the topic according to the scene, and generate a response according to the context and the big data model to obtain the text content of the question to be answered or to be corrected; the text processing unit is used to convert the text into a word vector sequence, convert the information to be answered uploaded by the user into a first word vector sequence, and convert the information to be corrected into a second word vector sequence; S3: construct a multi-layer neural network model, wherein the multi-layer neural network model includes an input layer, a plurality of cascaded convolutional layers, an encoder, a normalization layer, and a fully connected output layer arranged in sequence from front to back; S4: Obtain a sample set, and train the multi-layer neural network model based on the sample set, wherein the sample set is questions and corresponding answers obtained from a corpus database and historical question and answer information, wherein the historical question and answer information includes historical questions and historical answers corresponding to the historical questions; S5: input the word vector sequence into the multi-layer neural network model to obtain the answer that meets the preset conditions selected by the multi-layer neural network model from the preset sample set as the standard answer; S6: Evaluate the uploaded information to be corrected, compare the students' responses with the preset standard answers, and make quantitative assessments for each scoring point. When it is found that there is a deviation between the user's answer and the standard answer, provide the problem-solving logic of the standard answer and record it in the wrong question collection. By regularly analyzing the user's question and answer performance and homework correction records, compile a learning progress report based on the time period, and make personalized learning improvement suggestions accordingly.

2. The method for constructing an after-school service AI intelligent assistant according to claim 1 is characterized in that: The image processing in step S2 specifically includes the following contents: the image processing includes inputting the image into OCR recognition, using the pre-trained text classification FastText model to extract the feature information of the text in the image and the position coordinates of the bounding box of each OCR mark in the image, and obtaining M OCR marks for each image. For the mth mark (where l∈[1,...,M]), the amount of text content contained in the image depends on the value of M, and finally obtaining a 300-dimensional feature vector Xo, and obtaining the representation and position information of the image text; The extraction process can be expressed as: (token,Bo)=OCR Xo=FastText(token) Among them, Xo∈RM×300 and Bo∈RM×4 represent the image text features on the target image object and the coordinate position of the bounding box of each word in the image, respectively, and token represents the OCR tag.

3. The method for constructing an after-school service AI intelligent assistant according to claim 1 is characterized in that: The voice processing in step S2 specifically includes the following contents: Voice processing includes the AI ​​smart assistant interacting with the user, setting various chat topics according to various chat scenarios, and generating responses based on the context and big data models, and converting them into text. During this process, the user can continuously modify and correct inaccuracies caused by pronunciation and noise until the user confirms the final version.

4. The method for constructing an after-school service AI intelligent assistant according to claim 1 is characterized in that: The text processing in step S2 specifically includes the following: using a text encoder to convert the text into a vector representation, or using a pre-trained word embedding model to convert the text into a dense vector representation to obtain text features for describing the semantic and grammatical information of the text information.

5. The method for constructing an after-school service AI intelligent assistant according to claim 1, characterized in that: The multi-layer neural network model in step S3 specifically includes: the input layer is responsible for receiving the original data samples and converting them into a one-dimensional format for processing by the subsequent network layers; the network structure includes three consecutive convolutional layers, the first convolutional layer is equipped with 32 3x3 convolutional kernels, and the ReLU activation function is applied, and the step size is set to 1; the second convolutional layer immediately following it has 64 3x3 convolutional kernels, and the ReLU activation function is also used, and the step size is maintained at 1; the third convolutional layer is equipped with 128 3x3 convolutional kernels, and the ReLU activation function is continued to be used, and the step size is also 1; The encoder consists of four downsampling modules, each of which consists of two 3x3 convolutional layers, a ReLU activation function, and a 2x2 pooling layer. The purpose is to reduce the dimensionality of the input data and convert it into one-dimensional data. After the normalization layer, the one-dimensional data is feature extracted and normalized, and then these features are activated by the LeakyRelu activation function; The fully connected output layer receives the output of the third convolutional layer, converts it into a one-dimensional vector, and generates a discriminant value for classification or regression through the fully connected layer.

6. The method for constructing an after-school service AI intelligent assistant according to claim 1, characterized in that: Step S4 training the multi-layer neural network model based on the sample set specifically includes the following: forward propagation and back propagation parameter adjustment, first, in the forward propagation stage, the training data is input into the previously constructed multi-layer neural network, and calculations are performed layer by layer from the input layer to the output layer in the order of the network layers, and finally the prediction results of the model are obtained; then, these prediction results are compared with the actual values, and the loss value Loss is determined by calculating the error; During the forward propagation process, each network layer calculates the activation value and converts these values ​​into fixed-point data in the power domain so that they can be used as input for the next layer. This process continues until the activation value of the last network layer is calculated, which is the final result of the forward propagation. In the back propagation phase, the loss value is sent to the neural network, starting from the output layer and calculating forward layer by layer until it reaches the input layer. In this process, the stochastic gradient descent method with momentum is used to optimize the network parameters. Through continuous iterations, until the network parameters converge, the optimal network structure is obtained. In the calculation of loss value, one is the output value of the standard fully connected layer, and the first loss value Loss1 is calculated using the flexible maximum loss function; the other is the output value of the fully connected layer with regularization, and the second loss value Loss2 is calculated using the flexible maximum loss function with additional intervals. These two loss values ​​will be weighted and summed according to the pre-set weighting coefficients α and β to obtain the total loss value Total Loss = αLoss1 + βLoss2; Finally, by setting the number of iterations, the total loss value is repeatedly calculated and the parameters of the network structure model are optimized until the optimal multi-layer neural network model is reached, completing the training process.

7. The method for constructing an after-school service AI intelligent assistant according to claim 1, characterized in that: Verifying the multi-layer neural network model through questions and answers in the sample set includes the following: Verifying the performance of the trained neural network: Select a single question from the test set and pair it with all possible answers in the test set to form multiple sets of question-answer pairs (A, Q), input these pairs into the trained neural network to calculate the degree of match between the question and the answer, and sort these pairs according to the degree of match to evaluate the accuracy of the neural network in matching questions and answers. If the pairing of the question and the correct answer is in the top 5 list, the correct answer is considered to have been identified; if not, the identification is considered to have failed. Repeat this verification process for each question in the test set, and count the ratio of correctly matched questions to the total number of questions. If this ratio meets or exceeds the set threshold, the effectiveness of the neural network is confirmed; if it is lower than the threshold, the neural network needs to be retrained and tested until it reaches or exceeds the predetermined matching success rate; Apply a tested and verified neural network to calculate the matching degree between questions and answers: combine the question to be matched with multiple candidate answers into multiple question-answer pairs (A, Q), and input these combinations into a verified neural network to evaluate the matching degree between the question and each answer, and select the answer with the highest matching degree as the best answer to the question.

8. The method for constructing an after-school service AI intelligent assistant according to claim 7, characterized in that: If the recognition fails, it may also include: the preset sample library does not contain information about the question and its answer. Once the AI ​​smart assistant confirms that the current question is a new question that has not been encountered before, it will conduct online learning, update the parameters of the neural network model, and effectively predict the answer, and save the answer in the preset sample library.

9. The method for constructing an after-school service AI intelligent assistant according to claim 1, characterized in that: Step S6 specifically includes the following contents: Obtain standard answers, and preprocess the standard answers to extract keywords in the standard answers; determine the text information entropy of the answers according to the probability of each answer keyword appearing in the answers; compare the answers in the information to be corrected and preprocess them in the same way as the standard answers to obtain keywords of the answers to be corrected; perform vector conversion on the keywords of the standard answer and the keywords of the answers to be corrected to obtain the standard answer word vector corresponding to the keywords of the standard answer and the target word vector corresponding to the keywords of the answers to be corrected, and calculate the cosine similarity between the two; Where, s is the cosine similarity between each standard answer keyword and the keyword of the answer to be corrected; v1,i is the value of the standard answer keyword in the i-th dimension; v2,i is the value of the answer keyword to be corrected in the i-th dimension; According to the following formula, the cosine similarity is weighted averaged according to the text information entropy to obtain the processed similarity: Wherein, sg is the similarity after the processing; k is the number of elements in the similarity probability array; si is the value of the i-th element in the similarity probability array; h(x) is the text information entropy of the text to be processed; According to the probability of each answer keyword appearing in the answer, the information entropy of the answer is determined and calculated according to the following formula: h(x)=-∑P(x)×log2(P(key)); Wherein, h(x) is the information entropy of the answer; p(x) is the probability of each answer keyword appearing in the answer; x is the answer text, and key is the answer keyword.

10. A system for constructing an after-school service AI intelligent assistant based on a neural network, used to execute the method for constructing an after-school service AI intelligent assistant based on a neural network as described in any one of claims 1 to 9, characterized in that: The system includes the following modules: a first acquisition unit, used to acquire interactive information uploaded by a user, wherein the interactive information is information about questions to be answered uploaded by the user or information to be corrected including questions and answers; A processing unit pre-processes the user interaction information, including image processing, text processing and voice processing. The image processing unit is used to perform image recognition based on the image data to obtain the text content of the question to be answered or to be corrected. The voice processing unit is used to respond to the user's voice input, and the AI ​​intelligent assistant interacts with the user. At the same time, various chat topics are set according to various chat scenarios, and responses are generated according to the context and big data model to obtain the text content of the question to be answered or to be corrected; the text processing unit is used to convert the text into a word vector sequence, convert the information to be answered uploaded by the user into a first word vector sequence, and convert the information to be corrected into a second word vector sequence; A first construction unit constructs a multi-layer neural network model, wherein the multi-layer neural network model includes an input layer, a plurality of cascaded convolutional layers, an encoder, a normalization layer, and a fully connected output layer arranged in sequence from front to back; A second acquisition unit acquires a sample set, and trains the multi-layer neural network model based on the sample set, wherein the sample set is questions and corresponding answers acquired from a corpus database and historical question-and-answer information, wherein the historical question-and-answer information includes historical questions and historical answers corresponding to the historical questions; Determine the subunit, input the word vector sequence into the multi-layer neural network model, so as to obtain the answer that meets the preset conditions selected by the multi-layer neural network model from the preset sample set as the standard answer; The feedback sub-unit evaluates the uploaded information to be marked, compares the students' responses with the preset standard answers, and makes quantitative assessments on each scoring point. When a deviation is found between the user's answer and the standard answer, the unit provides the problem-solving logic of the standard answer and records it in the wrong question collection. By regularly analyzing the user's question and answer performance and homework marking records, it compiles a learning progress report based on the time period and makes personalized learning improvement suggestions accordingly.

Images