An artificial intelligence-based education knowledge graph construction method and system

Abstract

The application relates to the technical field of education data processing, and discloses an education knowledge graph construction method and system based on artificial intelligence, which comprises a multi-scale representation analysis unit, a structure parameter configuration unit, a multi-modal graph element assembly unit and an intelligent link optimization unit. The system first extracts the multi-layer convolution response distribution of a normalized teaching image by using a convolutional neural network, calculates a semantic coverage decay index reflecting the change of information coverage range with scale, then maps the index to node bearing potential, and calculates the number quota of visual anchor points and concept entities for the image according to the node bearing potential, then positions the visual area and extracts associated text concepts according to the number quota to establish a weighted image substructure, and finally performs relationship reasoning by using a graph completion model to finalize the final graph. The application can adaptively adjust the composition scale according to the knowledge density of the image content, solves the problem of imbalance of multi-modal graph structure density, and is suitable for a smart education scene.

Description

Technical Field

[0001] This invention relates to the field of educational data processing technology, and more specifically, to a method and system for constructing an educational knowledge graph based on artificial intelligence. Background Technology

[0002] With the development of online education and smart teaching, educational knowledge graphs, as core infrastructure connecting teaching resources and planning learning paths, are playing an increasingly important role. Traditional knowledge graph construction mainly relies on parsing textual data such as textbooks and lesson plans, extracting entities and relationships through natural language processing techniques. However, in actual teaching scenarios, a large amount of key knowledge does not exist solely in textual form but is carried within visual images such as function graphs, circuit diagrams, geometric diagrams, and flowcharts. Therefore, the construction of multimodal knowledge graphs that integrate visual and textual elements has become a current research hotspot.

[0003] In existing multimodal educational knowledge graph construction processes, deep learning models are typically used to process teaching images. Mainstream solutions often treat images as independent nodes or utilize general object detection algorithms to identify objects of several pre-defined categories from images and link them to the knowledge graph. For example, when processing an exercise containing a circuit diagram, the system might attempt to identify components such as resistors and power supplies in the diagram and establish connections between them and related concepts.

[0004] However, existing processing methods have significant limitations when dealing with the complexity of educational scenarios. The main problem is that existing algorithms typically employ fixed or static strategies to determine how much information to extract from an image. For example, the system might set a fixed extraction of the top N objects with the highest confidence for each image, or simply attach the image as a whole attribute to the text node.

[0005] This static strategy ignores the vast differences in content density among educational images. In actual textbook layouts, a chapter overview diagram might cover a dozen key concepts and complex logical relationships, while a local example diagram might only correspond to a single knowledge point. Using a uniform extraction strategy leads to severe structural imbalance: for high-density overview diagrams, a fixed extraction quantity results in the omission of a large amount of key information, causing incomplete relational links in the graph; while for low-density local graphs, forcibly extracting too much information introduces irrelevant noise, leading to redundant graph structure. This uncontrolled graph structure density ultimately affects the accuracy and computational convergence efficiency of upper-level applications (such as test question recommendation and path planning). Therefore, how to enable computer systems to perceive the richness of knowledge contained within images and adaptively allocate computational resources and structural space for graph construction is a pressing technical challenge in the current construction of educational knowledge graphs. Summary of the Invention

[0006] This invention provides a method and system for constructing educational knowledge graphs based on artificial intelligence, which solves the technical problems mentioned in the background.

[0007] Firstly, an artificial intelligence-based educational knowledge graph construction system includes:

[0008] The multi-scale representation analysis unit is configured to extract multi-layer convolutional response maps of standardized teaching images using convolutional neural networks, statistically analyze the response energy distribution of the multi-layer convolutional response maps, and calculate the semantic coverage attenuation index that reflects the change of image information with the theoretical receptive field scale.

[0009] The structural parameter configuration unit is configured to map the semantic coverage attenuation index to a node carrying potential value, and to calculate the visual anchor point quota and concept entity quota for the teaching image based on the node carrying potential value.

[0010] The multimodal primitive assembly unit is configured to locate visual attention regions in the teaching image according to the quota of the number of visual anchors to form a weighted image substructure, and to extract concept items from the associated text according to the quota of the number of concept entities to establish cross-modal connections.

[0011] The intelligent link optimization unit is configured to input the initial triplet into the convolutional graph completion model for relational reasoning, generate an educational knowledge graph data structure containing multimodal relational data, thereby optimizing the storage topology density of the multimodal graph data and accelerating the convergence of relational reasoning computation.

[0012] Secondly, an artificial intelligence-based method for constructing educational knowledge graphs, applied to any of the aforementioned artificial intelligence-based educational knowledge graph construction systems, includes:

[0013] Multi-layer convolutional response maps of standardized teaching images are extracted using convolutional neural networks, and the response energy distribution of the multi-layer convolutional response maps is statistically analyzed. A semantic coverage attenuation index that reflects the change of image information with the theoretical receptive field scale is calculated.

[0014] The semantic coverage attenuation index is mapped to the node carrying potential value, and the visual anchor point quota and concept entity quota for the teaching image are calculated based on the node carrying potential value.

[0015] Based on the quota of visual anchor points, the visual attention region is located in the teaching image to form a weighted image substructure, and based on the quota of conceptual entities, conceptual items are extracted from the associated text to establish cross-modal connections.

[0016] The initial triples are input into the convolutional graph completion model for relational reasoning, generating an educational knowledge graph data structure containing multimodal relational data. This optimizes the storage topology density of the multimodal graph data and accelerates the convergence of relational reasoning computation.

[0017] The beneficial effects of this invention include: by analyzing the decay law of response energy in convolutional neural networks with the change of receptive field scale, a quantitative index reflecting the semantic coverage breadth of teaching images is constructed, and this index is used to establish an adaptive mapping mechanism from image content complexity to graph construction quota. This mechanism enables the system to dynamically adjust the number of visual attention points and conceptual entities generated based on whether the image is an overview map carrying multiple knowledge points or a local map carrying a single knowledge point, effectively overcoming the problem of graph structure density imbalance caused by the fixed strategy used in traditional methods. This invention not only ensures the full expression of high-information images and the noise suppression of low-information images, significantly improving the topological quality of multimodal educational knowledge graphs, but also effectively reduces the storage overhead of redundant data and accelerates the convergence process of subsequent inference calculations by optimizing the sparsity distribution of graph data. Attached Figure Description

[0018] Figure 1 This is a flowchart of an artificial intelligence-based educational knowledge graph construction system according to the present invention. Detailed Implementation

[0019] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0020] Example 1: As Figure 1 As shown, an artificial intelligence-based educational knowledge graph construction system includes:

[0021] The multi-scale representation analysis unit is configured to extract multi-layer convolutional response maps of standardized teaching images using convolutional neural networks, statistically analyze the response energy distribution of the multi-layer convolutional response maps, and calculate the semantic coverage attenuation index that reflects the change of image information with the theoretical receptive field scale.

[0022] The structural parameter configuration unit is configured to map the semantic coverage attenuation index to a node carrying potential value, and to calculate the visual anchor point quota and concept entity quota for the teaching image based on the node carrying potential value.

[0023] The multimodal primitive assembly unit is configured to locate visual attention regions in the teaching image according to the quota of the number of visual anchors to form a weighted image substructure, and to extract concept items from the associated text according to the quota of the number of concept entities to establish cross-modal connections.

[0024] The intelligent link optimization unit is configured to input the initial triplet into the convolutional graph completion model for relational reasoning, generate an educational knowledge graph data structure containing multimodal relational data, thereby optimizing the storage topology density of the multimodal graph data and accelerating the convergence of relational reasoning computation.

[0025] Preferably, the method involves using a convolutional neural network to extract multi-layer convolutional response maps of standardized teaching images, including:

[0026] Step A: Preprocess and crop / normalize the original document page using the following formula to obtain the standardized teaching image:

[0027] ;

[0028] ;

[0029] in, Indicates the first One original document page; This represents the grayscale conversion operator; This indicates scaling the longer side of the image to the target size. And operators that scale proportionally on the shorter side; This means padding the image with zeros to make it larger with sides of length 1. The square operator; This represents the preset uniform resolution constant; A page image representing a uniform resolution; Indicates from The teaching image area cropped from the middle is in the coordinate system. Pixel value at; and These represent the mean and standard deviation of the pixel values ​​in the teaching image region, respectively. Represents the constant used for numerical stability; This refers to the standardized teaching image;

[0030] Step B: Input the normalized teaching image into a deep convolutional neural network, and extract the multi-layer convolutional response map using the following formula:

[0031] ;

[0032] in, This represents the total number of layers used for feature extraction; This represents the deep convolutional neural network from the input layer to the... Layer mapping function; Indicates the first The feature tensor output by the layer, i.e., the multi-layer convolutional response map; the deep convolutional neural network in the first layer... The layer has a defined theoretical receptive field scale.

[0033] The original document page is the input page of the educational document to be processed, including PDF pages or image pages of textbooks, courseware, and question banks. This can be obtained by scanning paper documents or directly reading electronic document files. The uniform resolution constant is a preset fixed value used to standardize the page image size, preferably 1024, which meets the common size range of educational document images. This value balances processing efficiency and feature extraction accuracy, adapting to the receptive field calculation requirements of subsequent convolutional neural networks. The pixel values ​​of the teaching image region are the brightness values ​​corresponding to each spatial coordinate point of the teaching image portion cropped from the uniform resolution page image. This can be obtained by locating the teaching image region using layout analysis tools and reading the grayscale value of each pixel within that region. The pixel mean is the arithmetic mean of the brightness values ​​of all pixels within the cropped teaching image region. The pixel standard deviation is the root mean of the sum of the squares of the differences between the brightness values ​​of all pixels within the cropped teaching image region and the pixel mean. The numerical stability constant is a very small, fixed value set to avoid errors in division operations when the variance is zero. It is preferably 10 to the power of -6. This value is small enough not to affect the accuracy of the normalization result and effectively avoids numerical calculation anomalies. The standardized teaching image is a standard image obtained by normalizing the mean and variance of the cropped teaching image region. The convolution kernel size is the side length of the filter used in the convolution operation of the convolutional neural network. It is preferably 3×3. A 3×3 convolution kernel can effectively capture local image features and has a moderate computational cost, suitable for extracting detailed features from educational images, such as lines in function graphs and component outlines in circuit diagrams. The stride parameter is the distance the convolution kernel moves across the image in each iteration. It is preferably 1 or 2. A stride of 1 preserves more image details, while a stride of 2 reduces the feature map size and computational cost. The stride can be flexibly selected according to the complexity of the educational image; for example, a stride of 2 is used for simple geometric figures, and a stride of 1 is used for complex circuit diagrams. The total number of feature extraction layers is selected from the convolutional neural network (CNN) to generate multi-layer convolutional response maps, preferably 5. Five layers of features can cover multi-scale information from local texture to global structure, adapting to the feature extraction needs of different levels of educational images, such as lower layers capturing line textures and higher layers capturing overall structure. The layer index is a sequence variable used to identify different convolutional layers, used to traverse the preset layers for feature extraction. The convolutional layer mapping function is the feature transformation rule from the input layer to the corresponding layer in the CNN. A combination structure of convolutional layers, batch normalization, and linear rectified activation functions is preferred, as this structure can effectively extract image features and alleviate the gradient vanishing problem, making it suitable for feature learning in educational images. The multi-layer convolutional response map is a feature tensor output from the preset layers of the CNN. The response map of each layer corresponds to the image feature representation at a specific theoretical receptive field scale.The total number of channels is the number of feature channels in the output feature tensor of each convolutional layer. Preferred values ​​are 32, 64, 128, 256, and 256, which gradually increase the number of channels as the layers deepen, so as to capture more complex features and adapt to the feature representation needs of educational images from simple to complex.

[0034] To ensure uniform resolution processing of educational documents, the following approach is adopted: scaling the long side and centering with zero values ​​to create squares. Images in educational documents are often bound to the textbook layout, and are uniformly sized to 1024×1024 squares. This ensures the comparability of receptive field scales for educational images from different document sources in subsequent convolutional neural networks, avoiding feature extraction biases caused by differences in original sizes. For example, if the original size of a function graph in one textbook is 800×600, and another is 1200×900, after processing with this strategy, both will become 1024×1024.

[0035] The layout analysis and cropping of teaching image areas includes: accurately separating captions and pure image areas based on the layout patterns of textbooks and courseware. Educational documents typically follow fixed layout patterns, such as captions immediately below or to the right of images, and clear spacing between text paragraphs and images. Using a layout detection model based on a masked region convolutional neural network, image areas can be accurately identified and cropped, avoiding interference from text textures in the feature extraction of educational images. For example, when cropping areas containing circuit diagrams, surrounding explanatory text will not be included.

[0036] The layer configuration of the deep convolutional neural network includes: five pre-defined convolutional layers bound to theoretical receptive field scales, specifically adapted to the multi-scale structural features of educational images. Educational images carry knowledge with multi-scale characteristics, such as the local details of components and the connection structures between components in a circuit diagram. The five-layer configuration can capture features at different scales: lower layers capture small-scale details such as lines and symbols, while higher layers capture large-scale information such as the overall structure and layout, providing a multi-scale feature foundation for the subsequent calculation of semantic coverage attenuation metrics.

[0037] The preferred value for the uniform resolution constant is 1024. This value is determined by considering the common size of educational document images, computer processing efficiency, and input requirements of convolutional neural networks. The size of 1024×1024 can preserve the key details of educational images without wasting computing resources due to excessive size.

[0038] The specific values ​​for the convolution kernel size and stride parameter are as follows: the convolution kernel size is uniformly set to 3, and the stride parameter is set to 1 in layers 1 and 5, and 2 in layers 2, 3, and 4. This configuration can gradually reduce the feature map size while ensuring feature extraction accuracy, thus balancing computational load and feature representation capability.

[0039] The preset number of layers is 5, with indices from 1 to 5. This means that the outputs of the first 5 convolutional layers of the convolutional neural network are extracted as multi-layer convolutional response maps. The theoretical receptive field scale of these 5 layers increases sequentially, covering features of educational images from local to global.

[0040] The specific implementation of layout analysis and cropping includes: employing a masked region convolutional neural network model trained on the Pabrinette dataset; cropping without a threshold setting, directly cropping based on the image region bounding boxes output by the model. This model exhibits excellent performance in document layout analysis tasks, accurately identifying image regions in educational documents and ensuring that the cropped teaching image regions are clean and free of interference.

[0041] Preferably, the response energy distribution of the multi-layer convolutional response map is statistically analyzed, and a semantic coverage attenuation index reflecting the change of image information with the theoretical receptive field scale is calculated, including:

[0042] Step A: Calculate the first step using the following formula. The response energy distribution and second-order Rayleigh entropy value of the multi-layer convolutional response map at each level:

[0043] ;

[0044] ;

[0045] ;

[0046] in, Indicates the first The hierarchical multi-layer convolutional response map in the channel and spatial location The value at that location; Indicates the first The total number of channels at each level; Represents the local response energy; This represents the normalized distribution of the response energy; Indicates the first The second-order Rayleigh entropy value of the hierarchy;

[0047] Step B: Calculate the semantic coverage attenuation index using the following formula:

[0048] ;

[0049] ;

[0050] in, Indicates the first The theoretical receptive field scale corresponding to the level; This represents the logarithmic value of the theoretical receptive field scale; This represents the second-order Rayleigh entropy value; and Representing sequences respectively and The arithmetic mean; Represents the natural logarithm; This represents the semantic coverage decay index.

[0051] Local response energy is the sum of the squares of all channel response values ​​at each spatial location in the multi-layer convolutional response map, used to quantify the feature activation intensity at that location. Total energy of the entire image is the sum of the local response energies at all spatial locations in the multi-layer convolutional response map, used to normalize the local response energies. Normalized response energy distribution is the result of dividing the local response energy at each spatial location by the total energy of the entire image, used to eliminate the influence of differences in energy scale between different layers. The second-order Rayleigh entropy is the negative logarithm of the sum of the squares of the normalized response energy distribution, used to characterize the dispersion of the energy distribution. The theoretical receptive field scale is the effective coverage of the original image by the corresponding convolutional layer in the convolutional neural network, recursively calculated from the kernel size and stride parameter. The logarithm of the theoretical receptive field scale is the natural logarithm of the theoretical receptive field scale, used to convert the scale variable into a form suitable for linear fitting. The second-order Rayleigh entropy sequence elements are the second-order Rayleigh entropy values ​​corresponding to each preset convolutional layer, forming the dependent variable sequence for linear regression. The semantic coverage decay index is the negative of the slope of the linear regression of the second-order Rayleigh entropy sequence with respect to the theoretical receptive field scale on the numerical sequence, used to quantify the rate at which image information decays with scale. The mean of the numerical sequence is the arithmetic mean of the numerical sequence with respect to the theoretical receptive field scale, used for linear regression calculation. The mean of the entropy sequence is the arithmetic mean of the second-order Rayleigh entropy sequence, used for linear regression calculation.

[0052] The second-order Rayleigh entropy value includes the spatial distribution of concept anchors that determine the knowledge content of educational images. The more dispersed the anchors, the more concepts the image covers, and the more dispersed the energy distribution, resulting in a higher second-order Rayleigh entropy value. For example, a chapter overview image contains multiple concept anchors, has a dispersed energy distribution, and a higher entropy value; a local example image has concentrated anchors, a concentrated energy distribution, and a lower entropy value. This relational logic is unique to educational scenarios and differs from the general application of entropy values ​​that only focus on visual features in images.

[0053] The theoretical correlation between receptive field scale and semantic coverage is based on the multi-scale knowledge representation characteristics of educational images. Lower-level convolutions, with their small receptive fields, capture local details of conceptual anchors, while higher-level convolutions, with their large receptive fields, aggregate anchors to form a global structure. As the receptive field scale increases, the semantic information of educational images converges from scattered local concepts to an aggregated overall theme, and the entropy value gradually decreases. This decay pattern reflects the change in semantic coverage.

[0054] The semantic coverage decay index is defined by inverting the slope of the linear regression, so that the index value is positively correlated with the semantic coverage. That is, the faster the decay, the larger the index, and the more image coverage concepts it corresponds to.

[0055] The specific recursive formula for the theoretical receptive field scale is as follows: Assume the initial receptive field is 1 and the initial cumulative stride is 1. The receptive field of the l-th layer is equal to the receptive field of the (l-1)-th layer plus (the size of the l-th convolutional kernel minus 1) multiplied by the cumulative stride of the (l-1)-th layer. The cumulative stride of the l-th layer is equal to the cumulative stride of the (l-1)-th layer multiplied by the stride of the l-th layer. For example, when the convolutional kernel size is 3 and the stride is 1, the receptive field of the second layer is 1 + (3-1) × 1 = 3; when the stride is 2, the receptive field of the third layer is 3 + (3-1) × 2 = 7.

[0056] The outlier handling strategy for linear regression fitting employs the three-standard-deviation method. This involves calculating the mean and standard deviation of the second-order Rayleigh entropy sequence, and removing outliers that deviate from the mean by more than three standard deviations before refitting. For example, if the entropy value of a certain level is significantly higher than that of other levels, it may be due to image noise; removing such outliers can prevent bias in the fitting results.

[0057] The specific measures for numerical stability in the calculation of second-order Rayleigh entropy are as follows: when calculating the sum of squares of the normalized response energy distribution, if the result approaches 0, add the minimum value of 10 to the power of negative 12, and then perform negative logarithmic calculation to avoid errors in logarithmic calculation.

[0058] Preferably, mapping the semantic coverage attenuation index to a node carrying potential value includes:

[0059] The node carrying capacity value is calculated using the following formula:

[0060] ;

[0061] in, This represents the semantic coverage attenuation index; This represents the preset scaling coefficient; This represents the preset bias calibration coefficient; Represents the natural exponential function; This represents the node's carrying potential value;

[0062] The scaling coefficient and the bias calibration coefficient Based on educational image sample set The fixed constant obtained by offline calibration is obtained by minimizing the following loss function:

[0063] ;

[0064] in, Indicates the total number of samples; Indicates the first The semantic coverage attenuation index of each sample image; Indicates the first The number of real concepts associated with each sample image; This represents the target value for normalized true concept coverage.

[0065] The semantic coverage attenuation index is a quantitative value reflecting the rate attenuation of educational image information as the theoretical receptive field scale changes. The scaling calibration coefficient is a fixed constant used to weight and adjust the semantic coverage attenuation index, preferably 1. This value allows for an initial linear fit between the semantic coverage attenuation index and the node carrying potential value, which can then be fine-tuned based on the educational image sample dataset. The bias calibration coefficient is a fixed constant used to adjust the offset of the linear transformation result, preferably 0. This ensures that no additional offset is needed initially, and the mapping result falls within a reasonable range through calibration using sample data. The linear transformation result is the value obtained by weighting the semantic coverage attenuation index with the scaling calibration coefficient and then superimposing it with the bias calibration coefficient. The node carrying potential value is a real value between zero and one after the linear transformation result is mapped by a sigmoid logistic function, representing the potential ability of the teaching image to carry conceptual entities. The total number of samples is the total number of educational image samples used for offline calibration of the scaling calibration coefficient and the bias calibration coefficient, preferably 10,000. This number of samples covers educational images of different subjects and types, ensuring the generalization of the calibration results. The semantic coverage decay index is the semantic coverage decay index corresponding to each educational image sample. The number of true concepts in a sample is the number of concept entities actually associated with each educational image sample, which can be obtained by parsing the image concept association files of the educational knowledge graph dataset, such as extracting the length of the concept list corresponding to each image from the image data file of the Mido educational knowledge graph. The normalization target value is the ratio of the number of true concepts in a sample to the maximum number of concepts; if the ratio is greater than one, it is rounded down to one, and used as a target reference value for the calibration process.

[0066] The offline training logic for calibration coefficients includes: using image concept association data from an educational knowledge graph dataset, and fitting an S-shaped mapping function by minimizing the squared error. The number of concept coverages in educational images exhibits a clear scene distribution pattern; for example, chapter overview images in middle school textbooks have a large number of associated concepts, while local example images have fewer. The calibration process fully utilizes this pattern to ensure that the mapping results accurately reflect the actual carrying capacity of the educational images. For example, a sample image is a math chapter overview image with 28 actual associated concepts, a maximum number of concepts set to 32, and a normalized target value of 0.875. Through fitting, the node carrying potential value after mapping the semantic coverage attenuation index of this image is made close to 0.875.

[0067] The node carrying capacity potential value is constrained to a range of zero to one, designed in conjunction with the concept carrying capacity limit of educational images. In educational scenarios, the number of concepts associated with a single teaching image is not infinite. Through the nonlinear mapping of the S-shaped logistic function, the difference between high-capacity and low-capacity images can be distinguished, while avoiding the impact of extreme values ​​on subsequent quota calculations. For example, in a local image carrying a single concept, the node carrying capacity potential value is close to 0.1, while in an overview image carrying multiple concepts, the node carrying capacity potential value is close to 0.9, clearly defining different carrying capacity ranges.

[0068] The design of the normalization target value includes setting a reasonable threshold for the maximum number of concepts to avoid deviations in calibration results caused by a few extreme multi-concept images. For example, setting the maximum number of concepts to 32, images exceeding this number will have their normalization target value uniformly set to one, ensuring that the calibration process focuses on the normal carrying range of most educational images and improving the accuracy of mapping.

[0069] The specific calibration of the scaling and bias calibration coefficients includes: obtaining them through offline training using the image sample dataset of the Midu Education Knowledge Graph. During training, the Adam optimization algorithm was used with a learning rate of 0.01, 500 iterations, a random seed of 0, and initial values ​​of α=1 and β=0. The optimal value for α after final calibration is 2.3, and the optimal value for β is 0.5. This result minimizes the squared error between the mapped node carrying potential value and the true concept coverage of the sample.

[0070] The maximum number of concepts was set to 32. When the number of concepts associated with various teaching images in middle school textbooks and courseware was statistically analyzed, it was found that 99% of the images had no more than 32 associated concepts. This value can cover the actual carrying capacity of the vast majority of educational images.

[0071] The optimization details for offline calibration include: setting the batch size to 256, traversing all samples in each iteration, adopting an early stopping strategy, stopping training when the squared error of the validation set no longer decreases after 10 consecutive iterations, and ensuring that the calibration results do not overfit.

[0072] Preferably, the calculation of the visual anchor point quota and concept entity quota for the teaching image based on the node carrying potential value includes:

[0073] The visual anchor point quantity quota and the concept entity quantity quota are calculated using the following formula:

[0074] ;

[0075] ;

[0076] in, This represents the node's carrying potential value; and These represent the preset minimum and maximum number of visual anchor points, respectively. and These represent the preset minimum and maximum number of concept entities, respectively. This represents the round-up operator; This indicates the quota of the number of visual anchor points; This indicates the quantity quota of the conceptual entity.

[0077] The minimum visual anchor point number constant is a preset lower limit fixed value for the number of visual anchor points, preferably 8. Educational images need at least 8 anchor points to cover core visual features, such as vertices and symbol positions in simple geometric figures, ensuring that no key concept anchor points are missed. The maximum visual anchor point number constant is a preset upper limit fixed value for the number of visual anchor points, preferably 64. This represents the concept carrying limit of educational images; exceeding 64 anchor points will lead to feature redundancy, adapting to the maximum anchor point requirements of complex circuit diagrams and chapter overview diagrams in middle school textbooks. The visual anchor point number quota is calculated based on the node carrying potential value and is used to limit the number of visual anchor points extracted in teaching images. The minimum concept entity number constant is a preset lower limit fixed value for the number of concept entities, preferably 2. A single teaching image should be associated with at least 2 core concepts, such as the physical quantities and calculation formulas associated with the diagrams in physics problems, to avoid insufficient concept extraction. The maximum concept entity number constant is a preset upper limit fixed value for the number of concept entities, preferably 32, to statistically analyze the number of concept associations in various teaching images in middle school textbooks and courseware. 99% of images are associated with no more than 32 concepts, covering the vast majority of scenarios. The concept entity quantity quota is calculated based on the node carrying potential value and is used to limit the number of concept items extracted from associated text. The node carrying potential value ranges from zero to one and represents the real value of the teaching image's potential capacity to carry concept entities.

[0078] The upper and lower limits for the number of anchor points and concepts include: Teaching images in middle school textbooks have clear knowledge-carrying patterns; simple local diagrams, such as function graphs in math examples, require fewer anchor points and have fewer concept connections; complex overview diagrams, such as system structure diagrams in biology chapters, require more anchor points and concepts. Setting an anchor point range of 8 to 64 and a concept range of 2 to 32 can accurately match this pattern. For example, in a diagram of a chemical experimental apparatus, the node carrying potential value is 0.6, and the visual anchor point quota is calculated as 8 + ⌈(64-8) × 0.6⌉ = 8 + 34 = 42, which covers the anchor points of each component of the apparatus without redundancy.

[0079] The rounding logic for quota calculation is based on the indivisible nature of concepts in educational scenarios. Even if the product result is an integer, rounding up ensures that at least the corresponding proportion of concepts are covered. For example, if the node carrying potential value is 0.25, the concept entity quota is 2 + ⌈(32-2)×0.25⌉ = 2 + 8 = 10. This avoids concept omissions caused by decimal quotas and ensures that each quota corresponds to the complete concept extraction requirements.

[0080] The node carries a linear mapping relationship between potential value and quota. By designing a minimum threshold and a proportional coefficient, the needs of low-potential and high-potential images are balanced. Low-potential images, such as local example images, can obtain basic quotas to meet core needs; high-potential images, such as chapter overview images, can obtain sufficient quotas to cover all related concepts.

[0081] The minimum number of visual anchor points is 8, and the maximum number of visual anchor points is 64. This range was determined by analyzing the anchor point requirements of 10,000 educational images of different disciplines and types, and can cover all scenarios from simple line drawings to complex system diagrams.

[0082] The minimum number of concept entities is 2, and the maximum number of concept entities is 32. Statistical analysis of teaching images in textbooks for subjects such as Chinese, mathematics, physics, chemistry, and biology in middle schools shows that 99% of the images are associated with a number of concepts between 2 and 32. This range can meet the concept extraction needs of the vast majority of educational images.

[0083] The boundary handling for rounding up includes: when the result of multiplying the difference between the minimum and maximum constants by the node carrying potential value is an integer, the result remains unchanged after rounding up; if it is a decimal, it is rounded up to the nearest integer. For example, (64-8)×0.5=28, after rounding up it is still 28, and the visual anchor quota is 8+28=36; (32-2)×0.3=9, after rounding up it is 9, and the concept entity quota is 2+9=11.

[0084] Preferably, the method of locating visual attention regions in the teaching image according to the quota of visual anchor points to construct a weighted image substructure includes:

[0085] Step A: Generate a visual attention heatmap and determine the set of visual anchor points using the following formula:

[0086] ;

[0087] ;

[0088] Wherein, it represents the feature tensor of a specific layer in the multi-layer convolutional response graph; and Represents the learnable convolution parameters; Represents the sigmoid activation function; This represents the aforementioned visual attention heatmap; This indicates the quota of the number of visual anchor points; Indicates according to Sort pixel positions lexicographically and select the first few. Operations at each position; Indicates the selected first The coordinates of each pixel position in the feature tensor; This represents the center coordinates of the visual attention region mapped back to the original space of the teaching image; Represents the set of visual anchor points;

[0089] Step B: Construct the weighted image substructure and calculate the edge weights using the following formula:

[0090] ;

[0091] ;

[0092] ;

[0093] in, This refers to the standardized teaching image; Indicates Centered on Perform trimming and filling operations based on the radius; Indicates the first A local image patch; This indicates that the node is embedded in the network; Represents the node feature vector; Representation of embedded relationships in the network; This represents a vector concatenation operation; Indicates from the anchor point Pointing to anchor point The relational feature vector; Represents the preset projection vector; Indicates inner product operation; This represents the node's carrying potential value; This indicates that the weighted image substructure starts from the anchor point. Pointing to anchor point The final weight of the edge.

[0094] The target layer index is a specific convolutional layer number selected from the multi-layer convolutional response map for generating the visual attention heatmap, preferably 3. This is because the features of the 3rd layer convolution can balance local details and mesoscale structure in educational images, adapting to the needs of concept anchor point localization, such as line features in function graphs and component outlines in circuit diagrams. Single-pixel convolution weights are learnable parameters used to weight feature channels in single-pixel convolution operations. Xavier initialization is preferred, as this ensures the parameter distribution adapts to the feature input range, avoiding gradient vanishing or exploding in the early stages of training. Single-pixel convolution bias is a learnable parameter used to adjust the output offset in single-pixel convolution operations. It is preferably initialized to 0, so no additional offset is needed in the initial state, and it can be adaptively optimized through training to adapt to the feature response range of educational images. The visual attention heatmap is a two-dimensional graph reflecting the conceptual association strength of each pixel location in the image after the target layer features are processed by single-pixel convolution and non-linear activation. The response score is the activation value of each pixel location in the visual attention heatmap, used for sorting and filtering key points. The visual anchor point quota limits the number of visual anchor points extracted. Keypoints are pixel positions selected from the visual attention heatmap in descending order of response score, with their number equal to the visual anchor point quota. Feature tensor coordinates are the spatial coordinates (x, y) of the keypoints within the target-level feature tensor. The original spatial center coordinates are the spatial coordinates after mapping the feature tensor coordinates back to the normalized teaching image, serving as the center of the visual attention region. The anchor point set is the set of all keypoints mapped to their original spatial center coordinates. The local image patch radius is the radius length of the local image cropped around the original spatial center coordinates, preferably 32. This radius covers the core area of ​​a single concept anchor point in the educational image, such as the vertices and surrounding lines of a geometric figure, or the complete outline of a chemical instrument. A local image patch is a fixed-size local image obtained by cropping and filling around the original spatial center coordinates with a set radius. The normalized teaching image is a standard teaching image after mean and variance normalization. The node embedding network is a neural network used to extract node feature vectors from the local image patches. A 3-layer convolutional structure with global average pooling is preferred, as this structure effectively extracts semantic features from the local image, adapting to the patch feature learning requirements of educational images. The relation embedding network is a neural network used to extract relation feature vectors from concatenated node feature vectors. It preferentially employs a two-layer fully connected network structure, which efficiently models the relationships between nodes and is suitable for both simple and complex logical relationships between concepts in educational images. Node feature vectors are high-dimensional vectors extracted from local image patches, representing the semantic information of the patch. Relation feature vectors are high-dimensional vectors extracted from the concatenation of each pair of node feature vectors, representing the relationships between nodes. A preset projection vector is a fixed high-dimensional vector used to calculate the inner product with the relation feature vectors, quantifying the strength of the relationship. It is preferably a random normally distributed vector with dimension 64 (mean 0, standard deviation 0.01), which balances computational complexity with the accuracy of relation representation.The inner product is the dot product of the relation feature vector and the preset projection vector, used to quantify the relation matching degree. The basic weight is obtained by applying a normalized exponential function to the inner product result, representing the strength of the basic association between nodes. The node carrying potential value is a real value representing the potential ability of the teaching image to carry conceptual entities. The final edge weight is the product of the basic weight and the node carrying potential value, representing the strength of the association between nodes in the weighted image substructure.

[0095] The selection of the target layer is based on the expressive rules of educational images. The low-level features of convolutional neural networks focus on local textures (such as lines and symbols), while the high-level features focus on global structures (such as overall layout). The third layer of features precisely covers the core expressive scale of the conceptual anchor points in educational images. For example, in a function overview graph in a math chapter, the third layer of features can simultaneously capture the local inflection points and mesoscale trends of the function curves, making the anchor point positioning more accurate.

[0096] The anchor point coordinate mapping rule uses a 4x scaling plus a 2-pixel offset. After the image in the educational document is downsampled by the convolutional network, the scale ratio between the feature tensor and the original image is 1:4. Adding a 2-pixel offset corrects the coordinate deviation caused by downsampling, ensuring that the anchor points are accurately aligned with the concept anchor points after being mapped back to the original image. For example, the coordinates of a resistor element in a physical circuit diagram are (20, 30) in the feature tensor, and after mapping, the original spatial coordinates are (82, 122), which are exactly aligned with the center of the element.

[0097] Edge weights consist of two components: node carrying potential, which reflects the overall conceptual coverage of the image, and base weight, which reflects the strength of local relationships between nodes. Multiplying these two values ​​ensures that the edge weights are both adapted to the strength of local relationships and to the overall carrying capacity of the image. For example, in a chapter overview image with high carrying potential, the edge weights are generally high, reflecting strong relationships between concepts; in a local example image with low carrying potential, the edge weights focus on core relationships and suppress noise.

[0098] The structure of the node and relationship embedding network includes: local image patches often contain the core information of a single concept (such as a test tube in a chemical experiment apparatus or a triangle in mathematics); 3 layers of convolution can fully extract the texture and shape features of the patch; and 2 layers of fully connected network can efficiently model the logical relationships between concepts (such as serial, parallel, and containment relationships).

[0099] The target level index is specifically set to 3, which corresponds to the 3rd layer of the convolutional neural network. Its feature map size is 256×256, and its theoretical receptive field scale is 9, which can accurately cover the mid-scale concept anchor points of educational images.

[0100] Training strategy for single-pixel convolution weights and biases: The Adam optimization algorithm is adopted, with a learning rate of 0.001. During the iteration process, the parameters are updated synchronously with those of other networks. The training objective is to minimize the KL divergence between the visual attention heatmap and the true distribution of concept anchors.

[0101] The specific value of the local image patch radius is 32, and the patch size after cropping is 64×64. This size can fully preserve the core features of a single concept anchor point, while avoiding redundant information and adapting to the input requirements of the node embedding network.

[0102] The preset projection vector has a specific dimension of 64, which is initialized and fixed using a random normal distribution (mean 0, standard deviation 0.01). This dimension can balance the accuracy of relation representation with computational efficiency, and is suitable for the quantitative needs of the relationship between concepts in educational images.

[0103] The structure of the node embedding network includes: a 64×64 local image patch as input, a first convolutional layer (3×3 kernel, 64 channels, stride 1), a second convolutional layer (3×3 kernel, 128 channels, stride 2), a third convolutional layer (3×3 kernel, 128 channels, stride 2), and a 128-dimensional node feature vector output after global average pooling.

[0104] The structure of the relation embedding network includes: input is a 256-dimensional concatenated node feature vector (two 128-dimensional vectors concatenated), a first fully connected layer (256→128 dimensions, ReLU activation), a second fully connected layer (128→64 dimensions), and output is a 64-dimensional relation feature vector.

[0105] The specific calculation of coordinate mapping includes: the x-value of the original spatial center coordinates = feature tensor x-coordinate × 4 + 2, the y-value = feature tensor y-coordinate × 4 + 2, where 4 is the scale ratio of the feature tensor to the original image, and 2 is the offset correction value to ensure accurate coordinate mapping.

[0106] Preferably, extracting concept items from associated text based on the concept entity quantity quota to establish cross-modal connections includes:

[0107] Step A: Construct the associated text and generate a set of concept candidates using the following formula:

[0108] ;

[0109] ;

[0110] in, A caption box indicating the image associated with the teaching image; This represents the five text blocks closest to the teaching image; Indicates optical character recognition operation; This represents a string concatenation operation; This refers to the associated text; Indicates to The set obtained by segmenting and enumerating continuous substrings of length within a preset range; Indicates when substring An indicator function that returns true if the character contains at least one Chinese character, letter, or number; This represents the candidate set of concepts;

[0111] Step B: Extract the concept feature vector and determine the concept item using the following formula:

[0112] ;

[0113] ;

[0114] in, Represents any candidate word in the concept candidate set; This represents a text convolutional neural network encoder. Represents the feature vector of the concept; Indicates the vector dimension; This indicates the quantity quota of the aforementioned conceptual entities; The Euclidean norm of a vector; Indicates by key value The candidate words are sorted lexicographically and the top words are selected. One operation; This represents the set of selected concept items.

[0115] The bound caption box is the rectangular area containing the caption text that is associated with the teaching image in the layout, obtained by matching the layout analysis results. The number of closest text blocks is a preset number of text blocks selected from the document layout that are closest to the teaching image, preferably 5. Based on the layout patterns of educational documents, the five text blocks surrounding the image are likely to contain concept descriptions related to the image, ensuring the relevance of the associated text. The set of closest text blocks is a specified number of text blocks selected from the document layout that are closest to the teaching image. The optical character recognition (OCR) result is a string obtained by recognizing the text content within the bound caption box and the closest text blocks. This can be obtained by recognizing the text block image using an OCR tool (such as Tesseract). The associated text is the complete text formed by concatenating the OCR results of the bound caption box and the OCR results of the set of closest text blocks. The delimiter set is a pre-defined combination of symbols used to segment related text, preferably spaces, commas, periods, semicolons, colons, parentheses, and line breaks, using common punctuation and formatting symbols found in educational texts to comprehensively segment the text into effective fragments. The concept candidate set is a set of candidate words obtained by enumerating all continuous substrings after segmenting the related text. The filtered concept candidate set is a set formed by retaining substrings containing Chinese characters, letters, or numbers from the concept candidate set, while removing meaningless symbol strings. The text convolutional neural network encoder is a neural network used to extract concept feature vectors from candidate words. A structure using character embeddings and multi-scale convolutional kernels is preferred, as this structure can effectively capture the semantic features of short educational concept texts and adapt to concept expressions of different lengths. The concept feature vector is a high-dimensional vector extracted from candidate words that represents the semantic information of the word. The vector dimension is the number of dimensions of the concept feature vector, preferably 128, as this dimension balances semantic representation accuracy and computational efficiency, adapting to the semantic complexity of educational concepts. The Euclidean norm is the L2 norm of a concept feature vector, used to quantify the magnitude of the vector and reflect the semantic feature strength of candidate words. The concept entity quantity quota is a numerical value that limits the number of concept items extracted from associated text. The concept item set is a set of candidate words selected from the filtered candidate concept set, sorted in descending order of Euclidean norm, with a quantity equal to the concept entity quantity quota. Cross-modal connection edges are graph edge structures used to connect teaching image nodes and concept item nodes, representing the relationship between them.

[0116] The construction of related text includes: the layout of educational documents follows the principle of image-text association; image captions directly explain the image content; and surrounding text blocks mainly revolve around the concepts involved in the image. For example, the caption of a physics circuit diagram is a schematic diagram of a series circuit, and the surrounding five text blocks contain descriptions of concepts such as resistance, current, and voltage. After being pieced together, they can comprehensively cover the relevant concepts in the image, ensuring the completeness of concept extraction.

[0117] The filtering rules for concept candidates include: educational concepts are mostly composed of Chinese characters, letters, or numbers, such as Newton's second law, DNA structure, and the Pythagorean theorem. Filtering meaningless symbol strings can reduce noisy candidates and improve the accuracy of concept extraction. For example, the symbol strings such as && and %% obtained after segmentation will be eliminated, and only valid candidates will be retained.

[0118] The concept ranking strategy employs a composite key combining Euclidean norm, length, and lexicographical order. The Euclidean norm reflects the strength of semantic features, the length matches a reasonable character range for educational concepts, and the lexicographical order resolves ranking conflicts. For example, when two candidate words have similar norms, the longer educational term (such as electromagnetic induction) will be selected first, ensuring the professionalism of the extracted concepts.

[0119] The structure of the text convolutional neural network encoder is adapted to the characteristics of short educational concept texts, which are mostly short texts of 2-20 characters. The use of multi-scale convolutional kernels (2 to 5) can capture the semantic information of concepts of different lengths. Character embedding can preserve glyph features and is adapted to educational concepts that mix Chinese and English (such as the working principle of CPU).

[0120] The specific number of text blocks in the main text is 5. This value was determined by analyzing the layout and associated text distribution of 10,000 educational images. Five text blocks can cover more than 95% of the relevant concept descriptions of the educational images.

[0121] The specific elements of the separator set are space, comma, period, semicolon, colon, parentheses, and line break. These symbols are the most commonly used segmentation symbols in educational texts and can comprehensively and without omission segment related texts.

[0122] The detailed structure of the text convolutional neural network encoder is as follows: The input is a sequence of characters of candidate words with a character embedding dimension of 64. The convolutional kernel sizes are 2, 3, 4, and 5, with 128 convolutional kernels of each size. After global max pooling, the kernels are concatenated and then mapped to a 128-dimensional concept feature vector through a linear layer.

[0123] The vector dimension is specifically set to 128, which controls the computational load while ensuring the accuracy of the semantic representation of educational concepts.

[0124] The length of the candidate concept substrings ranges from 2 to 20 characters. The character length of common concepts in the field of education is statistically analyzed, and 99% of the concept lengths fall within this range. This range can cover the vast majority of effective educational concepts and avoids overly short and meaningless substrings as well as overly long and redundant substrings.

[0125] Preferably, the initial triples are input into the convolutional graph completion model for relational reasoning, generating an educational knowledge graph data structure containing multimodal relational data. This optimizes the storage topology density of the multimodal graph data and accelerates the convergence of relational reasoning computation, including:

[0126] Step A: Calculate the predicted score of the candidate triples using the following formula:

[0127] ;

[0128] ;

[0129] ;

[0130] ;

[0131] in, The feature vector representing the head entity; Feature vectors representing relation types; and This represents the reshaped 2D width and height; This represents the two-dimensional feature map; Indicates a splicing operation; Represents a two-dimensional convolution operation; Indicates the convolution kernel parameters; Indicates a random deactivation operation; Represents the linear rectification activation function; Indicates the output of the convolutional layer; Indicates the flattening operation; Represents a linear projection matrix; Represents the projected feature vector; This represents a matrix containing the feature vectors of all candidate tail entities; Represents the sigmoid activation function; This represents the predicted score;

[0132] Step B: Merge the initial set of triples with the selected completed edges to generate the educational knowledge graph data structure. And store:

[0133] ;

[0134] in, Represents the initial set of triples; This indicates that based on the predicted score... Select the highest score The operation of completing edges; This indicates the preset quota for completing the task.

[0135] The initial triplet set is formed by collecting connection edges and cross-modal connection edges from the weighted image substructure, resulting in a triplet combination containing head entity, relation type, tail entity, and weight. The head entity feature vector is a high-dimensional semantic representation vector of the head entity in the initial triplet, extracted by a preceding node embedding network or a text convolutional neural network. The relation type feature vector is a high-dimensional semantic representation vector of the relation type in the initial triplet, preferably a 200-dimensional random normal distribution vector (mean 0, standard deviation 0.01). This dimension balances relation representation accuracy and computational efficiency, adapting to the relation complexity of educational knowledge graphs. The width of the two-dimensional feature map is the width dimension of the two-dimensional feature map formed by reshaping the head entity feature vector and relation type feature vector, preferably 10, which, in conjunction with the height of the two-dimensional feature map, fully unfolds the semantic information of entities and relations. The height of the two-dimensional feature map is the height dimension of the two-dimensional feature map formed by reshaping the head entity feature vector and relation type feature vector, preferably 20. A size of 10×20 ensures that the feature map is neither too sparse nor too dense, guaranteeing that the convolutional kernel effectively captures local interaction features. The 2D feature map is a 2D matrix formed by reshaping and concatenating the head entity feature vector and the relation type feature vector. It is used for convolution operations to capture the interaction features of entities and relations. The convolution kernel parameters are filter parameters used in the graph convolutional layer to extract local interaction features from the 2D feature map. A 3×3 kernel with 32 kernels is preferred, as it effectively captures the association information of adjacent features, and the number 32 balances feature extraction capability and computational cost. The number of feature maps is the number of feature maps output by the graph convolutional layer, preferably 32. This number covers various modes of entity and relation interaction, adapting to the diversity of relationships in educational knowledge graphs. The dropout rate is the random deactivation ratio used to prevent model overfitting, preferably 0.2 (feature map dropout) and 0.3 (projection dropout). This ratio effectively suppresses overfitting while preserving core feature information, adapting to the training requirements of graph completion tasks. The linear rectified activation function is an activation function used to enhance the non-linear expressive power of the model, outputting input values ​​greater than 0 and outputting 0 for input values ​​less than or equal to 0. The linear projection matrix is ​​used to map the flattened convolutional features back to the entity feature space. Ideally, the input dimension corresponds to a 200-dimensional output to match the entity embedding dimension, ensuring that the projected features can directly calculate the inner product with the tail entity feature vector. The flattening operation converts the two-dimensional convolutional feature map into a one-dimensional vector, preparing for linear projection. The projected feature vector is the one-dimensional vector output by the linear projection matrix, mapped back to the entity feature space, used to calculate similarity with the candidate tail entity feature vector. The candidate tail entity feature matrix is ​​a matrix containing all candidate tail entity feature vectors in the educational knowledge graph, accumulated from the previous feature extraction process. The inner product is the dot product of the projected feature vector and the candidate tail entity feature vector, used to quantify their semantic similarity.The sigmoid activation function is a non-linear function that maps the inner product result to a value between 0 and 1, used to generate predicted scores for candidate triples. The predicted score of a candidate triple is a probability value output by the sigmoid activation function, characterizing the validity of the candidate triple; a higher score indicates a more reasonable triple. The completion quota is a preset number of edges to complete, preferably 200,000, to cover most missing relationships in the educational knowledge graph while avoiding excessive completion leading to structural redundancy. The completion edges are a preset number of triples with the highest predicted scores selected from the candidate triples, used to supplement the missing relationships in the initial triple set. The final educational knowledge graph is an attribute graph data structure formed by merging the initial triple set and the completion edges, containing complete multimodal association data.

[0136] The completion priority of educational knowledge graph triples includes: the completion process prioritizes educational-specific relationships, such as the association between images and concepts, the association between concepts and exercises, and the prerequisite relationships between knowledge points, rather than random completion of general knowledge graphs. For example, if an initial triple contains a connection between a physical circuit diagram and the concept of resistance, the completion process will prioritize predicting the connection between the circuit diagram and related concepts such as current and voltage, ensuring that the completed relationships align with the needs of the teaching scenario.

[0137] The reshaping rules for 2D feature maps include reshaping entity and relation embeddings to a size of 10×20, which is designed based on the semantic complexity of entities and relations in the education domain. The semantic interactions of educational entities (such as mathematical concepts and physical laws) and relations (such as inclusion, derivation, and application) tend to be localized. This reshaping size enables the convolutional kernel to accurately capture these local interaction patterns, improving prediction accuracy.

[0138] Weighted completion strategies include: the weights of the initial triples (such as the edge weights of weighted image substructures) are included in the loss calculation, making the model more inclined to retain high-weight core relationships and weaken low-weight secondary relationships during completion. For example, high-weight function graph-function monotonicity associations will guide the model to prioritize completing the relationship between the function graph and related core concepts such as derivatives and extrema, ensuring completion quality.

[0139] The quota for the number of completed edges includes: the number of entities and relations in the knowledge graph of secondary education is limited, and the quota of 200,000 can cover the vast majority of missing relations, while avoiding the imbalance of graph structure density due to too many completed edges, thus balancing completeness and simplicity.

[0140] The width of the two-dimensional feature map is 10 and the height is 20. This size is determined by combining the entity embedding dimension of 200. The 200-dimensional embedding can be divided into a 10×20 two-dimensional structure to ensure that no information is lost during the reconstruction process.

[0141] The specific specifications of the convolution kernel parameters are 3×3 in size and 32 in number. They are initialized using a random normal distribution (mean 0, standard deviation 0.01) and are adaptively updated through backpropagation during training to adapt to the extraction of interactive features of entities and relationships.

[0142] The specific allocation of the dropout rate includes: a feature map dropout rate of 0.2 and a linear projection dropout rate of 0.3. This allocation method can specifically suppress overfitting at different stages and ensure the model's generalization ability.

[0143] The specific dimension of the linear projection matrix is ​​the same as the dimension of the output after the convolutional features are flattened, which corresponds to 200 dimensions. For example, after 32 3×3 convolutional kernels process a 10×20 input, the flattened dimension is 32×(10-3+1)×(20-3+1)=32×8×18=4608, and the projection matrix dimension is 4608×200, ensuring that the mapped dimension is consistent with the entity embedding dimension.

[0144] The specific value of the completion quota is 200,000. This value is determined by statistically analyzing the number of entities (approximately 10,000) and the average number of relations in the secondary education knowledge graph. This value can cover more than 95% of reasonable missing relations and avoid insufficient or redundant completion.

[0145] The entity and relation embeddings are initialized using a random normal distribution (mean 0, standard deviation 0.01) with a dimension of 200. This initialization method ensures that the embedding vectors are initially uniformly distributed, guaranteeing stable convergence during training.

[0146] Example 2: A method for constructing an educational knowledge graph based on artificial intelligence, applied to any of the aforementioned systems for constructing an educational knowledge graph based on artificial intelligence, comprising:

[0147] Multi-layer convolutional response maps of standardized teaching images are extracted using convolutional neural networks, and the response energy distribution of the multi-layer convolutional response maps is statistically analyzed. A semantic coverage attenuation index that reflects the change of image information with the theoretical receptive field scale is calculated.

[0148] The semantic coverage attenuation index is mapped to the node carrying potential value, and the visual anchor point quota and concept entity quota for the teaching image are calculated based on the node carrying potential value.

[0149] Based on the quota of visual anchor points, the visual attention region is located in the teaching image to form a weighted image substructure, and based on the quota of conceptual entities, conceptual items are extracted from the associated text to establish cross-modal connections.

[0150] The initial triples are input into the convolutional graph completion model for relational reasoning, generating an educational knowledge graph data structure containing multimodal relational data. This optimizes the storage topology density of the multimodal graph data and accelerates the convergence of relational reasoning computation.

[0151] It should be noted that the minimum number of visual anchor points is a constant. The value ranges from 4 to 10, representing the maximum number of visual anchor points. The value range is 32-128; the minimum concept entity quantity constant. The value ranges from 1 to 3, representing the maximum number of conceptual entities. The value range is 16-64. The values ​​are based on statistical analysis of the MEduKG dataset, covering 100,000 teaching images from various educational scenarios including primary, secondary, and vocational education. 99% of the images have a concept association count between 1 and 64, and an anchor point requirement between 4 and 128. Secondary school textbook images are preferred due to their moderate knowledge density. , , , Primary school textbooks can be selected , , , Vocational education courseware can be selected , , , .

[0152] It should be noted that the uniform resolution constant The value range is 512-2048, with 1024 being preferred. To balance feature extraction accuracy and computational efficiency, 512 is suitable for low-configuration devices or low-resolution scanned documents, 1024 is suitable for mainstream educational documents (textbooks, courseware, question banks), and 2048 is suitable for ultra-high-definition educational images (such as high-precision experimental device diagrams and complex circuit diagrams). It needs to match the receptive field scale of the subsequent convolutional neural network. 1024 corresponds to 5 convolutional layers in the EduCNN-5 network, with a receptive field covering 3-33 pixels, which can fully capture the local details and global structure of educational images; the original image resolution is higher than... When, scale proportionally to the length of the side. lower than When zero values ​​are filled in, .

[0153] It should be noted that the sample set consists of educational image samples from the MEduKG dataset, covering 12 subjects including mathematics, physics, chemistry, biology, Chinese language, and history, with a sample size of 10,000 images. Among them, images from the secondary school stage account for 70%, images from the primary school stage account for 20%, and images from the vocational education stage account for 10%. The number of real concepts was manually labeled, and the labeling accuracy rate reached over 98% after 3 rounds of cross-validation.

[0154] It should be noted that the normalized target value ,in The value is fixed at 64 (the maximum number of concept associations covering all educational scenarios), rather than the maximum number of true concepts in the sample; scaling coefficients. The preferred value is 2.3, which is the bias calibration coefficient. The preferred value is 0.5, which is obtained by calibration using the Adam optimization algorithm (learning rate 0.01, 500 iterations, random seed 0).

[0155] It should be noted that the supplementary quantity quota Positively correlated with the number of entities in the graph, the fitting formula is: ( (This refers to the total number of entities in the knowledge graph); the total number of entities in the secondary education knowledge graph is approximately 10,000, therefore, the preferred option is... Ensure that over 95% of reasonable missing relationships are covered. This applies to small-scale graphs with fewer than 5000 entities (e.g., a graph for a specific subject in elementary school). Take 100,000; large-scale maps with a total number of entities greater than 20,000 (such as the full-subject map of vocational education). Take 400,000; the core adjustment is to balance the completeness of the completion with the risk of redundancy, and avoid... Excessive size leads to a bloated graph structure.

[0156] It should be noted that the deep convolutional neural network used is the custom architecture EduCNN-5, which does not contain pooling layers but includes batch normalization (BN) layers. The configuration of each layer is as follows: Layer 1 (3×3 kernel, stride 1, 32 channels, BN layer + ReLU), Layer 2 (3×3 kernel, stride 2, 64 channels, BN layer + ReLU), Layer 3 (3×3 kernel, stride 2, 128 channels, BN layer + ReLU), Layer 4 (3×3 kernel, stride 2, 256 channels, BN layer + ReLU), Layer 5 (3×3 kernel, stride 1, 256 channels, BN layer + ReLU). The increasing number of channels is to capture more complex educational image features (from line texture to overall structure), and the absence of pooling layers is to preserve accurate spatial coordinate information to meet the needs of concept anchor point localization.

[0157] It should be noted that the supplementary node embedding network uses ReLU as the activation function. The first convolutional layer (3×3 kernel, 64 channels, stride 1), the second convolutional layer (3×3 kernel, 128 channels, stride 2), and the third convolutional layer (3×3 kernel, 128 channels, stride 2) output a 128-dimensional vector after global average pooling. The design is based on the fact that a 64×64 local image patch contains core features of a single concept, the three convolutional layers can fully extract texture and shape information, and the 128-dimensional vector can balance representation accuracy and computational cost.

[0158] It should be noted that the supplementary relation embedding network has the following activation function: ReLU; a fully connected first layer (256-dimensional input, 128-dimensional output) and a fully connected second layer (128-dimensional input, 64-dimensional output). The design is based on the fact that the relationships between concepts in educational images are mainly simple logical relationships (inclusion, association, and deduction). The two-layer fully connected network can efficiently model these relationships, and the 64-dimensional vector can adapt to the quantification requirements of the relationships.

[0159] It should be noted that the architecture of the text convolutional neural network encoder includes four convolutional layers (with kernel sizes of 2, 3, 4, and 5, each size containing 128 kernels, stride 1, and ReLU activation), a global max-pooling layer, and one linear layer (512-dimensional input, 128-dimensional output, and ReLU activation); the character embedding dimension is 64, and the character table size is 8192 (Unicode code points modulo). The multi-scale convolutional kernels are suitable for short educational concept texts of 2-20 characters, global max-pooling can extract the most critical semantic features, and the 128-dimensional output vector has the same dimension as the feature vector of the visual side nodes, facilitating cross-modal fusion.

[0160] It should be noted that the model structure of the convolutional graph completion model includes: one graph convolutional layer (3×3 convolutional kernels, 32 layers, stride 1, ReLU activation), a feature map dropout layer (rate 0.2), a flattening layer, a linear projection layer (input dimension = 32×18×18 = 10368, output dimension 200), and a projection dropout layer (rate 0.3). The convolutional kernels are initialized using a random normal distribution (mean 0, standard deviation 0.01). The 3×3 convolutional kernels can capture the local interaction features of entities and relationships, the 32 feature maps cover multiple interaction modes, and the 200-dimensional output vector has the same dimension as the entity embedding, ensuring the effectiveness of the inner product calculation.

[0161] It should be noted that the calculation of the theoretical receptive field scale includes: the theoretical receptive field scale is calculated using a recursive formula, with initial values... (Input layer receptive field), initial cumulative stride ;No. Layered sensory field , No. Layer cumulative stride ,in For the first Layer convolution kernel size, For the first Layer stride. The theoretical receptive field of each layer in the EduCNN-5 network is... , , , , Each level corresponds to a preset level, ensuring that the receptive field scale of each level is clearly defined.

[0162] It should be noted that the target layer is fixed at layer 3, which theoretically has a receptive field of 9 pixels. This layer can capture both local details (such as symbols and line inflection points) and mesoscale structures (such as the core area of ​​a single concept anchor) in educational images, thus adapting to the needs of concept anchor positioning. It does not dynamically adjust with image type because layer 3 features can stably capture concept anchors in multidisciplinary educational images (function graphs, circuit diagrams, schematic diagrams), and consistent selection ensures process consistency.

[0163] It should be noted that the number of related text blocks is fixed, including: a preset fixed number of 5. Based on the layout analysis and statistics of 10,000 multidisciplinary educational images, 5 nearest text blocks can cover more than 95% of the relevant concept descriptions of the images. Fewer than 3 blocks will miss key information, and more than 7 blocks will introduce redundancy. The distance is the Euclidean distance between the center of the image frame and the center of the text block frame in the document layout, calculated using the following formula: ,in The coordinates of the center of the image frame. The coordinates are the center coordinates of the text block box.

[0164] It should be noted that the weight settings for cross-modal connection edges include: the default weight of cross-modal connection edges is 1; or it is calculated based on the cosine similarity between the concept feature vector and the image feature vector, using the following formula: ,in The average value of the feature vectors of all nodes in the image. The cosine similarity function is used. Cross-modal connection edge weights and weighted image substructure edge weights are stored independently and participate equally in loss calculation during graph completion, collaboratively influencing the relationship inference results.

[0165] It should be noted that the optimization algorithm is limited to Adam, the learning rate is fixed at 0.001, and the number of iterations is fixed at 500. An early stopping strategy is adopted: training stops if the squared error on the validation set does not decrease for 10 consecutive iterations, and the model parameters with the smallest validation set error during the iteration process are saved. The Adam optimization algorithm is suitable for training multi-parameter models, the fixed learning rate and number of iterations ensure training stability, and the early stopping strategy avoids overfitting.

[0166] The loss function used in the knowledge graph completion model is the weighted binary cross-entropy loss function. Weighted loss prioritizes the training of high-weight core relationships, while label smoothing alleviates overfitting and adapts to the relationship distribution characteristics of educational knowledge graphs.

[0167] The training objectives for visual attention heatmaps include: calculating the KL divergence to predict the heatmap. Compared with the actual anchor point distribution ,in This is the normalized energy distribution of manually labeled concept anchor points in the MEduKG dataset (anchor point has an energy of 1, other locations have an energy of 0, and the sum after normalization is 1). Real anchor points were labeled by three experts in the field of education, and the labels were the core locations of the corresponding concepts in the images (e.g., the center of a resistor in a circuit diagram, or an inflection point in a function graph). The labeling results were used as training targets after a consistency check (Kappa coefficient ≥ 0.85).

[0168] The length of the candidate concept substrings ranges from 2 to 20 characters (Chinese / English). Based on statistical analysis of common concepts in the education field, 99% of Chinese education concepts are between 2 and 20 characters in length, while English concepts (such as professional terms and formula abbreviations) are slightly longer. Therefore, the length is appropriately extended to ensure that no valid concepts are missed, while avoiding excessively long and redundant substrings (such as complete sentence fragments).

[0169] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. An artificial intelligence-based educational knowledge graph construction system, characterized in that, include: The multi-scale representation analysis unit is configured to extract multi-layer convolutional response maps of standardized teaching images using convolutional neural networks, statistically analyze the response energy distribution of the multi-layer convolutional response maps, and calculate the semantic coverage attenuation index that reflects the change of image information with the theoretical receptive field scale. The structural parameter configuration unit is configured to map the semantic coverage attenuation index to a node carrying potential value, and to calculate the visual anchor point quota and concept entity quota for the standardized teaching image based on the node carrying potential value. The multimodal primitive assembly unit is configured to locate visual attention regions in the normalized teaching image according to the quota of the number of visual anchors to form a weighted image substructure, and to extract concept items from the associated text according to the quota of the number of concept entities to establish cross-modal connections. The intelligent link optimization unit is configured to input the initial triplet into the convolutional graph completion model for relational reasoning, generate an educational knowledge graph data structure containing multimodal relational data, thereby optimizing the storage topology density of the multimodal graph data and accelerating the convergence of relational reasoning computation; Extracting multi-layer convolutional response maps of normalized instructional images using convolutional neural networks, including: The original document page is processed by grayscale conversion, proportional scaling and edge filling to generate a page image with uniform resolution. The teaching image area is cropped by performing layout analysis on the page image. The teaching image area is then normalized by removing the mean and normalizing the variance to obtain the standardized teaching image. The standardized teaching image is input into a deep convolutional neural network, which contains multiple sequentially connected convolutional layers, each configured with a convolutional kernel size and stride parameter; The output feature tensor of the deep convolutional neural network at a preset level is extracted as the multi-layer convolutional response map, wherein the convolutional response map of each preset level corresponds to a theoretical receptive field scale on the normalized teaching image; The response energy distribution of the multi-layer convolutional response maps is statistically analyzed, and a semantic coverage attenuation index reflecting the change of image information with the theoretical receptive field scale is calculated, including: For each preset level of the multi-layer convolutional response map, the sum of squares of all channel response values ​​at each spatial location is calculated to obtain the local response energy. The local response energy is then divided by the total energy of the entire map at that level to generate a normalized response energy distribution. The negative logarithm of the sum of squares of the response energy distribution is then calculated to obtain the second-order Rayleigh entropy value. Obtain the logarithmic value of the theoretical receptive field scale corresponding to each preset level, and establish a corresponding sequence of the second-order Rayleigh entropy value with respect to the logarithmic value of the theoretical receptive field scale; The corresponding sequence is fitted with linear regression using the least squares method, the slope of the regression line is calculated, and the negative of the slope is used as the semantic coverage decay index.

2. The artificial intelligence-based educational knowledge graph construction system according to claim 1, characterized in that, Mapping the semantic coverage decay index to a node carrying potential value includes: The semantic coverage attenuation index is weighted using preset scaling coefficients and then superimposed with preset bias scaling coefficients to obtain a linear transformation result. The linear transformation result is input into an S-type logic function for nonlinear mapping to calculate a real value between zero and one, and this real value is used as the node carrying potential value.

3. The artificial intelligence-based educational knowledge graph construction system according to claim 2, characterized in that, The calculation of the visual anchor point quota and concept entity quota for the standardized teaching image based on the node carrying potential value includes: Obtain the preset minimum visual anchor number constant and maximum visual anchor number constant, calculate the difference between the minimum visual anchor number constant and the maximum visual anchor number constant and multiply it by the node carrying potential value, perform an up rounding operation on the obtained product result, and add the operation result to the minimum visual anchor number constant to obtain the visual anchor number quota. Obtain the preset minimum concept entity quantity constant and maximum concept entity quantity constant, calculate the difference between the minimum concept entity quantity constant and the maximum concept entity quantity constant and multiply it by the node carrying potential value, perform an up rounding operation on the obtained product result, and add the operation result to the minimum concept entity quantity constant to obtain the concept entity quantity quota.

4. The artificial intelligence-based educational knowledge graph construction system according to claim 3, characterized in that, Based on the specified number of visual anchor points, the visual attention region is located in the standardized teaching image to construct a weighted image substructure, including: Select the target layer features in the multi-layer convolution response map, perform single-pixel convolution and non-linear activation operations on the target layer features to generate a visual attention heatmap, and sort all pixel positions in the visual attention heatmap in descending order according to the response score. Select pixel positions with a number equal to the visual anchor point quota as key points, and map the coordinates of the key points back to the original space of the normalized teaching image to determine the center coordinates of the visual attention region. Using the center coordinates as a reference, a local image patch is cropped from the standardized teaching image, and the local image patch is input into a node embedding network to extract node feature vectors. For each pair of node feature vectors, they are concatenated and input into a relation embedding network to extract relation feature vectors. Calculate the inner product of the relation feature vector and the preset projection vector, perform a normalized exponential function operation on the inner product result to obtain the basic weight, and multiply the basic weight by the node carrying potential value to obtain the final weight of the connecting edge in the weighted image substructure.

5. The artificial intelligence-based educational knowledge graph construction system according to claim 4, characterized in that, Extracting concept items from associated text based on the aforementioned concept entity quantity quota to establish cross-modal connections includes: Obtain the caption text bound to the standardized teaching image and a preset number of text blocks in the document layout that are closest to the standardized teaching image; then concatenate the optical character recognition results of the caption text and the text blocks to generate the associated text. The associated text is segmented using a preset set of delimiters. All continuous substrings in the segmented fragments are enumerated, and substrings containing Chinese characters, letters, or numbers are retained to form a concept candidate set. Each candidate word in the concept candidate set is input into a text convolutional neural network to extract concept feature vectors. Calculate the Euclidean norm of the concept feature vector, sort the candidate words in the concept candidate set in descending order according to the value of the Euclidean norm, select the candidate words whose number is equal to the quota of the number of concept entities as the concept item, and establish a cross-modal connection edge from the normalized teaching image to the concept item.

6. The artificial intelligence-based educational knowledge graph construction system according to claim 5, characterized in that, The initial triples are input into the convolutional graph completion model for relational reasoning, generating an educational knowledge graph data structure containing multimodal relational data. This optimizes the storage topology density of the multimodal graph data and accelerates the convergence of relational reasoning computation, including: Collect the connection edges in the weighted image substructure and the cross-modal connection edges to construct an initial set of triples. Reshape the dimension of the head entity feature vector and the relation type feature vector in each triple and concatenate them to form a two-dimensional feature map. Two-dimensional convolution operation is performed on the two-dimensional feature map using a graph convolutional layer. After applying a linear rectified activation function and a random deactivation operation to the convolution result, the result is flattened and mapped back to the feature space through a linear projection layer to obtain the projected feature vector. The inner product of the projected feature vector and the feature vectors of all candidate tail entities in the graph is calculated. The predicted scores of the candidate triples are obtained by performing a sigmoid activation function operation. The preset number of complete edges with the highest predicted scores are selected and written into the initial triple set. The expanded set is then converted into an attribute graph data structure for persistent storage.

7. A method for constructing an educational knowledge graph based on artificial intelligence, applied to an educational knowledge graph construction system based on artificial intelligence as described in any one of claims 1-6, characterized in that, include: Multi-layer convolutional response maps of standardized teaching images are extracted using convolutional neural networks, and the response energy distribution of the multi-layer convolutional response maps is statistically analyzed. A semantic coverage attenuation index that reflects the change of image information with the theoretical receptive field scale is calculated. The semantic coverage attenuation index is mapped to the node carrying potential value, and the visual anchor point quota and concept entity quota for the standardized teaching image are calculated based on the node carrying potential value. Based on the quota of visual anchor points, the visual attention region is located in the normalized teaching image to form a weighted image substructure, and based on the quota of conceptual entities, conceptual items are extracted from the associated text to establish cross-modal connections. The initial triples are input into the convolutional graph completion model for relational reasoning, generating an educational knowledge graph data structure containing multimodal relational data. This optimizes the storage topology density of the multimodal graph data and accelerates the convergence of relational reasoning computation.

Images