Methods, apparatus, and equipment for analyzing knowledge mastery based on knowledge competitions

Abstract

This application provides a method, apparatus, and equipment for analyzing knowledge point mastery based on knowledge competitions, relating to the field of computer technology. In this application, firstly, target answer data and target answer images corresponding to the target personnel are acquired; secondly, feature mining is performed on the target answer data and target answer images respectively, outputting target answer data features and target answer image features; then, multi-focus correlation mining is performed on the target answer data features and target answer image features, outputting target answer fusion features; finally, based on the target answer fusion features, the target knowledge point mastery is analyzed and output. Based on the above, the relatively low reliability of knowledge point mastery analysis in existing technologies can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, and device for analyzing the mastery of knowledge points based on knowledge competitions. Background Technology

[0002] Knowledge competitions can help understand personnel's grasp of safety production knowledge, thus ensuring production safety. However, traditional methods for analyzing this knowledge mastery rely primarily on paper-based quizzes or single data analysis tools, such as quiz scores and response time, to assess students' or test-takers' knowledge. These methods have limitations, particularly in the accuracy and comprehensiveness of knowledge acquisition. They often fail to fully reflect the details of the participants' responses, neglecting deeper information gathered during the process. Therefore, relying solely on quiz data for knowledge mastery analysis has low reliability. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, apparatus and equipment for analyzing the mastery of knowledge points based on knowledge competitions, so as to improve the problem that the reliability of the analysis of the mastery of knowledge points in the prior art is relatively low.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] A method for analyzing knowledge point mastery based on knowledge competitions includes:

[0006] Obtain target answer data and target answer image corresponding to the target person, wherein the target answer data includes at least the content of the target person's answer based on the knowledge competition questions, and the target answer image includes at least the facial image of the target person during the answering process;

[0007] Feature mining is performed on the target answer data and the target answer image respectively, and the target answer data features and target answer image features are output;

[0008] The target answer data features and the target answer image features are subjected to multi-focus correlation mining to output target answer fusion features. Here, multi-focus refers to multiple answer time points or multiple answer contents in the answer process.

[0009] Based on the aforementioned target answer fusion features, the mastery of the target knowledge points is analyzed and output.

[0010] In a preferred embodiment of this application, in the aforementioned method for analyzing knowledge point mastery based on knowledge competitions, the step of performing feature mining based on the target answer data and the target answer image respectively, and outputting the target answer data features and the target answer image features, includes:

[0011] The target answer data and the target answer image are loaded into the target analysis network, wherein the target analysis network includes a feature mining sub-network, and the feature mining sub-network includes a first mining branch and a second mining branch with a network architecture different from the first mining branch;

[0012] Through the first mining branch, the target answer data is subjected to first mining to obtain target answer data features, wherein the first mining includes at least one of internal association mining of answer content and external association mining of answer content;

[0013] The target answer image is mined using the second mining branch to obtain its features. The second mining includes internal correlation mining and external correlation mining of the answer image.

[0014] In a preferred embodiment of this application, in the aforementioned method for analyzing knowledge point mastery based on knowledge competitions, the step of performing a first mining operation on the target answer data through the first mining branch to obtain the target answer data features includes:

[0015] The target answer data is loaded into the first mining branch;

[0016] Each response in the target answer data is embedded to form a response content embedding feature corresponding to each response. The response content has a one-to-one correspondence with the questions in the knowledge competition, and the response content includes the reference answer.

[0017] Cluster the embedded features of each of the responses to form at least one central feature;

[0018] For each of the aforementioned response content embedding features, based on the central feature corresponding to the response content embedding feature, association mining is performed on the response content embedding feature to form the response content association feature corresponding to the response content embedding feature.

[0019] By concatenating the associated features of each of the aforementioned responses, the target response data features are obtained.

[0020] In a preferred embodiment of this application, in the aforementioned method for analyzing knowledge point mastery based on knowledge competitions, the step of performing a second mining operation on the target answer image through the second mining branch to obtain the target answer image features includes:

[0021] The target answer image is loaded into the second mining branch;

[0022] Each local answer image in the target answer image is convolved to form a local answer image feature corresponding to each local answer image, wherein there is a one-to-one correspondence between the local answer image and the question in the knowledge competition;

[0023] Internal association mining is performed on each of the local answer image features to form image internal association features corresponding to each local answer image feature. In the process of internal association mining, after mining the local answer image features at different depths of semantic features, association mining is performed on the mined semantic features at different depths of semantic features.

[0024] External association mining is performed on each of the image internal association features to form an image external association feature corresponding to each image internal association feature. In the process of external association mining, after clustering each of the image internal association features, association mining is performed on each of the image internal association features based on the corresponding central features.

[0025] By concatenating the external features of each of the images, the target answer image features are obtained.

[0026] In a preferred embodiment of this application, in the aforementioned method for analyzing knowledge point mastery based on knowledge competitions, the step of performing multi-focus correlation mining on the target answer data features and the target answer image features to output target answer fusion features includes:

[0027] The target answer data features and the target answer image features are respectively segmented by sliding windows to form multiple sliding window answer data features and multiple sliding window answer image features. The target answer data features and the target answer image features have the same size, and the sliding window segmentation methods corresponding to the two sliding window segmentation methods are the same.

[0028] During the sequential fusion of the multiple sliding window answer data features, the multiple sliding window answer image features are fused sequentially to output the target answer fusion feature.

[0029] In a preferred embodiment of this application, in the aforementioned method for analyzing knowledge point mastery based on knowledge competitions, the step of sequentially fusing multiple sliding window answer image features and outputting the target answer fusion feature during the sequential fusion of the multiple sliding window answer data features includes:

[0030] In the first fusion stage, based on the features of the first sliding window answer data, the features of the second sliding window answer data are correlated and mined to form the answer data fusion features of the first fusion stage. Based on the features of the first sliding window answer image, the features of the second sliding window answer image are correlated and mined to form the answer image fusion features of the first fusion stage. Based on the answer image fusion features, the answer data fusion features are correlated and mined to form the answer fusion features of the first fusion stage.

[0031] In the second and subsequent fusion stages, the target answer fusion features are determined based on the sliding window answer data features and sliding window answer image features of the second and subsequent stages, combined with the answer image fusion features and answer fusion features of the first fusion stage.

[0032] In a preferred embodiment of this application, in the aforementioned method for analyzing knowledge point mastery based on knowledge competitions, the step of determining the target answer fusion features in the second and subsequent fusion stages, based on the features of the sliding window answer data and the sliding window answer image, and combined with the answer image fusion features and answer fusion features from the first fusion stage, includes:

[0033] For each subsequent fusion stage, based on the answer fusion features of the previous fusion stage, the sliding window answer data features of the current fusion stage are correlated and mined to form the answer data fusion features of the current fusion stage. Based on the answer image fusion features of the previous fusion stage, the sliding window answer image features of the current fusion stage are correlated and mined to form the answer image fusion features of the current fusion stage. Based on the answer image fusion features of the current fusion stage, the answer data fusion features of the current fusion stage are correlated and mined to form the answer fusion features of the current fusion stage.

[0034] Based on the answer fusion characteristics of the last fusion stage, the target answer fusion characteristics are determined.

[0035] In a preferred embodiment of this application, the aforementioned method for analyzing knowledge point mastery based on knowledge competitions also includes:

[0036] Acquire training answer data and training answer images, wherein the training answer data includes at least the content of the answers to knowledge competition questions, and the training answer images include at least the facial images during the answering process;

[0037] Using a candidate analysis network, feature mining is performed on the training answer data and the training answer image respectively, outputting training answer data features and training answer image features. In addition, the training answer data features and the training answer image features are subjected to multi-focus association mining, outputting training answer fusion features. Based on the training answer fusion features, the mastery of training knowledge points is analyzed and output. The candidate analysis network is a neural network model to be trained.

[0038] Based on the error between the mastery status of the training knowledge points and the knowledge point mastery status labels, the network parameters of the candidate analysis network are updated to form the target analysis network.

[0039] This application also provides a device for analyzing knowledge point mastery based on knowledge competitions, which further includes:

[0040] Acquire training answer data and training answer images, wherein the training answer data includes at least the content of the answers to knowledge competition questions, and the training answer images include at least the facial images during the answering process;

[0041] Using a candidate analysis network, feature mining is performed on the training answer data and the training answer image respectively, outputting training answer data features and training answer image features. In addition, the training answer data features and the training answer image features are subjected to multi-focus association mining, outputting training answer fusion features. Based on the training answer fusion features, the mastery of training knowledge points is analyzed and output. The candidate analysis network is a neural network model to be trained.

[0042] Based on the error between the mastery status of the training knowledge points and the knowledge point mastery status labels, the network parameters of the candidate analysis network are updated to form the target analysis network.

[0043] Based on the above, this application also provides an electronic device, including:

[0044] Memory, used to store computer programs;

[0045] A processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned method for analyzing the mastery of knowledge points based on knowledge competitions.

[0046] The method, apparatus, and equipment provided in this application for analyzing knowledge point mastery based on knowledge competitions first acquire target answer data and target answer images corresponding to the target personnel; second, feature mining is performed on the target answer data and target answer images respectively, outputting target answer data features and target answer image features; then, multi-focus correlation mining is performed on the target answer data features and target answer image features to output target answer fusion features; finally, the target knowledge point mastery is analyzed and output based on the target answer fusion features. Based on the above, by fusing answer images on the basis of answer data, and extracting data such as the respondent's facial expressions from the answer images, it is possible to focus on the respondent's psychological state and attention distribution during the answering process. This information helps to more comprehensively assess their mastery of knowledge points, thus improving the problem of relatively low reliability in the analysis of knowledge point mastery in existing technologies. Attached Figure Description

[0047] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings.

[0048] Figure 1 A structural block diagram of an electronic device provided in an embodiment of this application.

[0049] Figure 2 This is a flowchart illustrating the knowledge point mastery analysis method based on knowledge competition provided in an embodiment of this application.

[0050] Figure 3 This is a schematic diagram illustrating multi-stage feature fusion provided in an embodiment of this application.

[0051] Figure 4 A block diagram of a knowledge point mastery analysis device based on a knowledge competition provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0054] like Figure 1 As shown in the illustration, this application provides an electronic device. The electronic device may include a memory, a processor, and a knowledge point mastery analysis device based on a knowledge competition.

[0055] Specifically, the memory and the processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, the memory and the processor can be electrically connected via one or more communication buses or signal lines. The knowledge point mastery analysis device based on knowledge competition includes at least one software functional module stored in the memory in the form of software or firmware. The processor is used to execute executable computer programs stored in the memory, such as the software functional modules and computer programs included in the knowledge point mastery analysis device based on knowledge competition, to implement the knowledge point mastery analysis method based on knowledge competition provided in this application embodiment.

[0056] Optionally, the memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0057] Furthermore, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0058] Understandable. Figure 1 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 1 The more or fewer components shown, or having the same Figure 1The different configurations shown may include, for example, a communication unit for exchanging information with other devices (such as image acquisition devices).

[0059] Combination Figure 2 This application also provides a method for analyzing knowledge point mastery based on a knowledge competition, applicable to the aforementioned electronic device. The method steps defined in the relevant process of the knowledge point mastery analysis method based on the knowledge competition can be implemented by the electronic device. The following will describe... Figure 2 The specific process shown will be explained in detail.

[0060] Step S110: Obtain the target answer data and target answer image corresponding to the target personnel.

[0061] In this embodiment, the electronic device can acquire target answer data and target answer images corresponding to the target personnel. The target answer data includes at least the content of the target personnel's answers to knowledge competition questions (for example, the target answer data can be generated by the target personnel directly answering using a mobile phone, computer, or other terminal device, or it can be generated by scanning a paper test paper). The target answer image includes at least a facial image of the target personnel during the answering process (i.e., it must at least contain facial expression information; in addition, images of other body parts, such as the hand area, can also be acquired to collect hand movement information; different content output rates can also reflect different levels of mastery).

[0062] Step S120: Perform feature mining based on the target answer data and the target answer image respectively, and output the target answer data features and the target answer image features.

[0063] In this embodiment, after acquiring the target answer data and the target answer image, the electronic device can perform feature mining based on the target answer data and the target answer image respectively, outputting target answer data features and target answer image features. That is, key semantic information can be extracted from the target answer data and the target answer image respectively, ensuring the accuracy of subsequent processing. Furthermore, both the target answer data features and the target answer image features can be represented as vectors.

[0064] Step S130: Perform multi-focus correlation mining on the target answer data features and the target answer image features to output the target answer fusion features.

[0065] In this embodiment, after mining the target answer data features and the target answer image features, the electronic device can perform multi-focus association mining on the target answer data features and the target answer image features to output target answer fusion features. Here, multi-focus refers to multiple answer time points or multiple answer contents during the answering process. Thus, during the association mining process, the data dimension (reflecting the direct mastery of knowledge points) and the image dimension (reflecting the indirect mastery of knowledge points) can be mutually fused and reinforced. Furthermore, due to the existence of multiple focus points during the association mining process, high-precision fusion can be achieved, thereby further improving the semantic representation accuracy of the output target answer fusion features.

[0066] Step S140: Based on the target answer fusion features, analyze and output the mastery status of the target knowledge points.

[0067] In this embodiment, after outputting the target answer fusion features, the electronic device can analyze and output the mastery status of the target knowledge points based on the target answer fusion features. For example, steps S120-S140 above can be implemented through a neural network model, which may include a fully connected network (which may belong to the feature analysis subnetwork described later). The fully connected network performs fully connected processing on the target answer fusion features to obtain corresponding fully connected features. Through fully connected processing, the size of the mapped fully connected features can be 1*1, that is, it includes a parameter. This parameter can be mapped to a specified interval (such as 0-1, 1-10, or 1-100, etc.) through a linear mapping function (such as a linear activation function or identity mapping function), thereby obtaining the mastery status of the target knowledge points. The larger the value obtained by mapping, the higher the mastery level; the smaller the value obtained by mapping, the lower the mastery level.

[0068] Based on the above, by integrating the answer data with answer images, and extracting data such as the respondent's facial expressions from the answer images, it is possible to focus on the respondent's psychological state and attention distribution during the answering process. This information helps to more comprehensively assess their mastery of knowledge points. Therefore, it can improve the problem of relatively low reliability in the analysis of knowledge point mastery in existing technologies.

[0069] Firstly, regarding step S120, it should be noted that the specific methods for feature mining based on the target answer data and the target answer image are not limited and can be selected according to actual needs.

[0070] For example, in an alternative implementation, considering that the target answer data and the target answer image belong to different dimensions of information, namely text data and image data respectively, and the content they focus on is also different, in order to ensure the reliability of feature mining, the above step S120 can further include steps S121, S122 and S123, as follows.

[0071] Step S121: Load the target answer data and the target answer image into the target analysis network.

[0072] In this embodiment, the target answer data and the target answer image are loaded into a target analysis network for subsequent mining, analysis, and other processing. The target analysis network (a trained neural network model, the specific training process of which is described later) includes a feature mining sub-network (which may also include an association mining sub-network and a feature analysis sub-network; the association mining sub-network can be used to execute step S130, and the feature analysis sub-network can be used to execute step S140). The feature mining sub-network includes a first mining branch and a second mining branch with a network architecture different from the first mining branch.

[0073] Step S122: Through the first mining branch, perform first mining on the target answer data to obtain the target answer data features.

[0074] In this embodiment, after loading the target answer data, the first mining branch can be used to perform a first mining operation on the target answer data to obtain target answer data features. The first mining includes at least one of internal association mining and external association mining of the answer content; that is, only internal association mining, only external association mining, or both. In other embodiments, word embedding processing can be performed on the target answer data, and then the resulting word vectors can be added or concatenated to form target answer data features, thereby improving mining efficiency.

[0075] Step S123: Through the second mining branch, perform a second mining on the target answer image to obtain the target answer image features.

[0076] In this embodiment, after loading the target answer image, the second mining branch can be used to perform a second mining on the target answer image to obtain target answer image features. The second mining includes both internal and external correlation mining of the answer image; both are performed because the semantic information represented by the image is relatively rich and consists of indirect features. Therefore, sufficient mining is required to discard invalid features, capture more effective features, and improve the reliability of subsequent processing. In other embodiments, the target answer image can also be convolved to form target answer image features, thereby improving mining efficiency.

[0077] It is understood that the specific method of performing the first mining on the target answer data in step S122 above is not limited. For example, in an alternative implementation, in order to balance the efficiency and reliability of the overall processing, step S122 above may further include the following, based on association mining:

[0078] First, the target answer data can be loaded into the first mining branch, so that feature mining can be performed in the first mining branch;

[0079] Secondly, each response in the target answer data can be embedded to form a response content embedding feature corresponding to each response. The response content has a one-to-one correspondence with the questions in the knowledge competition, and the response content includes a reference answer. That is, a response includes the actual answer to a question (i.e., the content of the target person's response) and a reference answer. For example, in safe production, which of the following measures is the most effective in preventing fire accidents? A) When using open flames, strict control of the fire source is not required; B) Regularly inspect and maintain equipment to ensure there are no leaks; C) All chemical substances are randomly piled up, avoiding frequent handling; D) There is a lack of emergency response plans, and fire-fighting equipment is not required; Reference Answer: B) Regularly inspect and maintain equipment to ensure there are no leaks; For example, for "ensure there are no leaks", through word segmentation embedding, we can get: "ensure": [0.1, -0.2, 0.3, 0.4, -0.5, 0.6, -0.7, 0.8, -0.9, ..., 1.0]; "no": [0.2, 0.1, -0.3, 0.5, -0.2, 0.7, -0.1, 0.3, 0.4, ... ..., -0.6]; "Leakage": [-0.5, 0.3, 0.2, -0.4, 0.1, -0.3, 0.6, -0.7, 0.8, ..., 0.2]; "Phenomenon": [0.6, -0.1, 0.4, -0.5, 0.9, -0.2, 0.1, 0.7, -0.3, ..., -0.8]; Then, the word vectors can be added together to obtain the corresponding response content embedding features, or the word vectors can be concatenated to obtain the corresponding response content embedding features. Since concatenation results in a larger feature size, the concatenated features can also be further downsampled to obtain the response content embedding features;

[0080] Then, the embedded features of each of the responses can be clustered to form at least one central feature; for example, K-means clustering can be used to cluster the embedded features of each response.

[0081] Subsequently, for each of the aforementioned response content embedding features, based on the central feature corresponding to the response content embedding feature, association mining can be performed on the response content embedding feature to form the response content association feature corresponding to the response content embedding feature. The association mining can be implemented through the cross-attention mechanism, which will not be elaborated here.

[0082] Finally, each of the aforementioned response content association features can be concatenated to obtain the target response data features, such as (response content association feature 1, response content association feature 2, response content association feature 3, response content association feature 4, ..., response content association feature n).

[0083] It is understood that the specific method of performing the second mining on the target answer image in step S123 above is not limited. For example, in an alternative implementation, in order to balance the efficiency and reliability of the overall processing, step S123 above may further include the following, based on association mining:

[0084] First, the target answer image can be loaded into the second mining branch, so that further mining can be carried out in the second mining branch;

[0085] Secondly, each local answer image in the target answer image can be convolved to form local answer image features corresponding to each local answer image. The local answer images have a one-to-one correspondence with the questions in the knowledge competition. For example, the collected image set can be sampled, such as extracting a frame image for the answering process of each question to characterize the facial expressions and semantics of the target person in the process of answering the question.

[0086] Then, internal association mining can be performed on each of the local answer image features to form image internal association features corresponding to each local answer image feature. In the process of internal association mining, after performing semantic feature mining at different depths on the local answer image features, association mining is performed on the semantic features at different depths. For example, for a local answer image feature, mean pooling and maximum value size can be performed respectively to obtain a first pooling feature and a second pooling feature. Then, cross-attention processing can be performed on the second pooling feature based on the first pooling feature to obtain the corresponding image internal association features. Alternatively, cross-attention processing can be performed on the first pooling feature based on the second pooling feature to obtain the corresponding image internal association features.

[0087] Then, external association mining can be performed on each of the image internal association features to form the image external association features corresponding to each image internal association feature. In the process of external association mining, after clustering each of the image internal association features, association mining is performed on each of the image internal association features based on the corresponding central features. The specific processing procedure can be referred to the explanation of step S122 above, and will not be repeated here.

[0088] Finally, each of the aforementioned external image association features can be concatenated to obtain the target answer image features, such as (external image association feature 1, external image association feature 2, external image association feature 3, external image association feature 4, ..., external image association feature n).

[0089] Secondly, regarding step S130, it should be noted that the specific method for multi-focus association mining of the target answer data features and the target answer image features is not limited and can be selected according to actual needs.

[0090] For example, in an alternative implementation, the target answer data features and the target answer image features can be concatenated. Then, the concatenated features can be processed by convolution, pooling, and fully connected layers to obtain the target answer fusion features. Specifically, to achieve multi-focus association mining, the corresponding answer content association features and image external association features (such as those with the same corresponding question) can be processed by convolution, pooling, and fully connected layers to obtain corresponding local features. These local features can then be concatenated to form the target answer fusion features.

[0091] For example, in another alternative implementation, in order to further improve the accuracy of association mining while realizing multi-focus association mining, the above step S130 may further include the following steps S131 and S132, the specific contents of each step are as follows.

[0092] Step S131: Perform sliding window segmentation on the target answer data features and the target answer image features respectively to form multiple sliding window answer data features and multiple sliding window answer image features.

[0093] In this embodiment, the target answer data features and the target answer image features can be segmented using sliding windows to form multiple sliding window answer data features and multiple sliding window answer image features (one-to-one correspondence, and all with the same size). The target answer data features and the target answer image features have the same size, and the sliding window segmentation methods corresponding to the two types of segmentation are identical. For example, the target answer data features and the target answer image features can be loaded into an association mining sub-network, and then the sliding window size and stride (which can be formed during training) carried by the association mining sub-network can be used for sliding window segmentation processing.

[0094] Step S132: During the process of sequentially fusing the multiple sliding window answer data features, multiple sliding window answer image features are sequentially fused to output the target answer fusion feature.

[0095] In this embodiment of the application, after forming multiple sliding window answer data features and multiple sliding window answer image features, multiple sliding window answer image features can be fused sequentially during the process of sequentially fusing the multiple sliding window answer data features, and the target answer fusion feature is output. That is to say, not only is it necessary to perform association mining on the multiple sliding window answer data features, but also to fuse the multiple sliding window answer image features during the association mining process. In this way, fine-grained association mining can be achieved, thereby improving the accuracy of association mining.

[0096] It is understood that in step S132 above, the specific method of sequentially fusing multiple sliding window answer image features during the sequential fusion of the multiple sliding window answer data features is not limited. For example, in an alternative implementation, in order to achieve full fusion of the multiple sliding window answer data features and the multiple sliding window answer image features, multiple stages of fusion can be performed to ensure the gradient and accuracy of semantic fusion. Based on this, step S132 above can further include steps S132a and S132b, the specific contents of which are as follows.

[0097] Step S132a: In the first fusion stage, based on the features of the first sliding window answer data, the features of the second sliding window answer data are correlated and mined to form the answer data fusion features of the first fusion stage; based on the features of the first sliding window answer image, the features of the second sliding window answer image are correlated and mined to form the answer image fusion features of the first fusion stage; and based on the answer image fusion features, the answer data fusion features are correlated and mined to form the answer fusion features of the first fusion stage.

[0098] In this embodiment, in the first fusion stage, association mining (i.e., cross-attention processing) is performed on the second sliding window answer data features based on the first sliding window answer data features to form the answer data fusion features of the first fusion stage. Similarly, association mining (i.e., cross-attention processing) is performed on the second sliding window answer image features based on the first sliding window answer image features to form the answer image fusion features of the first fusion stage. Furthermore, based on the answer... The image fusion features of the question data are used to perform association mining on the fusion features of the answer data (i.e., cross-attention processing is applied to the fusion features of the answer data based on the image fusion features of the question data), forming the answer fusion features of the first fusion stage. It should be noted that, in order to avoid semantic distortion due to the increase in network depth after multiple cross-attention processing, after performing association mining on the fusion features of the answer data based on the image fusion features of the question data, the results of the association mining, the first sliding window answer data features, and the second sliding window answer data features are averaged to obtain the answer fusion features of the first fusion stage.

[0099] In step S132b, in the second and subsequent fusion stages, the target answer fusion features are determined based on the sliding window answer data features and sliding window answer image features of the second and subsequent stages, combined with the answer image fusion features and answer fusion features of the first fusion stage.

[0100] In this embodiment of the application, after the answer image fusion features and answer fusion features of the first fusion stage are formed, the target answer fusion features can be determined in the second and subsequent fusion stages based on the sliding window answer data features and sliding window answer image features of the second and subsequent fusion stages, combined with the answer image fusion features and answer fusion features of the first fusion stage.

[0101] It is understood that the specific method for determining the target answer fusion features in step S132b above is not limited and can be selected according to actual needs. For example, in an alternative implementation, in order to achieve the gradual fusion of semantic information and ensure the accuracy of fusion, step S132b above can further include the following (in conjunction with...). Figure 3 ):

[0102] First, for each subsequent fusion stage (such as the second fusion stage, the third fusion stage), based on the answer fusion features of the previous fusion stage, association mining is performed on the sliding window answer data features of the current fusion stage to form the answer data fusion features of the current fusion stage. Then, based on the answer image fusion features of the previous fusion stage, association mining is performed on the sliding window answer image features of the current fusion stage to form the answer image fusion features of the current fusion stage. Finally, based on the answer image fusion features of the current fusion stage, association mining is performed on the answer data fusion features of the current fusion stage to form the answer fusion features of the current fusion stage. It should be noted that, to avoid semantic distortion due to increased network depth after multiple association mining operations, after performing association mining on the answer data fusion features of the current fusion stage based on the answer image fusion features, the results of the association mining and the sliding window answer data features of the current fusion stage can be averaged to obtain the answer fusion features of the current fusion stage.

[0103] Secondly, the target answer fusion feature can be determined based on the answer fusion features of the last fusion stage. For example, the answer fusion features of the last fusion stage can be directly used as the target answer fusion feature, or, based on other needs, the answer fusion features of the last fusion stage can be further processed to obtain the target answer fusion feature.

[0104] Thirdly, regarding steps S120, S130, and S140, it should be noted that, to ensure the effective execution of each step, a trained target analysis network can be used. Therefore, the knowledge point mastery analysis method based on knowledge competitions can also include the step of training the target analysis network. The specific content of this step can be as follows:

[0105] First, training answer data and training answer images can be obtained. The training answer data includes at least the content of the answers based on the knowledge competition questions, and the training answer images include at least the facial images during the answering process. The specific content is as described in the relevant explanation of step S110 above.

[0106] Secondly, using a candidate analysis network, feature mining is performed on the training answer data and the training answer image respectively (e.g., through the aforementioned feature mining sub-network; the specific processing procedure can be referred to the explanation of step S120 above), outputting training answer data features and training answer image features. Furthermore, the training answer data features and training answer image features are subjected to multi-focus association mining (e.g., through the aforementioned association mining sub-network; the specific processing procedure can be referred to the explanation of step S130 above), outputting training answer fusion features. Based on the training answer fusion features, the mastery of training knowledge points is analyzed and output (e.g., through the aforementioned feature analysis sub-network; the specific processing procedure can be referred to the explanation of step S140 above). The candidate analysis network is a neural network model to be trained.

[0107] Then, based on the error (such as mean square error) between the knowledge point mastery status and the knowledge point mastery status label, the network parameters of the candidate analysis network can be updated to form the target analysis network. For example, the network parameters of the candidate analysis network can be updated along the direction of reducing the error (the specific update process can refer to the relevant existing technology, which will not be described in detail here) until the error converges, thereby obtaining the target analysis network (i.e. the candidate analysis network after training).

[0108] Combination Figure 4 This application also provides a knowledge point mastery analysis device based on a knowledge competition, applicable to the aforementioned electronic devices. The knowledge point mastery analysis device based on a knowledge competition may include a data acquisition module, a feature mining module, an association mining module, and a feature analysis module.

[0109] Specifically, the data acquisition module can be used to acquire target answer data and target answer images corresponding to the target personnel. The target answer data at least includes the content of the target personnel's answers to the knowledge competition questions, and the target answer images at least include facial images of the target personnel during the answering process. In this embodiment, the data acquisition module can be used to perform... Figure 2 The relevant content regarding the data acquisition module in step S110 shown can be found in the preceding description of step S110.

[0110] Specifically, the feature mining module can be used to perform feature mining based on the target answer data and the target answer image respectively, and output the target answer data features and the target answer image features. In this embodiment, the feature mining module can be used to perform... Figure 2 The relevant content regarding the feature mining module in step S120 shown can be found in the previous description of step S120.

[0111] Specifically, the association mining module can be used to perform multi-focus association mining on the target answer data features and the target answer image features, outputting target answer fusion features. Here, multi-focus refers to multiple answer time points or multiple answer contents during the answering process. In this embodiment, the association mining module can be used to execute... Figure 2 The relevant content regarding the association mining module in step S130 shown can be found in the previous description of step S130.

[0112] Specifically, the feature analysis module can be used to analyze and output the mastery of target knowledge points based on the target answer fusion features. In this embodiment, the feature analysis module can be used to perform... Figure 2 The relevant content regarding the feature analysis module in step S140 shown can be found in the previous description of step S140.

[0113] In this embodiment of the application, corresponding to the above-described method for analyzing knowledge point mastery based on a knowledge competition applied to the electronic device, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program, which executes the various steps of the method for analyzing knowledge point mastery based on a knowledge competition when it runs.

[0114] The steps executed by the aforementioned computer program during runtime will not be described in detail here; please refer to the explanation of the knowledge point mastery analysis method based on knowledge competition described above.

[0115] In summary, the method, apparatus, and equipment for analyzing knowledge point mastery based on knowledge competitions provided in this application first acquire target answer data and target answer images corresponding to the target personnel; second, feature mining is performed on the target answer data and target answer images respectively, outputting target answer data features and target answer image features; then, multi-focus correlation mining is performed on the target answer data features and target answer image features to output target answer fusion features; finally, the target knowledge point mastery is analyzed and output based on the target answer fusion features. Based on the above, by fusing answer images on the basis of answer data, and extracting data such as the respondent's facial expressions from the answer images, it is possible to focus on the respondent's psychological state and attention distribution during the answering process. This information helps to more comprehensively assess their mastery of knowledge points, thus improving the problem of relatively low reliability in the analysis of knowledge point mastery in existing technologies.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0117] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0118] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0119] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for analyzing knowledge point mastery based on knowledge competitions, characterized in that, include: Obtain target answer data and target answer image corresponding to the target person, wherein the target answer data includes at least the content of the target person's answer based on the knowledge competition questions, and the target answer image includes at least the facial image of the target person during the answering process; The target answer data and the target answer image are loaded into a target analysis network, wherein the target analysis network includes a feature mining sub-network, the feature mining sub-network includes a first mining branch and a second mining branch with a network architecture different from the first mining branch; through the first mining branch, the target answer data is subjected to first mining to obtain target answer data features, wherein the first mining includes at least one of internal association mining of the answer content and external association mining of the answer content; through the second mining branch, the target answer image is subjected to second mining to obtain target answer image features, wherein the second mining includes internal association mining of the answer image and external association mining of the answer image; The target answer data features and the target answer image features are segmented using sliding windows to form multiple sliding window answer data features and multiple sliding window answer image features. The target answer data features and the target answer image features have the same size, and the sliding window segmentation methods corresponding to the two types of segmentation are the same. In the first fusion stage, based on the first sliding window answer data features, association mining is performed on the second sliding window answer data features to form the answer data fusion features for the first fusion stage. Similarly, based on the first sliding window answer image features, association mining is performed on the second sliding window answer image features to form the answer image fusion features for the first fusion stage. Furthermore, based on these answer image fusion features, association mining is performed on the answer data fusion features to form the answer fusion features for the first fusion stage. In the second and subsequent fusion stages, the target answer fusion features are determined based on the second and subsequent sliding window answer data features and sliding window answer image features, combined with the answer image fusion features and answer fusion features from the first fusion stage. Based on the aforementioned target answer fusion features, the mastery of the target knowledge points is analyzed and output.

2. The method for analyzing knowledge point mastery based on knowledge competitions according to claim 1, characterized in that, The step of performing a first mining operation on the target answer data through the first mining branch to obtain the target answer data features includes: The target answer data is loaded into the first mining branch; Each response in the target answer data is embedded to form a response content embedding feature corresponding to each response. The response content has a one-to-one correspondence with the questions in the knowledge competition, and the response content includes the reference answer. Cluster the embedded features of each of the responses to form at least one central feature; For each of the aforementioned response content embedding features, based on the central feature corresponding to the response content embedding feature, association mining is performed on the response content embedding feature to form the response content association feature corresponding to the response content embedding feature. By concatenating the associated features of each of the aforementioned responses, the target response data features are obtained.

3. The method for analyzing knowledge point mastery based on knowledge competitions according to claim 1, characterized in that, The step of performing a second mining operation on the target answer image through the second mining branch to obtain the target answer image features includes: The target answer image is loaded into the second mining branch; Each local answer image in the target answer image is convolved to form a local answer image feature corresponding to each local answer image, wherein there is a one-to-one correspondence between the local answer image and the question in the knowledge competition; Internal association mining is performed on each of the local answer image features to form image internal association features corresponding to each local answer image feature. In the process of internal association mining, after mining the local answer image features at different depths of semantic features, association mining is performed on the mined semantic features at different depths of semantic features. External association mining is performed on each of the image internal association features to form an image external association feature corresponding to each image internal association feature. In the process of external association mining, after clustering each of the image internal association features, association mining is performed on each of the image internal association features based on the corresponding central features. By concatenating the external features of each of the images, the target answer image features are obtained.

4. The method for analyzing knowledge point mastery based on knowledge competitions according to claim 1, characterized in that, The step of determining the target answer fusion features in the second and subsequent fusion stages, based on the sliding window answer data features and sliding window answer image features, and combined with the answer image fusion features and answer fusion features from the first fusion stage, includes: For each subsequent fusion stage, based on the answer fusion features of the previous fusion stage, the sliding window answer data features of the current fusion stage are correlated and mined to form the answer data fusion features of the current fusion stage. Based on the answer image fusion features of the previous fusion stage, the sliding window answer image features of the current fusion stage are correlated and mined to form the answer image fusion features of the current fusion stage. Based on the answer image fusion features of the current fusion stage, the answer data fusion features of the current fusion stage are correlated and mined to form the answer fusion features of the current fusion stage. Based on the answer fusion characteristics of the last fusion stage, the target answer fusion characteristics are determined.

5. The method for analyzing knowledge point mastery based on knowledge competitions according to any one of claims 1-4, characterized in that, Also includes: Acquire training answer data and training answer images, wherein the training answer data includes at least the content of the answers to knowledge competition questions, and the training answer images include at least the facial images during the answering process; Using a candidate analysis network, feature mining is performed on the training answer data and the training answer image respectively, outputting training answer data features and training answer image features. In addition, the training answer data features and the training answer image features are subjected to multi-focus association mining, outputting training answer fusion features. Based on the training answer fusion features, the mastery of training knowledge points is analyzed and output. The candidate analysis network is a neural network model to be trained. Based on the error between the mastery status of the training knowledge points and the knowledge point mastery status labels, the network parameters of the candidate analysis network are updated to form the target analysis network.

6. A device for analyzing knowledge point mastery based on knowledge competitions, characterized in that, include: The data acquisition module is used to acquire target answer data and target answer images corresponding to the target personnel. The target answer data includes at least the content of the target personnel's answers based on the knowledge competition questions, and the target answer images include at least the facial images of the target personnel during the answering process. A feature mining module is used to load the target answer data and the target answer image into a target analysis network. The target analysis network includes a feature mining sub-network, which includes a first mining branch and a second mining branch with a network architecture different from the first mining branch. The first mining branch performs a first mining operation on the target answer data to obtain target answer data features. The first mining operation includes at least one of internal association mining and external association mining of the answer content. The second mining branch performs a second mining operation on the target answer image to obtain target answer image features. The second mining operation includes both internal association mining and external association mining of the answer image. The association mining module is used to perform sliding window segmentation on the target answer data features and the target answer image features respectively, forming multiple sliding window answer data features and multiple sliding window answer image features. The target answer data features and the target answer image features have the same size, and the sliding window segmentation methods corresponding to the two types of segmentation are the same. In the first fusion stage, based on the first sliding window answer data features, association mining is performed on the second sliding window answer data features to form the answer data fusion features of the first fusion stage. Similarly, based on the first sliding window answer image features, association mining is performed on the second sliding window answer image features to form the answer image fusion features of the first fusion stage. Furthermore, based on these answer image fusion features, association mining is performed on the answer data fusion features to form the answer fusion features of the first fusion stage. In the second and subsequent fusion stages, the target answer fusion features are determined based on the second and subsequent sliding window answer data features and sliding window answer image features, combined with the answer image fusion features and answer fusion features of the first fusion stage. The feature analysis module is used to analyze and output the mastery of target knowledge points based on the target answer fusion features.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the knowledge point mastery analysis method based on knowledge competition as described in any one of claims 1-5.

Images