Circuit exercise classification method and device based on visual language model and electronic equipment

Abstract

The invention provides a circuit exercise classification method and device based on a visual language model and electronic equipment. The method comprises the steps that circuit exercises to be classified are acquired; based on a pre-trained visual language model, predicting to obtain knowledge point categories corresponding to the circuit exercises; wherein the visual language model is obtained by performing fine adjustment on a training sample data set formed by the subject map and the corresponding knowledge point label based on the pre-training model. According to the method, various elements and topological structures in a circuit diagram are identified in a visual input mode and a character input mode, key information in question stems of circuit exercises is extracted at the same time, and then rapid and accurate classification of the circuit exercises is achieved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a circuit exercise classification method, device and electronic equipment based on a visual language model. Background Art

[0002] In the field of education, especially in the process of teachers teaching and students learning, the classification of knowledge points in circuit exercises is very important. However, due to the high degree of specialization of circuit knowledge and the complexity of circuit questions, identifying and classifying the knowledge points of circuit exercises has become a tedious and time-consuming task.

[0003] Due to limitations in training data, the world's most powerful general multimodal models GPT4V and Claude3 are unable to classify knowledge points in images of circuit subjects.

[0004] Therefore, how to solve the problem that existing technologies are unable to quickly and accurately classify circuit exercises is an important issue that needs to be urgently addressed in the field of deep learning. Summary of the invention

[0005] The present invention provides a circuit exercise classification method, device and electronic equipment based on a visual language model, which are used to overcome the defect that the prior art cannot classify circuit exercises quickly and accurately, and realize the fast and accurate classification of circuit exercises.

[0006] On the one hand, the present invention provides a circuit exercise classification method based on a visual language model, comprising: obtaining circuit exercises to be classified; predicting the knowledge point category corresponding to the circuit exercises based on a pre-trained visual language model; wherein the visual language model is obtained by fine-tuning a training sample data set composed of a question image and its corresponding knowledge point labels based on the pre-trained model.

[0007] Furthermore, fine-tuning the visual language model specifically includes: collecting original exercise sets from different distribution domains; preprocessing the original exercise sets to obtain a training sample data set, wherein each training sample in the training sample data set includes a question image and its corresponding knowledge point label, and the question image includes a question stem and an accompanying picture; dividing the training sample data set into a training set, a validation set, and a test set; and fine-tuning and testing the pre-trained model according to the training set, the validation set, and the test set, respectively, to obtain a pre-trained visual language model.

[0008] Furthermore, the preprocessing of the original exercise set to obtain a training sample data set includes: converting the original exercise set into editable text and extracting circuit principle exercises from the editable text; and matching corresponding knowledge point tags from a pre-stored knowledge point library based on the circuit principle exercises to obtain the training sample data set.

[0009] Furthermore, the pre-stored knowledge point library includes at least secondary knowledge points and tertiary knowledge points; wherein the secondary knowledge points contain multiple tertiary knowledge points.

[0010] Furthermore, the step of fine-tuning the pre-trained model according to the validation set specifically includes: monitoring the performance index of the pre-trained model on the validation set; and terminating the training in advance when the performance index no longer improves or begins to decrease.

[0011] Furthermore, fine-tuning the visual language model specifically includes: collecting original exercise sets from different distribution domains; preprocessing the original exercise sets to obtain a training sample data set, wherein each training sample in the training sample data set includes a question image and its corresponding knowledge point label, and the question image includes a question stem, an accompanying image, and an accompanying image text description; dividing the training sample data set into a training set, a validation set, and a test set; and fine-tuning and testing the pre-trained model according to the training set, the validation set, and the test set, respectively, to obtain a pre-trained visual language model.

[0012] In a second aspect, the present invention also provides a circuit exercise classification device based on a visual language model, comprising: a circuit exercise acquisition module, used to acquire circuit exercises to be classified; a circuit exercise classification module, used to predict the knowledge point category corresponding to the circuit exercise based on a pre-trained visual language model; wherein the visual language model is obtained by fine-tuning a training sample data set composed of a question image and its corresponding knowledge point labels based on the pre-trained model.

[0013] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the circuit exercise classification method based on the visual language model as described above is implemented.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described circuit exercise classification methods based on a visual language model.

[0015] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the circuit exercise classification method based on the visual language model as described in any one of the above is implemented.

[0016] The circuit exercise classification method based on the visual language model provided by the present invention obtains the circuit exercises to be classified, and predicts the knowledge point categories corresponding to the circuit exercises based on the pre-trained visual language model; wherein the visual language model is fine-tuned based on the pre-trained model according to the training sample data set composed of the question diagram and its corresponding knowledge point labels. The method uses two types of input, visual and text, to identify various components and topological structures in the circuit diagram, and extract key information from the stem of the circuit exercises, thereby realizing fast and accurate classification of circuit exercises. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of a circuit exercise classification method based on a visual language model provided by an embodiment of the present invention.

[0019] Figure 2 It is a schematic diagram of a circuit exercise in the circuit exercise classification method based on a visual language model provided in an embodiment of the present invention.

[0020] Figure 3 It is a structural schematic diagram of a circuit exercise classification device based on a visual language model provided by an embodiment of the present invention.

[0021] Figure 4 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be noted that the existing technology is limited by training data, and the strongest general multimodal models GPT4V and Claude3 are unable to classify knowledge points for images of circuit subjects.

[0024] In view of this, the present invention proposes a circuit exercise classification method based on a visual language model. Specifically, Figure 1A flow chart of a circuit exercise classification method based on a visual language model provided in an embodiment of the present invention is shown.

[0025] like Figure 1 As shown, the method includes steps S110-S120, and steps S110-S120 and related steps will be described in detail below.

[0026] S110, obtaining circuit exercises to be classified.

[0027] It is understandable that in order to classify circuit exercises, it is first necessary to obtain circuit exercises to be classified. In this embodiment, circuit exercises come from many sources, including but not limited to textbooks, academic websites and forums, exercise software, university course webpages, professional books and manuals, library resources, and social media and educational communities.

[0028] That is to say, circuit exercises can be obtained from corresponding sources according to actual needs, and no specific limitation is made here.

[0029] S120, predicting the knowledge point category corresponding to the circuit exercise based on a pre-trained visual language model; wherein the visual language model is obtained by fine-tuning a training sample data set consisting of a question image and its corresponding knowledge point labels based on a pre-trained model.

[0030] It can be understood that, based on the circuit exercises to be classified obtained in step S110, further, in step S120, the circuit exercises to be classified are input into a pre-trained visual language model to obtain the classification results corresponding to the output circuit exercises, that is, the knowledge point categories corresponding to the circuit exercises.

[0031] Specifically, the visual language model used in this embodiment is obtained by fine-tuning a pre-trained model through a constructed training sample data set, wherein each training sample in the training sample data set includes at least a title image and its corresponding knowledge point label, and the title image includes at least a title and a picture.

[0032] Regarding the title and the accompanying figure, the title is like "Consider a system consisting of two resistors ( , ) and a current source, find the node voltage. ".

[0033] Regarding knowledge point labels, knowledge point labels refer to the knowledge point category labels of the corresponding questions. Knowledge point category labels include but are not limited to the application of Ohm's law, node voltage method, branch current method, and transient response analysis.

[0034] Similarly, the categories of knowledge points finally output by the visual language model include but are not limited to the application of Ohm's law, node voltage method, branch current method, transient response analysis, logic gate operation, sequential logic circuits, etc.

[0035] It should be noted that the pre-training model in this embodiment can be set according to actual needs and is not specifically limited here.

[0036] For example, in a specific embodiment, the pre-trained model used is the open source Tongyi Qianwen-VL model.

[0037] In another specific embodiment, the pre-trained model used is any one of VisualBERT, ViLBERT, VL-BERT, CLIP (Contrastive Language-Image Pre-training) and UNITER (Unified Multi-modal Transformer).

[0038] In this embodiment, by obtaining circuit exercises to be classified, and based on a pre-trained visual language model, the knowledge point category corresponding to the circuit exercises is predicted; wherein the visual language model is fine-tuned based on the pre-trained model according to the training sample data set consisting of the question image and its corresponding knowledge point labels. This method uses two types of input, visual and text, to identify various components and topological structures in the circuit diagram, and extract key information from the stem of the circuit exercises, thereby achieving rapid and accurate classification of circuit exercises.

[0039] On the basis of the above embodiments, the fine-tuning process of the visual language model will be described in detail below.

[0040] Fine-tuning the visual language model specifically includes: collecting original exercise sets from different distribution domains; preprocessing the original exercise sets to obtain a training sample data set, each training sample in the training sample data set includes a question image and its corresponding knowledge point label, and the question image includes a question stem and an accompanying picture; dividing the training sample data set into a training set, a validation set, and a test set; fine-tuning and testing the pre-trained model according to the training set, the validation set, and the test set, respectively, to obtain a pre-trained visual language model.

[0041] First, in order to ensure that the visual language model can generalize to various types of circuit exercises, it is necessary to collect original exercise sets from different distribution domains. Specifically, electronic versions of various circuit principle exercise sets can be obtained from book websites.

[0042] For example, in a specific embodiment, the circuit exercises in the textbooks "Circuit Principles Study Guide and Exercise Solutions", "Circuit" and "Circuit Principles Study Guide and Exercise Collection" are organized into an original exercise collection.

[0043] Then, since the electronic version of the original exercise book cannot be edited, the collected original exercise book needs to be preprocessed to obtain a training sample data set, specifically including: converting the original exercise book into editable text and extracting circuit principle exercises from the editable text; according to the circuit principle exercises, matching the corresponding knowledge point labels from the pre-stored knowledge point library to obtain a training sample data set.

[0044] Specifically, the original exercise book in PDF version can be converted into editable text through optical character recognition (OCR) and manual annotation, and on this basis, all circuit principle exercises in the editable text are extracted. The circuit principle exercises here include question stems and illustrations.

[0045] Then, the extracted circuit principle exercises are organized into the form of "question image-knowledge point label" pairs through manual labeling and entity matching, thereby constructing a training sample data set.

[0046] Among them, the knowledge point labels are matched from the pre-stored knowledge point library. In order to enable the visual language model to adapt to various scenarios, the pre-stored knowledge point library includes at least secondary knowledge points and tertiary knowledge points; among them, the secondary knowledge points contain multiple tertiary knowledge points.

[0047] The first-level knowledge points are the broadest concepts, such as "circuit theory", the second-level knowledge points are more specific concepts, such as "DC circuit", "AC circuit", "analog circuit", "digital circuit", and the third-level knowledge points are more specific details, such as "Kirchhoff's law", "Ohm's law", "series circuit", "parallel circuit", and "mixed circuit".

[0048] Each training sample in the training sample data set corresponds to one or more representative knowledge point labels, and the one or more knowledge point labels include both secondary knowledge points and tertiary knowledge points.

[0049] Next, the constructed training sample data set is divided into training set, validation set and test set.

[0050] Specifically, 80% of the training sample data set can be randomly selected as the training set, and the remaining 20% ​​can be used as the test set (10%) and the validation set (10%). Of course, 70% of the training sample data set can also be randomly selected as the training set, and the remaining 30% can be equally divided, with 15% as the validation set and 15% as the test set, which is not specifically limited here.

[0051] Finally, according to the divided training set, validation set and test set, the pre-trained model is fine-tuned and tested respectively to obtain the pre-trained visual language model.

[0052] Among them, the training set is used to train the pre-trained model, allowing the pre-trained model to learn the mapping relationship from input to output, and update the model weights and biases through continuous iteration to minimize the loss function on the training set.

[0053] The validation set is used to adjust the hyperparameters of the pre-trained model, such as learning rate, network architecture, etc. During the training process, overfitting is prevented by monitoring the model performance on the validation set.

[0054] For example, in a specific embodiment, the step of fine-tuning the pre-trained model according to the validation set specifically includes: monitoring the performance indicators of the pre-trained model on the validation set; and terminating the training in advance when the performance indicators no longer improve or begin to decrease.

[0055] The test set is used to evaluate the performance of the trained visual language model on unknown data, so as to measure the final performance of the visual language model.

[0056] During the fine-tuning and testing process, the title image is used as the model input, the predicted knowledge point category is used as the model output, and the difference between the predicted knowledge point category and the knowledge point label is used as the training loss. The pre-trained model is continuously adjusted and optimized to obtain a pre-trained visual perception model.

[0057] In a specific embodiment, since the training sample data sets come from different books, the distribution domains of the circuit exercise samples will also be different. In order to better reflect the generalization ability of the model, the circuit exercises in the textbooks "Circuit Principle Learning Guide and Exercises" and "Circuits" are organized as data set a, and the circuit exercises in the textbook "Circuit Principle Learning Guide and Exercises" are organized as data set b. 80% of data set a is randomly selected as the training set, and the remaining 20% ​​is used as the test set a and the verification set, and data set b is used as the test set b.

[0058] On the test set a, which has the same distribution as the training set, the visual language model achieved 76.1% accuracy, 74.4% recall, and an F1 score of 0.740.

[0059] On the test set b outside the training set distribution domain, the visual language model also obtained an accuracy of 58.3%, a recall of 53.8% and an F1 score of 0.532. This shows that the visual language model provided in this embodiment can judge the category of knowledge points according to the characteristics of circuit exercises and has a certain generalization ability.

[0060] In this embodiment, by collecting original exercise sets from different distribution domains and preprocessing the original exercise sets, a training sample data set is obtained. Each training sample in the training sample data set includes a question image and its corresponding knowledge point label. The question image includes a question stem and a matching picture. Then the training sample data set is divided into a training set, a validation set, and a test set. According to the training set, the validation set, and the test set, the pre-trained model is fine-tuned and tested respectively to obtain a pre-trained visual language model. Thus, based on the pre-trained visual language model, the knowledge point category corresponding to the circuit exercise is predicted. This method uses two types of input, visual and text, to identify various components and topological structures in the circuit diagram, and extract key information in the question stem of the circuit exercise, thereby realizing fast and accurate classification of the circuit exercise.

[0061] On the basis of the above embodiments, further, fine-tuning the visual language model specifically includes: collecting original exercise sets from different distribution domains; preprocessing the original exercise sets to obtain a training sample data set, each training sample in the training sample data set includes a question image and its corresponding knowledge point label, and the question image includes a question stem, an accompanying image, and an accompanying image text description; dividing the training sample data set into a training set, a validation set, and a test set; according to the training set, the validation set, and the test set, respectively fine-tuning and testing the pre-trained model to obtain a pre-trained visual language model.

[0062] It is understandable that compared with the previous embodiment of fine-tuning the visual language model, in addition to the question stem, illustrations, and knowledge point labels, this embodiment adds more information about circuit exercises (illustration text descriptions) when constructing the training sample data set. The illustration text descriptions include but are not limited to the categories and functions of each circuit element in the illustration.

[0063] When fine-tuning the visual language model, the stem, illustrations, and text descriptions of circuit exercises are used as model input, the predicted knowledge point categories are used as model output, and the difference between the predicted knowledge point categories and the knowledge point labels is used as the training loss. The pre-trained model is continuously adjusted and optimized to obtain a pre-trained visual perception model.

[0064] It should be noted that, in this embodiment, except for fine-tuning the input of the visual language model to add a text description of the picture, the rest is the same as the above embodiment and will not be described in detail here.

[0065] In this embodiment, a visual language model is obtained by fine-tuning the pre-trained model using a training sample data set consisting of question stems, pictures, text descriptions of pictures, and knowledge point labels, which effectively improves the performance of the visual language model and enables the visual language model to quickly and accurately classify knowledge points in circuit exercises.

[0066] In addition, in some embodiments, a specific example of the circuit exercise classification method based on the visual language model provided in this embodiment is given.

[0067] First, obtain the circuit exercises to be classified.

[0068] The circuit exercise is "As shown in Figure 7-2-35, the DC voltage source in the circuit , the circuit has reached a stable state before the switch is activated. When switch S is closed, calculate the current after switching. ", which is equivalent to the question stem. Figure 7-2-35 in the example can refer to the example provided in this embodiment Figure 2 , Figure 2 A schematic diagram of a circuit exercise diagram of a circuit exercise classification method based on a visual language model provided by an embodiment of the present invention is shown.

[0069] Then, the circuit exercises are input into the pre-trained visual language model to obtain the knowledge point categories corresponding to the circuit exercises.

[0070] Specifically, the knowledge point categories corresponding to the circuit exercises generated by the visual language model are "'Three-element method of first-order circuit', 'power', 'resistance', 'initial value', 'equivalent replacement of power supply', 'Kirchhoff's law', 'zero input response and zero state response'".

[0071] The correct answers to this circuit exercise are "'Three elements method of first-order circuit', 'power', 'dynamic circuit', 'resistance', 'initial value', 'Kirchhoff's law', 'zero input response and zero state response'".

[0072] It can be seen that the circuit exercises are classified using the circuit exercise classification method based on the visual language model provided in this embodiment with a high accuracy.

[0073] Corresponding to the circuit exercise classification method based on the visual language model described in the above embodiments, the present invention also provides a circuit exercise classification device based on the visual language model.

[0074] Specifically, Figure 3 A schematic diagram of the structure of a circuit exercise classification device based on a visual language model provided by an embodiment of the present invention is shown.

[0075] like Figure 3As shown, the device includes: a circuit exercise acquisition module 310, used to acquire circuit exercises to be classified; a circuit exercise classification module 320, used to predict the knowledge point category corresponding to the circuit exercise based on a pre-trained visual language model; wherein the visual language model is obtained by fine-tuning a training sample data set composed of a question image and its corresponding knowledge point labels based on the pre-trained model.

[0076] In this embodiment, the circuit exercises to be classified are obtained through the circuit exercise acquisition module 310; the knowledge point category corresponding to the circuit exercises is predicted by the circuit exercise classification module 320 based on the pre-trained visual language model; wherein the visual language model is obtained by fine-tuning the training sample data set composed of the question diagram and its corresponding knowledge point labels based on the pre-trained model. The device uses two types of input, visual and text, to identify various components and topological structures in the circuit diagram, and extract key information from the stem of the circuit exercises, thereby realizing fast and accurate classification of circuit exercises.

[0077] It should be noted that the circuit exercise classification device based on the visual language model provided in this embodiment and the circuit exercise classification method based on the visual language model described in the above embodiments can be referred to each other, and will not be described in detail here.

[0078] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 810 may call the logic instructions in the memory 830 to execute a circuit exercise classification method based on a visual language model, the method comprising: obtaining circuit exercises to be classified; predicting the knowledge point category corresponding to the circuit exercises based on a pre-trained visual language model; wherein the visual language model is fine-tuned based on a training sample data set composed of a question image and its corresponding knowledge point labels according to a pre-trained model.

[0079] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0080] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the circuit exercise classification method based on the visual language model provided by the above-mentioned methods, and the method includes: obtaining circuit exercises to be classified; based on a pre-trained visual language model, predicting the knowledge point category corresponding to the circuit exercise; wherein the visual language model is obtained by fine-tuning a training sample data set composed of a question image and its corresponding knowledge point labels based on the pre-trained model.

[0081] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the circuit exercise classification method based on a visual language model provided by the above-mentioned methods, the method comprising: obtaining circuit exercises to be classified; based on a pre-trained visual language model, predicting the knowledge point category corresponding to the circuit exercises; wherein the visual language model is obtained by fine-tuning a training sample data set composed of a question image and its corresponding knowledge point labels based on the pre-trained model.

[0082] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0083] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A circuit exercise classification method based on visual language model, It is characterized in that include: Obtain circuit exercises to be classified; Based on the pre-trained visual language model, predict the knowledge point category corresponding to the circuit exercise; The visual language model is obtained by fine-tuning a training sample data set consisting of a pre-trained model according to the title image and its corresponding knowledge point labels.

2. The circuit exercise classification method based on the visual language model according to claim 1, It is characterized in that Fine-tuning the visual language model, specifically including: Collect original exercises from different distribution domains; Preprocessing the original exercise set to obtain a training sample data set, wherein each training sample in the training sample data set includes a question image and a corresponding knowledge point label, and the question image includes a question stem and an accompanying image; Dividing the training sample data set into a training set, a validation set and a test set; The pre-trained model is fine-tuned and tested according to the training set, the validation set and the test set to obtain a pre-trained visual language model.

3. The circuit exercise classification method based on the visual language model according to claim 2, It is characterized in that The preprocessing of the original exercise set to obtain a training sample data set includes: Converting the original exercise book into editable text, and extracting circuit principle exercises from the editable text; According to the circuit principle exercises, corresponding knowledge point labels are matched from a pre-stored knowledge point library to obtain the training sample data set.

4. The circuit exercise classification method based on the visual language model according to claim 3, It is characterized in that The pre-stored knowledge point library includes at least secondary knowledge points and tertiary knowledge points; wherein the secondary knowledge points contain multiple tertiary knowledge points.

5. The circuit exercise classification method based on the visual language model according to claim 2, It is characterized in that The step of fine-tuning the pre-trained model according to the verification set specifically includes: Monitoring the performance indicators of the pre-trained model on the validation set; Terminate training early when the performance metric stops improving or starts to degrade.

6. The circuit exercise classification method based on the visual language model according to claim 1, It is characterized in that Fine-tuning the visual language model, specifically including: Collect original exercises from different distribution domains; Preprocessing the original exercise set to obtain a training sample data set, wherein each training sample in the training sample data set includes a question image and its corresponding knowledge point label, and the question image includes a question stem, an accompanying image, and an accompanying image text description; Dividing the training sample data set into a training set, a validation set and a test set; The pre-trained model is fine-tuned and tested according to the training set, the validation set and the test set to obtain a pre-trained visual language model.

7. A circuit exercise classification device based on visual language model, It is characterized in that include: A circuit exercise acquisition module, used for acquiring circuit exercises to be classified; A circuit exercise classification module, used for predicting the knowledge point category corresponding to the circuit exercise based on a pre-trained visual language model; The visual language model is obtained by fine-tuning a training sample data set consisting of a pre-trained model according to the title image and its corresponding knowledge point labels.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the circuit exercise classification method based on the visual language model as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the circuit exercise classification method based on a visual language model as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the circuit exercise classification method based on a visual language model as described in any one of claims 1 to 6 is implemented.

Images