An experimental process evaluation method and device based on AI

Through the AI-based experimental process evaluation method, using models such as YOLOv4, ResNet18 and Faster R-CNN, the problems of students' wiring correctness and errors and facial expression behavior are solved, and the automated and fair evaluation of students' experimental ability is realized, and experimental teaching is improved. efficiency and safety.

CN114373206BActive Publication Date: 2024-11-26CIVIL AVIATION UNIV OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111612419.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-27
Publication Date
2024-11-26
Estimated Expiration
2041-12-27

AI Technical Summary

Technical Problem

In the prior art, students' wiring errors cannot be automatically judged, and facial expressions and behavioral actions are difficult to identify, making it difficult for teachers to conduct fair and fair experimental ability evaluation.

Method used

Using an AI-based experimental process evaluation method, by inputting preset wiring diagram information and obtaining line wiring information, we determine whether the wiring is correct, and using YOLOv4 and ResNet18 models to identify facial expressions and behavioral actions, combining Faster R-CNN algorithm and VGG16 model , realize a comprehensive evaluation of students' experimental ability.

Benefits of technology

It realizes automated and fair evaluation of students' experimental ability, reduces teachers' teaching burden, improves the efficiency and safety of experimental teaching, and can effectively understand the teaching effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114373206B_ABST
    Figure CN114373206B_ABST
Patent Text Reader

Abstract

The present invention provides an AI-based experimental process evaluation method and device, which relates to the technical field of artificial intelligence process evaluation, and specifically includes the following steps: inputting preset wiring diagram information, obtaining line wiring information and wiring completion instructions; in response to the wiring completion instruction, determining whether the wiring is correct based on the preset wiring diagram information and line wiring information; if so, obtaining the first experimental image information and performing facial expression recognition and behavioral action recognition on the experimental image information; if not, obtaining error information, and executing the preset wiring diagram information, obtaining line wiring information and wiring completion instructions. The method and device provided by the present invention can alleviate the technical problems in the prior art that the correctness of students' wiring cannot be automatically judged, facial expressions and behavioral actions are difficult to recognize, and teachers cannot fairly and impartially evaluate students' experimental abilities, so as to comprehensively evaluate students' experimental abilities and understand the teaching experimental effects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence process evaluation, and in particular to an AI-based experimental process evaluation method and device. Background Art

[0002] In the past, teachers evaluated students' experimental ability through their experimental preparation, experimental reports, and impressions of students. There was a lack of quantitative analysis of the experimental process, and the evaluation of students' experimental ability was not supported by data. In the process of experimental teaching, for electrical experiments, students often connect the wires first, and teachers check them, and then power on the power after confirming that they are correct, which increases the burden on teachers. At the same time, when conducting experimental evaluations, it is impossible to identify students' facial expressions and behavioral movements during the experiment; it is impossible to identify and count students' disgust, neutral and happy facial expressions; lying down, looking around, and playing with mobile phones, etc., and it is impossible to make a fair and just evaluation of students' experimental ability, and it is difficult to understand the effect of experimental teaching. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide an experimental process evaluation method and device based on AI to alleviate the technical problems in the prior art that students' wiring cannot be automatically judged, facial expressions and behavioral actions are difficult to recognize, and teachers cannot conduct a fair and just evaluation of students' experimental abilities, so as to comprehensively evaluate the experimental abilities of students and understand the teaching experimental effects.

[0004] The present invention provides an experimental process evaluation method based on AI, which specifically comprises the following steps:

[0005] Input the preset wiring diagram information, obtain the line wiring information and wiring completion instructions;

[0006] In response to the wiring completion instruction, determining whether the wiring is correct based on the preset wiring diagram information and the line wiring information;

[0007] If yes, obtaining the first experimental image information and performing facial expression recognition and behavior action recognition on the experimental image information;

[0008] If not, error information is obtained, and the preset wiring diagram information is executed to obtain line wiring information and wiring completion instructions.

[0009] Preferably, the steps of acquiring experimental image information and performing facial expression recognition and behavioral action recognition on the experimental image information include:

[0010] Using the YOLOv4 algorithm to transform the pixels of the first experimental image information to obtain the second experimental image information;

[0011] A VGG16 model is constructed to perform facial expression recognition on the second experimental image information, and a ResNet18 model is constructed to perform behavioral action recognition on the second experimental image information.

[0012] Preferably, the step of using the YOLOv4 algorithm to convert the first experimental image information pixels to obtain the second experimental image information includes:

[0013] The step of using the YOLOv4 algorithm to convert the pixels of the first experimental image information to obtain the second experimental image information includes:

[0014] Constructing a YOLOv4 network structure diagram, wherein the YOLOv4 network structure diagram includes a backbone feature extraction network, a first enhanced feature extraction network, a second enhanced feature extraction network, and a prediction result;

[0015] The backbone feature extraction network includes a Darknet convolutional layer and a residual network structure, and the backbone feature extraction network is used to obtain an effective feature layer of the first experimental image information;

[0016] The first enhanced feature network is used to perform pooling processing on the effective feature layer of the first experimental image information, and perform convolution processing on the effective feature layer of the first experimental image information after the pooling processing to obtain the feature layer of the first residual network of the first experimental image information;

[0017] Upsampling the first residual network structure layer to obtain a feature layer of the second residual network of the first experimental image information, performing feature fusion and convolution processing on the first residual network feature layer of the first experimental image information and a feature layer of the second residual network of the first experimental image information to obtain the first feature layer of the first experimental image information, wherein the second residual network is the penultimate layer of the residual network structure;

[0018] Upsampling the second residual network structure layer to obtain a feature layer of the third residual network of the first experimental image information, and performing feature fusion and convolution processing on a feature layer of the third residual network of the first experimental image information and the first feature layer to obtain a second feature layer of the first experimental image information, wherein the third residual network is the third to last layer of the residual network structure;

[0019] Downsampling the third residual network to obtain a feature layer of the second residual network for obtaining the first experimental image information, and performing feature fusion and convolution processing on the second feature layer of the first experimental image information and a feature layer of the second residual network to obtain a third feature layer of the first experimental image information;

[0020] Downsampling the second residual network structure layer to obtain the feature layer of the first residual network for obtaining the first experimental image information, and performing feature fusion and convolution processing on the third feature layer for obtaining the first experimental image information to obtain the fourth feature layer of the first experimental image information;

[0021] The predicted result obtains the second experimental image information based on the second feature layer of the first experimental image information, the third feature layer of the first experimental image information and the fourth feature layer of the first experimental image information.

[0022] Preferably, the step of determining whether the wiring is correct based on the preset wiring diagram information and the line wiring information includes:

[0023] Acquire a set of wires to be connected, a head end of a wire to be connected, and a terminal end of a wire to be connected based on the preset wiring diagram information;

[0024] Acquire the line connection information to acquire the beginning and the end of the line connection, and compare the beginning and the end of the line connection with the beginning and the end of the wire to be connected respectively to determine whether the line connection information determines whether the connection is correct;

[0025] If yes, the wire to be connected is deleted from the wire set to be connected, and the step of acquiring the wire set to be connected, the head end of the wire to be connected and the end of the wire to be connected based on the preset wiring diagram information is performed;

[0026] If not, a wiring error message will be displayed.

[0027] Preferably, the step of obtaining the line connection information to obtain the starting end and the ending end of the line connection, and comparing the starting end and the ending end of the line connection with the starting end and the ending end of the wire to be connected respectively to determine whether the line connection information is correct comprises:

[0028] Constructing a Faster R-CNN neural network Using the Faster R-CNN algorithm to compare the head end and the end of the line connection with the head end and the end of the wire to be connected to determine whether the line connection information is correct

[0029] Generate a CNN model based on the line wiring information to obtain the line wiring information feature map, the Faster R-CNN neural network includes a convolution layer, an RPN network layer, a pooling layer, and a classification and regression layer;

[0030] Inputting the line connection information into a convolutional layer to obtain a line connection information feature map;

[0031] Outputting the line connection information feature map to the RPN network layer to select feature regions and obtain a line connection information region feature map, and outputting the line connection information feature map to the pooling layer to perform image conversion on the line connection information feature map;

[0032] The line wiring information region feature map and the line wiring information feature map after pooling are input into the classification and regression layer to perform feature comparison on the line wiring information region feature map and the line wiring information feature map after pooling, so as to respectively compare the starting end of the line wiring and the end of the line wiring with the starting end of the wire to be connected and the end of the wire to be connected.

[0033] Preferably, the VGG16 model includes 16 weight layers and 5 pooling layers, the 16 weight layers include 13 convolutional layers and 3 fully connected layers, and the VGG16 model uses softmax as an activation function.

[0034] Preferably, the ResNet18 model includes a first convolutional layer, 4 ResBlocks, each of which contains 2 Basic Blocks, each of which contains two convolutional layers, and a final fully connected layer.

[0035] In another aspect, the present invention provides an AI-based experimental process evaluation device, comprising:

[0036] Instruction acquisition module: used to obtain preset wiring diagram information, line wiring information and wiring completion instructions;

[0037] A determination module: configured to respond to the wiring completion instruction and determine whether the wiring is correct based on the preset wiring diagram information and the line wiring information;

[0038] If yes, obtaining the first experimental image information and performing facial expression recognition and behavior action recognition on the experimental image information;

[0039] If not, error information is obtained, and the preset wiring diagram information is executed to obtain line wiring information and wiring completion instructions.

[0040] The embodiments of the present invention bring the following beneficial effects: The present invention provides an AI-based experimental process evaluation method and device, which specifically includes the following steps: inputting preset wiring diagram information, obtaining line wiring information and wiring completion instructions; in response to the wiring completion instructions, determining whether the wiring is correct based on the preset wiring diagram information and line wiring information; if so, obtaining the first experimental image information and performing facial expression recognition and behavioral action recognition on the experimental image information; if not, obtaining error information, and executing the preset wiring diagram information, obtaining line wiring information and wiring completion instructions. The method and device provided by the present invention can alleviate the technical problems in the prior art that the correctness of students' wiring cannot be automatically judged, facial expressions and behavioral actions are difficult to recognize, and teachers cannot fairly and impartially evaluate students' experimental abilities, so as to comprehensively evaluate the polarity of students' experimental abilities and understand the teaching experimental effect.

[0041] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flow chart of the first AI-based experimental process evaluation method provided in an embodiment of the present invention;

[0045] Figure 2 A YOLOv4 network structure diagram of an AI-based experimental process evaluation method provided in an embodiment of the present invention;

[0046] Figure 3 A flow chart of a second AI-based experimental process evaluation method provided in an embodiment of the present invention;

[0047] Figure 4 A flow chart of the Faster R-CNN algorithm of the first AI-based experimental process evaluation method provided in an embodiment of the present invention;

[0048] Figure 5 A VGG16 model structure diagram of an AI-based experimental process evaluation method provided in an embodiment of the present invention;

[0049] Figure 6 A ResNet18 model structure diagram of an AI-based experimental process evaluation method provided in an embodiment of the present invention;

[0050] Figure 7 A schematic diagram of the coordinates of the wiring holes and the positions of the conductors in an AI-based experimental process evaluation method provided in an embodiment of the present invention;

[0051] Figure 8 A flow chart of a third AI-based experimental process evaluation method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] At present, in previous experimental teaching, teachers evaluate students' experimental ability through students' experimental preparation, experimental reports and impressions of students. There is a lack of quantitative analysis of the experimental process, and the evaluation of students' experimental ability has no data support. Based on this, an AI-based experimental process evaluation method and device provided in an embodiment of the present invention can alleviate the technical problems in the prior art that students' wiring cannot be automatically judged, facial expressions and behavioral actions are difficult to recognize, and teachers cannot conduct fair and just evaluation of students' experimental ability, so as to comprehensively evaluate the polarity of students' experimental ability and understand the effect of teaching experiments.

[0054] To facilitate understanding of this embodiment, an AI-based experimental process evaluation method disclosed in an embodiment of the present invention is first introduced in detail.

[0055] like Figure 1 As shown, the present invention provides an experimental process evaluation method based on AI, which specifically includes the following steps:

[0056] Input the preset wiring diagram information, obtain the line wiring information and wiring completion instructions;

[0057] In response to the wiring completion instruction, determining whether the wiring is correct based on the preset wiring diagram information and the line wiring information;

[0058] If yes, obtaining the first experimental image information and performing facial expression recognition and behavior action recognition on the experimental image information;

[0059] If not, error information is obtained, and the preset wiring diagram information is executed to obtain line wiring information and wiring completion instructions.

[0060] like Figure 8 As shown, preferably, the steps of acquiring experimental image information and performing facial expression recognition and behavioral action recognition on the experimental image information include:

[0061] Using the YOLOv4 algorithm to transform the pixels of the first experimental image information to obtain the second experimental image information;

[0062] A VGG16 model is constructed to perform facial expression recognition on the second experimental image information, and a ResNet18 model is constructed to perform behavioral action recognition on the second experimental image information.

[0063] Combination Figure 2 Preferably, the step of using the YOLOv4 algorithm to convert the first experimental image information pixels to obtain the second experimental image information includes:

[0064] Constructing a YOLOv4 network structure diagram, wherein the YOLOv4 network structure diagram includes a backbone feature extraction network, a first enhanced feature extraction network, a second enhanced feature extraction network, and a prediction result;

[0065] The backbone feature extraction network includes a Darknet convolutional layer and a residual network structure, and the backbone feature extraction network is used to obtain an effective feature layer of the first experimental image information;

[0066] The first enhanced feature network is used to perform pooling processing on the effective feature layer of the first experimental image information, and perform convolution processing on the effective feature layer of the first experimental image information after the pooling processing to obtain the feature layer of the first residual network of the first experimental image information;

[0067] Upsampling the first residual network structure layer to obtain a feature layer of the second residual network of the first experimental image information, performing feature fusion and convolution processing on the first residual network feature layer of the first experimental image information and a feature layer of the second residual network of the first experimental image information to obtain the first feature layer of the first experimental image information, wherein the second residual network is the penultimate layer of the residual network structure;

[0068] Upsampling the second residual network structure layer to obtain a feature layer of the third residual network of the first experimental image information, and performing feature fusion and convolution processing on a feature layer of the third residual network of the first experimental image information and the first feature layer to obtain a second feature layer of the first experimental image information, wherein the third residual network is the third to last layer of the residual network structure;

[0069] Downsampling the third residual network to obtain a feature layer of the second residual network for obtaining the first experimental image information, and performing feature fusion and convolution processing on the second feature layer of the first experimental image information and a feature layer of the second residual network to obtain a third feature layer of the first experimental image information;

[0070] Downsampling the second residual network structure layer to obtain the feature layer of the first residual network for obtaining the first experimental image information, and performing feature fusion and convolution processing on the third feature layer for obtaining the first experimental image information to obtain the fourth feature layer of the first experimental image information;

[0071] The predicted result obtains the second experimental image information based on the second feature layer of the first experimental image information, the third feature layer of the first experimental image information and the fourth feature layer of the first experimental image information.

[0072] In an embodiment provided by the present invention, the first enhanced feature extraction network is an SPP structure, and the second enhanced feature extraction network is a PANet structure;

[0073] It should be noted that in the embodiment provided by the present invention, the first experimental image information is obtained as a picture of (416, 416, 3), and after a Darknet convolution and a series of residual network structures, it is continuously downsampled to obtain higher semantic information, and finally the last three feature layers with higher semantic information are obtained for the next step. Then, the effective feature layer of (13, 13, 1024) is convolved three times to enter the SPP structure. The SPP structure uses maximum pooling of different sizes and performs maximum pooling on the input feature layer. The pooled results are stacked and then convolved three times. Secondly, the feature layer after the three convolutions is upsampled once to obtain a feature layer of (26, 26, 512), and the feature layer of (26, 26, 512) obtained by the backbone feature network is stacked to realize feature fusion, and then convolution is performed five times; upsampling, feature fusion and convolution operations are continued to obtain a feature layer of (52, 52, 256). After that, the PANet structure is continued to be used to further deepen the feature extraction, and downsampling, feature fusion and convolution operations are continued to obtain effective features. Finally, Yolo Head predicts the results of the extracted features

[0074] Preferably, the step of determining whether the wiring is correct based on the preset wiring diagram information and the line wiring information includes:

[0075] like Figure 3 as well as Figure 7 As shown, based on the preset wiring diagram information, a set of wires to be connected, the head ends of the wires to be connected, and the end ends of the wires to be connected are obtained;

[0076] Acquire the line connection information to acquire the beginning and the end of the line connection, and compare the beginning and the end of the line connection with the beginning and the end of the wire to be connected respectively to determine whether the line connection information determines whether the connection is correct;

[0077] If yes, the wire to be connected is deleted from the wire set to be connected, and the step of acquiring the wire set to be connected, the head end of the wire to be connected and the end of the wire to be connected based on the preset wiring diagram information is performed;

[0078] If not, a wiring error message will be displayed.

[0079] like Figure 4 As shown, preferably, the step of obtaining the line wiring information to obtain the head end and the end end of the line wiring, and comparing the head end and the end end of the line wiring with the head end and the end end of the wire to be connected respectively to determine whether the line wiring information is correct includes:

[0080] Constructing a Faster R-CNN neural network using a Faster R-CNN algorithm to compare the head end of the line connection and the end of the line connection with the head end of the wire to be connected and the end of the wire to be connected respectively to determine whether the line connection information is correct;

[0081] Generate a CNN model based on the line wiring information to obtain the line wiring information feature map, the Faster R-CNN neural network includes a convolution layer, an RPN network layer, a pooling layer, and a classification and regression layer;

[0082] Inputting the line connection information into a convolutional layer to obtain a line connection information feature map;

[0083] Outputting the line connection information feature map to the RPN network layer to select feature regions and obtain a line connection information region feature map, and outputting the line connection information feature map to the pooling layer to perform image conversion on the line connection information feature map;

[0084] The line wiring information region feature map and the line wiring information feature map after pooling are input into the classification and regression layer to perform feature comparison on the line wiring information region feature map and the line wiring information feature map after pooling, so as to respectively compare the starting end of the line wiring and the end of the line wiring with the starting end of the wire to be connected and the end of the wire to be connected.

[0085] Preferably, Figure 5 As shown, the VGG16 model includes 16 weight layers and 5 pooling layers, the 16 weight layers include 13 convolutional layers and 3 fully connected layers, and the VGG16 model uses softmax as the activation function.

[0086] Furthermore, the VGG16 model has a total of 6 block structures, and the number of channels in each block structure is the same. VGG16 contains a total of 16 weight layers, including 13 convolutional layers and 3 fully connected layers; it also contains 5 pooling layers. Among them, 13 convolutional layers and 5 pooling layers are responsible for feature extraction, and the last 3 fully connected layers are responsible for completing the classification task;

[0087] Preferably, Figure 6 As shown, the ResNet18 model includes a convolutional layer at the beginning, 4 ResBlocks, each ResBlock contains 2 Basic Blocks, each Basic Block contains two convolutional layers, and a final fully connected layer, that is, 1+4*2*2+1=18.

[0088] Furthermore, the ResNet18 model contains a total of 18 weight layers, including 17 convolutional layers and 1 fully connected layer. The network structure contains two different basic blocks. The first is the part that implements the labeled jump connection. It performs residual structure operations while the number of channels remains unchanged, such as the solid jump lines in the network diagram; the second jump structure has a changed number of channels, such as the virtual jump lines in the network diagram. The convolutional layer and the pooling layer are responsible for feature extraction, and the fully connected layer is responsible for completing the classification task.

[0089] In the embodiment provided by the present invention, in order to evaluate the experimental process, finally, the classified behavioral actions and facial expressions as well as the number of wiring errors and the experimental completion time are input into the performance evaluation module.

[0090] The classified actions and expressions are input into the scoring criteria module according to the categories for judgment. If there are any illegal behaviors (lying down, looking around, playing with mobile phones, etc.), corresponding points will be deducted; the judgment of expressions will be added or subtracted accordingly (disgust will be deducted, neutral will not be deducted, and happiness will be added). The number of wiring errors and the time to complete the experiment are combined to give the students a score for the experimental process. The steps of the scoring system are as follows (assuming that the students follow the experimental process and complete the experiment within the specified time):

[0091] Set a student's initial score of 10 points, and add or subtract points accordingly for the discrimination results. For example: if the student is identified to be lying down, subtract 1 point; continue to identify whether there is any movement of looking around, if so, subtract 2 points; continue to identify whether there is any movement of playing with the phone, if so, subtract 3 points, and use the cumulative subtraction measure for judgment. Facial expressions are judged by the reward and punishment measures of disgust deduction, neutrality no points or no points, and happiness plus points. The number of wiring errors is judged by the reward and punishment measures of 0 errors plus 1 point, 1 error minus 1 point, and more than 1 error minus 2 points. The time to complete the experiment is judged by the reward measures of 0-20 minutes plus 3 points, 21-25 minutes plus 2 points, 26-30 minutes plus 1 point, and more than 30 minutes plus 0 points. The scoring criteria are shown in Table 1;

[0092] Table 1 Scoring criteria

[0093]

[0094] Embodiment 2:

[0095] Embodiment 2 of the present invention provides an AI-based experimental process evaluation device, comprising:

[0096] A determination module: configured to respond to the wiring completion instruction and determine whether the wiring is correct based on the preset wiring diagram information and the line wiring information;

[0097] If yes, obtaining the first experimental image information and performing facial expression recognition and behavior action recognition on the experimental image information;

[0098] If not, error information is obtained, and the preset wiring diagram information is executed to obtain line wiring information and wiring completion instructions.

[0099] The present invention has the following specific advantages:

[0100] (1) Based on traditional experimental teaching, the present invention uses video monitoring to automatically judge whether the wiring of electrical experiments is correct or not. This not only reduces the teaching burden of teachers, but also standardizes students' operations, thereby reducing the occurrence of experimental teaching accidents.

[0101] (2) The present invention adopts the currently popular and mature deep learning method, and uses behavior recognition technology and face recognition technology to extract and classify students' behavior movements and facial expressions during the experimental operation, and calculates the students' experimental scores through a set of evaluation systems to achieve a relatively fair and just effect. At the same time, it can also feedback students' learning effects, provide teachers with useful information, and help improve experimental teaching.

[0102] (3) The present invention solves the problem that teachers only rely on students' experimental preparation and experimental reports in the current experimental teaching process, but ignore the students' experimental process, which is unable to comprehensively evaluate students' experimental ability. It not only greatly reduces teachers' teaching energy, but also improves students' learning awareness through video supervision.

[0103] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0104] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0106] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0107] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0108] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An AI-based experimental process evaluation method, characterized in that: The specific steps include: Input the preset wiring diagram information, obtain the line wiring information and wiring completion instructions; In response to the wiring completion instruction, determining whether the wiring is correct based on the preset wiring diagram information and the line wiring information; If yes, obtaining the first experimental image information and performing facial expression recognition and behavior action recognition on the experimental image information; If not, then obtaining error information, and executing the preset wiring diagram information, obtaining line wiring information and wiring completion instructions; The step of determining whether the wiring is correct based on the preset wiring diagram information and the line wiring information comprises: Acquire a set of wires to be connected, a head end of a wire to be connected, and a terminal end of a wire to be connected based on the preset wiring diagram information; Acquire the line connection information to acquire the beginning and the end of the line connection, and compare the beginning and the end of the line connection with the beginning and the end of the wire to be connected respectively to determine whether the line connection information determines whether the connection is correct; If yes, the wire to be connected is deleted from the wire set to be connected, and the step of acquiring the wire set to be connected, the head end of the wire to be connected and the end of the wire to be connected based on the preset wiring diagram information is performed; If not, a wiring error message will be displayed; The step of obtaining the line connection information to obtain the head end and the end end of the line connection, and comparing the head end and the end end of the line connection with the head end and the end end of the wire to be connected respectively to determine whether the line connection information is correct comprises: Constructing a Faster R-CNN neural network Using the Faster R-CNN algorithm to compare the head end and the end of the line connection with the head end and the end of the wire to be connected to determine whether the line connection information is correct Generate a CNN model based on the line wiring information to obtain the line wiring information feature map, the Faster R-CNN neural network includes a convolution layer, an RPN network layer, a pooling layer, and a classification and regression layer; Inputting the line connection information into a convolutional layer to obtain a line connection information feature map; Outputting the line connection information feature map to the RPN network layer to select feature regions and obtain a line connection information region feature map, and outputting the line connection information feature map to the pooling layer to perform image conversion on the line connection information feature map; The line wiring information region feature map and the line wiring information feature map after pooling are input into the classification and regression layer to perform feature comparison on the line wiring information region feature map and the line wiring information feature map after pooling, so as to respectively compare the starting end of the line wiring and the end of the line wiring with the starting end of the wire to be connected and the end of the wire to be connected.

2. The method according to claim 1, characterized in that The steps of acquiring experimental image information and performing facial expression recognition and behavior action recognition on the experimental image information include: Using the YOLOv4 algorithm to transform the pixels of the first experimental image information to obtain the second experimental image information; A VGG16 model is constructed to perform facial expression recognition on the second experimental image information, and a ResNet18 model is constructed to perform behavioral action recognition on the second experimental image information.

3. The method according to claim 2, characterized in that The step of using the YOLOv4 algorithm to convert the pixels of the second experimental image information to obtain the second experimental image information includes: Constructing a YOLOv4 network structure diagram, wherein the YOLOv4 network structure diagram includes a backbone feature extraction network, a first enhanced feature extraction network, a second enhanced feature extraction network, and a prediction result; The backbone feature extraction network includes a Darknet convolutional layer and a residual network structure, and the backbone feature extraction network is used to obtain an effective feature layer of the first experimental image information; The first enhanced feature network is used to perform pooling processing on the effective feature layer of the first experimental image information, and perform convolution processing on the effective feature layer of the first experimental image information after the pooling processing to obtain the feature layer of the first residual network of the first experimental image information; Upsampling the first residual network structure layer to obtain a feature layer of the second residual network of the first experimental image information, performing feature fusion and convolution processing on the first residual network feature layer of the first experimental image information and a feature layer of the second residual network of the first experimental image information to obtain the first feature layer of the first experimental image information, wherein the second residual network is the penultimate layer of the residual network structure; Upsampling the second residual network structure layer to obtain a feature layer of the third residual network of the first experimental image information, and performing feature fusion and convolution processing on a feature layer of the third residual network of the first experimental image information and the first feature layer to obtain a second feature layer of the first experimental image information, wherein the third residual network is the third to last layer of the residual network structure; Downsampling the third residual network to obtain a feature layer of the second residual network for obtaining the first experimental image information, and performing feature fusion and convolution processing on the second feature layer of the first experimental image information and a feature layer of the second residual network to obtain a third feature layer of the first experimental image information; Downsampling the second residual network structure layer to obtain the feature layer of the first residual network for obtaining the first experimental image information, and performing feature fusion and convolution processing on the third feature layer for obtaining the first experimental image information to obtain the fourth feature layer of the first experimental image information; The predicted result obtains the second experimental image information based on the second feature layer of the first experimental image information, the third feature layer of the first experimental image information and the fourth feature layer of the first experimental image information.

4. The method according to claim 2, characterized in that: The VGG16 model includes 16 weight layers and 5 pooling layers, the 16 weight layers include 13 convolutional layers and 3 fully connected layers, and the VGG16 model uses softmax as the activation function.

5. The method according to claim 2, characterized in that: The ResNet18 model includes a convolutional layer at the beginning, 4 ResBlocks, each of which contains 2 Basic Blocks, each of which contains two convolutional layers, and a final fully connected layer.

6. An AI-based experimental process evaluation device, characterized in that: include: Instruction acquisition module: used to obtain preset wiring diagram information, line wiring information and wiring completion instructions; A determination module: configured to respond to the wiring completion instruction and determine whether the wiring is correct based on the preset wiring diagram information and the line wiring information; If yes, obtaining the first experimental image information and performing facial expression recognition and behavior action recognition on the experimental image information; If not, then obtaining error information, and executing the preset wiring diagram information, obtaining line wiring information and wiring completion instructions; Wherein, the step of determining whether the wiring is correct based on the preset wiring diagram information and the line wiring information includes: Acquire a set of wires to be connected, a head end of a wire to be connected, and a terminal end of a wire to be connected based on the preset wiring diagram information; Acquire the line connection information to acquire the beginning and the end of the line connection, and compare the beginning and the end of the line connection with the beginning and the end of the wire to be connected respectively to determine whether the line connection information determines whether the connection is correct; If yes, the wire to be connected is deleted from the wire set to be connected, and the step of acquiring the wire set to be connected, the head end of the wire to be connected and the end of the wire to be connected based on the preset wiring diagram information is performed; If not, a wiring error message will be displayed; The step of obtaining the line connection information to obtain the head end and the end end of the line connection, and comparing the head end and the end end of the line connection with the head end and the end end of the wire to be connected respectively to determine whether the line connection information is correct comprises: Constructing a Faster R-CNN neural network Using the Faster R-CNN algorithm to compare the head end and the end of the line connection with the head end and the end of the wire to be connected to determine whether the line connection information is correct Generate a CNN model based on the line wiring information to obtain the line wiring information feature map, the Faster R-CNN neural network includes a convolution layer, an RPN network layer, a pooling layer, and a classification and regression layer; Inputting the line connection information into a convolutional layer to obtain a line connection information feature map; Outputting the line connection information feature map to the RPN network layer to select feature regions and obtain a line connection information region feature map, and outputting the line connection information feature map to the pooling layer to perform image conversion on the line connection information feature map; The line wiring information region feature map and the line wiring information feature map after pooling are input into the classification and regression layer to perform feature comparison on the line wiring information region feature map and the line wiring information feature map after pooling, so as to respectively compare the starting end of the line wiring and the end of the line wiring with the starting end of the wire to be connected and the end of the wire to be connected.