A circuit connection parsing method based on image deep learning

Abstract

The application discloses a circuit connection analysis method based on image deep learning, comprising the following steps: first, adjusting an original image to a fixed size, then performing preliminary feature extraction on the adjusted original image and downsampling to obtain a first feature map, extracting position features from the first feature map and processing to obtain a second feature map, judging whether the features in the second feature map match, calculating loss according to the matching relationship, respectively performing multiple upsampling on the first feature map and performing information labeling, respectively calculating segmentation loss and connection point positioning partial loss, inputting an image, outputting a segmentation result of a wire, an end point position corresponding to the wire and a feature vector corresponding to the end point, and judging whether the end points are in a connection state according to the feature vector corresponding to the end points. The application judges whether the circuit is connected correctly by the student during the experiment through the above steps, so that the teacher can better know the mastery of the student on the knowledge points.

Description

Technical Field

[0001] This invention relates to the field of teaching experiments, and in particular to a circuit connection analysis method based on image deep learning. Background Technology

[0002] In the process of physics experiment teaching, in order to deepen students' understanding of knowledge points, improve the effectiveness of teaching experiments, and cultivate students' practical operation ability and engineering application ability, it is necessary to conduct teaching test experiments for students.

[0003] After students complete the teaching experiments, it is necessary to evaluate whether the circuits they connected are accurate in order to provide corrections. However, due to the large number of students, teachers cannot ensure that every student receives individual attention, resulting in teachers being unable to ascertain the understanding of some students' grasp of the knowledge points, thus affecting the teaching effectiveness. Therefore, a circuit connection analysis method based on image deep learning is needed to solve the above problems. Summary of the Invention

[0004] The purpose of this invention is to provide a circuit connection analysis method based on image deep learning, so as to facilitate the detection of whether the student circuit is connected correctly.

[0005] To address the aforementioned technical problems, this invention provides a circuit connection analysis method based on image deep learning, comprising the following steps:

[0006] Resize the original image to a fixed size;

[0007] Preliminary feature extraction is performed on the adjusted original image, and then downsampling is performed to obtain the first feature map;

[0008] The positional features are extracted and processed at the location of the first feature map to obtain the second feature map;

[0009] The matching of features in the second feature map is determined based on their distance from the unit sphere, and the loss is calculated based on the matching relationship. The method for determining whether features in the second feature map match is as follows:

[0010] The determination is made based on the distance of the described positional features on the unit sphere;

[0011] Matching features are close to each other on the unit sphere, while all other non-matching features are far apart on the unit sphere.

[0012] The first feature map is upsampled multiple times and labeled with information. The segmentation loss and the connection point localization loss are calculated respectively.

[0013] Input image, output wire segmentation result, wire endpoint position and endpoint feature vector;

[0014] The endpoint is determined to be in a connected state based on the feature vector corresponding to the endpoint.

[0015] Furthermore, the step of obtaining the first feature map includes: extracting preliminary features from the adjusted original image using a convolutional neural network, downsampling the preliminary features, and obtaining the first feature map.

[0016] Furthermore, the step of obtaining the second feature map includes: using the ROI-Pooling method to extract the position features corresponding to the coordinate downsampling position of the first feature map, and obtaining the second feature map by adjusting the position features to a uniform size.

[0017] Furthermore, the loss calculation is a point-to-point comparison loss calculation, and the point-to-point comparison loss calculation formula is:

[0018]

[0019] Where i and j are any two matching points. and Let represent their corresponding local features respectively. sim is the cosine similarity function. The numerator of this loss will increase the cosine similarity of the matching points, and the denominator is all the non-matching points. This part will decrease the cosine similarity of all the non-matching points.

[0020] The denominator consists of all the mismatched points, which reduces the cosine similarity of all the mismatched points.

[0021] Furthermore, the method for upsampling the first feature map is as follows: a convolutional network with residuals is connected after the first feature map, and transposed convolution is performed.

[0022] Furthermore, the information annotation includes traverse information annotation and connector information annotation. The segmentation loss is obtained by calculating the L2 function loss based on the traverse information annotation, and the connector localization loss is obtained by calculating the cross-entropy function loss based on the connector information annotation.

[0023] Furthermore, the method for determining the connection state of the endpoints is as follows: calculate the distance between the feature vectors corresponding to the endpoints pairwise on a unit sphere. If the distance is less than a certain threshold, the two endpoints are considered to be in a connected state; otherwise, they are considered to be in a disconnected state.

[0024] Compared with the prior art, the present invention has at least the following beneficial effects:

[0025] This invention uses a series of algorithms to output the segmentation results of the wires, the position of the corresponding endpoints of the wires, and the feature vectors corresponding to the endpoints, in order to determine whether the endpoints are in a connected state, that is, to identify the connection relationship between the wires, thereby completing the detection of students' experimental performance, thereby reducing the workload of teachers and making it easier for teachers to understand students' mastery of knowledge points.

[0026] This invention is highly robust and can still perform the detection function when the wire connection is in a state of crossing or obstruction. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the circuit connection analysis method based on image deep learning of the present invention.

[0028] Figure 2 This is a flowchart illustrating the training mode of the circuit connection parsing method based on image deep learning of the present invention.

[0029] Figure 3 This is a flowchart illustrating the inference / testing mode of the circuit connection parsing method based on image deep learning of the present invention. Detailed Implementation

[0030] The circuit connection parsing method based on image deep learning of the present invention will be described in more detail below with reference to the schematic diagrams, which illustrate preferred embodiments of the present invention. It should be understood that those skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Therefore, the following description should be understood as being of broad knowledge to those skilled in the art and is not intended to limit the present invention.

[0031] The invention is described more specifically by way of example in the following paragraphs with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description and claims. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0032] like Figure 1 As shown in the figure, this embodiment of the invention proposes a circuit connection parsing method based on image deep learning, which includes the following steps:

[0033] Resize the original image to a fixed size;

[0034] Preliminary feature extraction is performed on the adjusted original image, and then downsampling is performed to obtain the first feature map;

[0035] The positional features are extracted and processed at the location of the first feature map to obtain the second feature map;

[0036] Determine whether the features in the second feature map match, and calculate the loss based on the matching relationship;

[0037] The first feature map is upsampled multiple times and labeled with information. The segmentation loss and the connection point localization loss are calculated respectively.

[0038] Input image, output wire segmentation result, wire endpoint position and endpoint feature vector;

[0039] The endpoint is determined to be in a connected state based on the feature vector corresponding to the endpoint.

[0040] In this embodiment, the algorithm is first trained by simulation. When testing is required, the image of the experiment performed by the student is input, thereby outputting the segmentation result of the wire, the position of the endpoint of the wire, and the feature vector of the endpoint. Based on the feature vector of the endpoint, it is determined whether the endpoint is in a connected state, that is, the connection relationship between the wires is identified, thereby completing the detection of the student's experimental situation. This reduces the workload of the teacher and makes it easier for the teacher to better understand the students' mastery of the knowledge points.

[0041] The following are preferred embodiments of the circuit connection parsing method based on image deep learning to clearly illustrate the content of the present invention. It should be understood that the content of the present invention is not limited to the following embodiments, and other improvements made by conventional technical means by those skilled in the art are also within the scope of the present invention.

[0042] Specifically, the steps for obtaining the first feature map include: extracting preliminary features from the adjusted original image using a convolutional neural network, downsampling the preliminary features, and obtaining the first feature map. Preferably, the preliminary features are downsampled by a factor of 0.25.

[0043] Convolutional neural networks (CNNs) are a type of feedforward neural network that includes convolutional computation and has a deep structure. They are one of the representative algorithms of deep learning.

[0044] Furthermore, to facilitate the extraction of positional features from the first feature map, the ROI-Pooling method is used to extract the positional features corresponding to the coordinate downsampling positions of the first feature map, and these features are then resized. Specifically, the positional features are adjusted to a uniform size and denoted as a second feature map for easy comparison with the first feature map.

[0045] Furthermore, the method for determining whether features in the second feature map match is as follows: the distance (or similarity) of the position features on the unit sphere is used for judgment. Matching features are close to each other on the unit sphere, while all other unmatched features are far apart on the unit sphere.

[0046] Furthermore, the loss calculation is a point-to-point comparison loss calculation, and the point-to-point comparison loss calculation formula is:

[0047]

[0048] Where i and j are any two matching points. and Let represent the corresponding local features respectively. sim is the cosine similarity function. The numerator of this loss will increase the cosine similarity of the matching points. The denominator is all the non-matching points, which will decrease the cosine similarity of all non-matching points.

[0049] Minimizing the contrastive loss described above is equivalent to maximizing the mutual information between any two matched points; therefore, the theoretical basis of this loss is mutual information maximization. This loss converges faster than other types of losses, has a sound theoretical basis, and ultimately achieves the best results. The above formula treats the point contrastive loss matrix as an undirected graph loss. The advantage of this is that it eliminates the need to consider the order of points. In reality, we lack a good criterion for distinguishing the order of connected points in an image, and undirected graphs effectively circumvent this problem.

[0050] Furthermore, the method for upsampling the first feature map is as follows: a convolutional network with residuals is appended to the first feature map, and then transposed convolution is performed. Upsampling the first feature map facilitates the subsequent calculation of segmentation loss and localization loss.

[0051] Furthermore, the information annotation includes traverse information annotation and connector information annotation. The segmentation loss is obtained by calculating the L2 function loss based on the traverse information annotation, and the connector localization loss is obtained by calculating the cross-entropy function loss based on the connector information annotation.

[0052] The L2 function loss is also called the Least Square Error (LSE). LSE minimizes the true value. and predicted value Difference between The sum of the squares. The formula is as follows:

[0053]

[0054] Mean absolute error;

[0055] The formula for cross-entropy loss is as follows:

[0056]

[0057] Where f is the neural network activation function, y is the label, and x is the input image.

[0058] In summary, the segmentation loss and the loss from locating the connection point can be quickly calculated using the annotation of traverse information and connection point information.

[0059] Furthermore, the method for determining the connection state of the endpoints is as follows: calculate the distance between the feature vectors corresponding to the endpoints pairwise on a unit sphere. If the distance is less than a certain threshold, the two endpoints are considered to be in a connected state; otherwise, they are considered to be in a disconnected state.

[0060] In summary, when students take tests, the system recognizes the images of students' operations, and then outputs the segmentation results of the wires, the corresponding endpoint positions of the wires, and the feature vectors corresponding to the endpoints through an internal algorithm. Finally, the system compares the feature vectors corresponding to the endpoints to determine the distance between the endpoint positions. If the distance is less than a threshold, it is considered a connected state; if it is greater than the threshold, it is considered a disconnected state, thereby determining whether the wires are connected incorrectly.

[0061] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A circuit connection analysis method based on image deep learning, characterized in that, Includes the following steps: Resize the original image to a fixed size; Preliminary feature extraction is performed on the adjusted original image, and then downsampling is performed to obtain the first feature map; The positional features are extracted and processed at the location of the first feature map to obtain the second feature map; The matching of features in the second feature map is determined based on their distance from the unit sphere, and the loss is calculated based on the matching relationship. The method for determining whether features in the second feature map match is as follows: The determination is made based on the distance of the described positional features on the unit sphere; Matching features are close to each other on the unit sphere, while all other non-matching features are far apart on the unit sphere. The first feature map is upsampled multiple times and labeled with information. The segmentation loss and the connection point localization loss are calculated respectively. The input image is a line segmentation result, the endpoint position of the line, and the feature vector corresponding to the endpoint. The feature vector corresponding to the endpoint is used to determine the connection status of the endpoint. The endpoint is determined to be in a connected state based on the feature vector corresponding to the endpoint.

2. The circuit connection parsing method based on image deep learning as described in claim 1, characterized in that, The steps for obtaining the first feature map include: The first feature map is obtained by extracting preliminary features from the adjusted original image using a convolutional neural network and then downsampling the preliminary features.

3. The circuit connection parsing method based on image deep learning as described in claim 1, characterized in that, The steps to obtain the second feature map include: The ROI-Pooling method is used to extract the positional features corresponding to the coordinate downsampling positions of the first feature map. By adjusting the positional features to a uniform size, the second feature map is obtained.

4. The circuit connection parsing method based on image deep learning as described in claim 1, characterized in that, The loss calculation is a point-to-point comparison loss calculation, and the formula for the point-to-point comparison loss calculation is: Where i and j are any two matching points. and Let represent the corresponding local features respectively, sim is the cosine similarity, the numerator of which increases the cosine similarity of the matching points, and the denominator is all the non-matching points, which decreases the cosine similarity of all the non-matching points.

5. The circuit connection parsing method based on image deep learning as described in claim 1, characterized in that, The method for upsampling the first feature map is as follows: a convolutional network with residuals is connected after the first feature map, and then transposed convolution is performed.

6. The circuit connection parsing method based on image deep learning as described in claim 1 or 5, characterized in that, The information annotation includes conductor information annotation and connection point information annotation; The segmentation loss is obtained by calculating the L2 function loss based on the conductor information annotation; The loss for locating the connection points is calculated using the cross-entropy function based on the connection point information annotations.

7. The circuit connection parsing method based on image deep learning as described in claim 1, characterized in that, The method for determining the endpoint connection status is as follows: Calculate the distance between each pair of feature vectors corresponding to the endpoints on a unit sphere. If the distance is less than a certain threshold, the two endpoints are considered to be connected; otherwise, they are considered disconnected.

Images