A method for generating and storing physical experiment circuit diagrams based on neural networks.

Abstract

This application relates to a method and storage medium for generating physical experiment circuit diagrams based on neural networks, and pertains to the field of image recognition technology. It includes the following steps: acquiring an image of the experimental circuit diagram to be recognized and performing image processing on it, wherein the experimental circuit diagram includes component diagrams and wire connection diagrams; segmenting the experimental circuit diagram based on a neural network to obtain component heatmaps and wire segmentation diagrams; calculating the segmentation loss of the component heatmaps and wire segmentation diagrams; and using the component heatmaps and wire segmentation diagrams as input to a sub-convolutional network to generate a physical circuit diagram, wherein the physical circuit diagram consists only of circuit symbols. This application has the effect of facilitating the detection of whether students' circuit connections are correct.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method and storage medium for generating physical experimental circuit diagrams based on neural networks. Background Technology

[0002] Physics circuit diagram recognition refers to the use of machines to extract key information from circuit diagrams in practice problems, thereby further analyzing the knowledge attributes within the diagrams. As a crucial prerequisite for automated solution technology for physics circuit problems, the accuracy of circuit diagram recognition directly impacts the accuracy of subsequent reasoning and solutions.

[0003] In related technologies, students, under the guidance of their instructors, need to connect various components using wires during physics electrical circuit experiments. After the circuit is connected, the instructor needs to check it to determine if the connection is correct. However, the instructor needs to check each student's circuit diagram individually, and since the connection is made using wires, the checking process is time-consuming and laborious, requiring improvement. Summary of the Invention

[0004] To facilitate the detection of whether students' circuit connections are correct, this application provides a method for generating physical experiment circuit diagrams based on neural networks and a storage medium.

[0005] Firstly, the method for generating physical experiment circuit diagrams based on neural networks provided in this application adopts the following technical solution:

[0006] A method for generating physical experiment circuit diagrams based on neural networks includes the following steps:

[0007] The image of the experimental circuit diagram to be identified is acquired and processed. The experimental circuit diagram to be identified includes component diagrams and wire connection diagrams.

[0008] The experimental circuit diagram is segmented based on a neural network to obtain component thermal diagrams and wire segmentation diagrams.

[0009] Calculate the segmentation loss of the element's thermal diagram and the wire segmentation diagram;

[0010] The element heatmap and wire segmentation map are used as inputs to the sub-convolutional network to generate a physical circuit diagram, wherein the physical circuit diagram consists only of circuit symbols.

[0011] By adopting the above technical solution, 2D images with wire connections are processed by neural networks and converted into physical circuit diagrams consisting only of circuit symbols. This makes it easier to understand the connection status of students' circuits, and instructors can judge whether students' circuit connections are correct by judging the physical circuit diagram, which greatly improves the convenience and speed of detection.

[0012] Preferably, the step of acquiring an image of the experimental circuit diagram to be identified and processing it includes: acquiring and storing the actual image of the experimental circuit diagram to be identified;

[0013] The actual image is resized and padded to a set size to generate a standard image;

[0014] The image data of the standard image is acquired, and the standard image data is subjected to image enhancement and noise reduction processing.

[0015] By adopting the above technical solution, it is easier to further process the acquired images, reducing the possibility of the acquired images being too blurry or having too much interference affecting the subsequent generation of physical circuits.

[0016] Preferably, the step of segmenting the experimental circuit diagram based on a neural network to obtain component thermal maps and wire segmentation maps includes:

[0017] The experimental circuit diagram was used as the input to the neural network.

[0018] The experimental circuit diagram is segmented based on the generator encoder network and decoder network to obtain component heat maps and wire segmentation diagrams; the network corresponding to the component heat map and the network corresponding to the wire segmentation diagram are obtained;

[0019] By adopting the above technical solution, the experimental circuit is transformed into a component thermal diagram and a wire connection diagram, which facilitates the segmentation and differentiation of experimental images and improves the accuracy of image segmentation.

[0020] Preferably, the step of calculating the segmentation loss of the component heatmap and the conductor segmentation diagram includes: annotating the component heatmap and the conductor segmentation diagram, wherein the component heatmap is annotated using a Gaussian map and the conductor segmentation diagram is annotated using a binary map;

[0021] Obtain the mask annotations corresponding to the component thermal diagram and wire segmentation diagram.

[0022] The segmentation loss of the element's thermal map and the wire segmentation map is calculated based on relevant algorithms.

[0023] Preferably, the relevant algorithm includes: L seg (x)=λ1E x [(f e (x)-y ele ) 2 ]+λ2E x [(f l (x)-y line ) 2 ]; where, L seg To segment the loss term, fe and f l These are the network portions for obtaining the element's heatmap and the wire segmentation diagram, respectively. ele and y line These are the mask labels corresponding to the component thermal diagram and the wire segmentation diagram, respectively.

[0024] Preferably, the step of using the component heatmap and wire segmentation map as input to the sub-convolutional network and generating a physical circuit diagram includes:

[0025] The component heatmap and wire segmentation diagram are processed based on generative adversarial network, and a physical circuit diagram is generated based on discriminator.

[0026] Obtain the actual annotated circuit diagram after annotation;

[0027] Optimization training is performed based on the discriminator to distinguish between the generated physical circuit diagram and the actual labeled circuit diagram;

[0028] The discriminator subnetwork determines whether the input experimental image and the generated physical circuit diagram are consistent, and the contrastive loss is used for training.

[0029] Preferably, in the process of processing the component heatmap and wire segmentation map based on the generative adversarial network, the results of the component heatmap and wire segmentation map are subjected to stop-gradient processing when training the generative adversarial network, that is, the results of the component heatmap and wire segmentation map are optimized only by the segmentation loss.

[0030] Preferably, the optimization objective of the discriminator is to distinguish between the generated physical circuit diagram and the experimental circuit diagram when training the discriminator, and to deceive the discriminator when training the generator.

[0031] After the generative adversarial network is trained, the discriminator part is discarded, and only the generator part and the circuit diagram generation sub-network are retained.

[0032] Preferably, the optimization objective includes the sum of adversarial loss, segmentation loss, and contrast loss.

[0033] Secondly, the computer-readable storage medium provided in this application adopts the following technical solution:

[0034] A computer-readable storage medium, characterized in that the storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the neural network-based physical experiment circuit diagram generation method as described in any of the preceding claims.

[0035] In summary, this application includes at least one of the following beneficial technical effects:

[0036] 1. The 2D image with wire connections is processed by a neural network and converted into a physical circuit diagram consisting only of circuit symbols. This makes it easier to understand the connection status of students' circuits, and the instructors can judge whether the students' circuit connections are correct by judging the physical circuit diagram, which greatly improves the convenience and speed of the test.

[0037] 2. The network uses the segmentation results as input to train the generator and discriminator, ensuring that the generator only uses the segmentation results to generate circuit diagrams. Therefore, the role of the segmentation network is to filter out background and other irrelevant information, extracting circuit components and connections. The generative adversarial network (GAN) learns through continuous adversarial interaction until it reaches Nash equilibrium. Under this condition, the discriminator cannot distinguish between genuine and fake generated samples; that is, the probability distribution of the generated images is almost identical to that of the labeled images. The contrastive loss penalizes the model in cases where the input and generated circuit diagrams are inconsistent, ensuring a one-to-one correspondence between the generated circuit diagram and the input image. Attached Figure Description

[0038] Figure 1 This embodiment of the present application mainly illustrates the flowchart of the method for generating physical experiment circuit diagrams based on neural networks;

[0039] Figure 2 This is a flowchart illustrating sub-step S1;

[0040] Figure 3 This is a flowchart illustrating sub-step S2;

[0041] Figure 4 This is a flowchart illustrating sub-step S3;

[0042] Figure 5 This is a flowchart illustrating sub-step S4. Detailed Implementation

[0043] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the application and are not intended to limit the scope of the application.

[0044] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the inventive concept. As part of this specification, some of the accompanying drawings of this disclosure are block diagrams illustrating structures and devices to avoid complicating the disclosed principles. For clarity, not all features of the actual embodiment need to be described. Furthermore, the language used in this disclosure has been primarily chosen for readability and instructional purposes and may not have been chosen to define or limit the subject matter of the invention, thus requiring the necessary claims to determine such inventive subject matter. References to “an embodiment” or “an embodiment” in this disclosure mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment, and multiple references to “an embodiment” or “an embodiment” should not be construed as necessarily referring to the same embodiment.

[0045] Reference Figure 1 A method for generating physical experiment circuit diagrams based on neural networks includes the following steps:

[0046] S1. Acquire an image of the experimental circuit diagram to be identified and perform image processing on it. The experimental circuit diagram to be identified includes a component diagram and a wire connection diagram.

[0047] Specifically, step S1 includes the following steps:

[0048] Reference Figure 2 S101. Acquire and store the actual image of the experimental circuit diagram to be identified. A camera or other imaging tool can be used to photograph the experimental circuit diagram on the lab bench, and the photographed circuit diagram can be stored.

[0049] S102. The actual image is resized and padded to a set size to generate a standard image. The set size includes 640x640, which facilitates unified image management.

[0050] S103. Acquire the image data of the standard image, and perform image enhancement and denoising processing on the standard image data. Image enhancement processing can increase the brightness and resolution of the image, while denoising processing can remove interference present in the image using image denoising techniques.

[0051] S2. Segment the experimental circuit diagram based on a neural network to obtain component thermal maps and wire segmentation diagrams. Step S2 specifically includes the following steps:

[0052] Reference Figure 3 S201. Use the experimental circuit diagram as the input to the neural network;

[0053] S202. Based on the generator encoder network and decoder network, the experimental circuit diagram is segmented to obtain the component heat map and wire segmentation diagram;

[0054] S203. Obtain the network corresponding to the component's thermal diagram and the network corresponding to the wire segmentation diagram.

[0055] S3. The step of calculating the segmentation loss of the element's thermal diagram and the conductor segmentation diagram. Specifically, it includes the following sub-steps:

[0056] Reference Figure 4 S301. Label the component thermal diagram and the conductor segmentation diagram. The component thermal diagram is labeled using a Gaussian map, and the conductor segmentation diagram is labeled using a binary map.

[0057] S302. Obtain the mask annotations corresponding to the component thermal diagram and wire segmentation diagram.

[0058] S303. Calculate the segmentation loss of the component thermal map and the conductor segmentation map based on relevant algorithms. In this embodiment, the relevant algorithms include:

[0059] L seg (x)=λ1E x [(f e (x)-y ele ) 2 ]+λ2E x [(f l (x)-y line ) 2 ]; where, L seg To segment the loss term, f e and f l These are the network portions for obtaining the element's heatmap and the wire segmentation diagram, respectively. ele and y line These are the mask labels corresponding to the component thermal diagram and the wire segmentation diagram, respectively.

[0060] The results of component heatmaps and wire segmentation maps are subjected to stop-gradient processing during the training of the generative adversarial network (GAN), meaning that the results of component heatmaps and wire segmentation maps are optimized only by the segmentation loss. The discriminator's optimization objective during discriminator training is to distinguish between the generated physical circuit diagram and the experimental circuit diagram; during generator training, its optimization objective is to deceive the discriminator. After the GAN is trained, the discriminator part is discarded, and only the generator part and the circuit diagram generation sub-network S4 are retained. The component heatmaps and wire segmentation maps are used as input to the sub-convolutional network to generate the physical circuit diagram. Specifically, this includes the following steps:

[0061] Reference Figure 5S401. The heat map and wire segmentation diagram of the components are processed based on the generative adversarial network, and a physical circuit diagram is generated based on the discriminator.

[0062] S402. Obtain the actual annotated circuit diagram after annotation;

[0063] S403. Optimize training based on the discriminator's judgment of the generated physical circuit diagram and the actual labeled circuit diagram;

[0064] S404. The discriminator sub-network determines whether the input experimental image and the generated physical circuit diagram are consistent, and contrastive loss is used for training. The optimization objective consists of three parts: adversarial loss, segmentation loss, and contrastive loss.

[0065] L(x)=L GAN (x)+L seg (x)+L contrastive (x)

[0066]

[0067]

[0068]

[0069] Where L GAN To counteract the loss term, G and D are the generator and discriminator, respectively, z is the generated circuit diagram, y is the labeled circuit diagram, and λ is a weighted adjustable hyperparameter;

[0070] L seg To segment the loss term, f e and f l These are the network portions for obtaining the element's heatmap and the wire segmentation diagram, respectively, yele and y line These are the mask labels for the component heatmap and the conductor segmentation diagram, respectively. The heatmap is labeled using a Gaussian map, and the conductor segmentation diagram is labeled using a binary map.

[0071] L contrastive To compare the loss term, z is the generated physical circuit diagram and y is the actual labeled circuit diagram. This term ensures that the generated circuit diagram corresponds to the input x. Otherwise, the network will learn trivial solutions, that is, regardless of the input, it will generate the same circuit diagram, which is generally called "mode collapse".

[0072] By optimizing the aforementioned loss, the network uses the segmentation results as input to train the generator and discriminator. This ensures that the generator only uses the segmentation results to generate circuit diagrams. Therefore, the role of the segmentation network is to filter out background and other irrelevant information, extracting circuit components and connections. The generative adversarial network (GAN) learns through continuous adversarial interaction until it reaches Nash equilibrium. Under this condition, the discriminator cannot distinguish between genuine and fake generated samples; that is, the probability distribution of the generated images is almost identical to that of the labeled images. The contrastive loss penalizes the model in cases where the input and generated circuit diagrams are inconsistent, ensuring a one-to-one correspondence between the generated circuit diagram and the input image.

[0073] This application also discloses a readable storage medium storing a method for generating physical experimental circuit diagrams based on neural networks, which can be loaded by a processor and executed as described above. Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented using software plus necessary general-purpose hardware platforms, and of course, hardware can also be used, but in many cases the former is a better implementation. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0074] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for generating physical experiment circuit diagrams based on neural networks, characterized in that, Includes the following steps: The image of the experimental circuit diagram to be identified is acquired and processed. The experimental circuit diagram to be identified includes component diagrams and wire connection diagrams. The experimental circuit diagram is segmented based on a neural network to obtain component thermal diagrams and wire segmentation diagrams. Calculate the segmentation loss of the element's thermal diagram and the wire segmentation diagram; The component heatmap and wire segmentation diagram are processed based on a generative adversarial network, and a physical circuit diagram is generated based on a discriminator; wherein the physical circuit diagram consists only of circuit symbols. Obtain the actual annotated circuit diagram; Optimization training is performed based on the discriminator to distinguish between the generated physical circuit diagram and the actual labeled circuit diagram; The discriminator subnetwork determines whether the input experimental image and the generated physical circuit diagram are consistent, and the contrastive loss is used for training during training. The contrast loss is calculated using the following formula: L contrastive For comparison of losses, z represents the generated physical circuit diagram, y represents the actual labeled circuit diagram, and λ represents the weighted adjustable hyperparameter.

2. The method for generating physical experiment circuit diagrams based on neural networks according to claim 1, characterized in that: The steps of acquiring an image of the experimental circuit diagram to be identified and processing it include: Acquire and store actual images of the experimental circuit diagram to be identified; The actual image is resized and padded to a set size to generate a standard image; The image data of the standard image is acquired, and the standard image data is subjected to image enhancement and noise reduction processing.

3. The method for generating physical experiment circuit diagrams based on neural networks according to claim 1, characterized in that: The step of segmenting the experimental circuit diagram based on a neural network to obtain component thermal maps and wire segmentation maps includes: The experimental circuit diagram was used as the input to the neural network. The experimental circuit diagram is segmented based on the generator encoder network and decoder network to obtain component heat map and wire segmentation diagram; Obtain the network corresponding to the component's heatmap and the network corresponding to the wire segmentation diagram.

4. The method for generating physical experiment circuit diagrams based on neural networks according to claim 1, characterized in that: The step of calculating the segmentation loss of the element thermal diagram and the conductor segmentation diagram includes: The component thermal maps and conductor segmentation diagrams are labeled. The component thermal maps are labeled using Gaussian map plots, and the conductor segmentation diagrams are labeled using binary plots. Obtain the mask annotations corresponding to the component thermal diagram and wire segmentation diagram; The segmentation loss of the element's thermal map and the wire segmentation map is calculated based on relevant algorithms.

5. The method for generating physical experiment circuit diagrams based on neural networks according to claim 4, characterized in that: The relevant algorithms include: L seg (x)=λ1E x [f e (x)-y ele ) 2 ]+λ2E x [(f1(x)-y line ) 2 ]; where, L seg To segment the loss term, f e f1 and f1 are the network parts for obtaining the element's heat map and wire segmentation map, respectively, y ele and y line These are the mask labels corresponding to the component thermal diagram and the wire segmentation diagram, respectively.

6. The method for generating physical experiment circuit diagrams based on neural networks according to claim 1, characterized in that: In the process of processing the component heatmap and wire segmentation map based on the generative adversarial network, the results of the component heatmap and wire segmentation map are subjected to stop-gradient processing when training the generative adversarial network, that is, the results of the component heatmap and wire segmentation map are optimized only by the segmentation loss.

7. The method for generating physical experiment circuit diagrams based on neural networks according to claim 1, characterized in that: The optimization objective of the discriminator is to distinguish between the generated physical circuit diagram and the experimental circuit diagram when training the discriminator, and to deceive the discriminator when training the generator. After the generative adversarial network is trained, the discriminator part is discarded, and only the generator part and the circuit diagram generation sub-network are retained.

8. The method for generating physical experiment circuit diagrams based on neural networks according to claim 7, characterized in that: The optimization objective includes the sum of adversarial loss, segmentation loss, and contrastive loss; specifically, L(x)=L GAN (x)+L seg (x)+L contrastive (x) Where L is the optimization objective, L GAN To counteract the loss term, G is the generator, D is the discriminator, z is the generated circuit diagram, y is the actual labeled circuit diagram, and λ is a weighted adjustable hyperparameter; L seg To segment the loss term, f e To obtain the network for the element's thermal map, f l To obtain the network of wire segmentation diagram, y ele and y line These are the mask labels corresponding to the component thermal diagram and the wire segmentation diagram; L contrastiv For comparison of loss terms.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the neural network-based physical experiment circuit diagram generation method as described in any one of claims 1 to 8.

Images