Intelligent identification method and system for bread board circuit

Abstract

The invention discloses an intelligent identification method and system for a bread board circuit, and the method comprises the following steps: obtaining a bread board image, extracting a bread board region, and obtaining a bread board region image; obtaining a bread board contour, a jack contour and an electronic component contour; calculating to obtain two-dimensional coordinates of each jack in the bread board area image; obtaining a contour segmentation image of each electronic component; inputting the contour segmentation image of each electronic component into a pre-trained component type classification model to obtain the type of each electronic component; and inputting the jacks in the preset range of the contour segmentation image of each electronic component into a pre-trained jack state classification model to obtain a jack state: establishing a connection relationship between the tail end area of the pin part of each electronic component and the coordinates of the nearest actually inserted jack, and obtaining a circuit connection relationship of each electronic component. The circuit built on the bread board can be automatically identified, and a tester can conveniently and rapidly judge whether the built circuit is correct or not.

Description

Technical Field

[0001] The present invention relates to the technical field of circuit design, and more particularly to an intelligent identification method and system for a breadboard circuit. Background Art

[0002] At present, circuit construction and verification are indispensable and important links in electronic engineering education and practice. Traditional circuit verification methods rely on manual inspection or the use of professional testing equipment, which is not only time-consuming and labor-intensive, but also prone to errors. With the advancement of science and technology, especially the rapid development of image processing and deep learning technology, new solutions have been provided for the automatic identification of circuit components.

[0003] Therefore, how to provide an intelligent breadboard circuit recognition method and system, which can automatically identify the circuit built on the breadboard and facilitate the test personnel to quickly determine whether the built circuit is correct is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] In view of this, an object of the present invention is to provide an intelligent identification method and system for breadboard circuits.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] On the one hand, the present invention provides an intelligent identification method for a breadboard circuit, comprising the following steps:

[0007] S1: Acquire a breadboard image and extract the breadboard area to obtain a breadboard area image;

[0008] S2: Binarizing the breadboard area image and performing contour extraction to obtain the breadboard contour, the jack contour and the electronic component contour;

[0009] S3: using the center of gravity of the jack outline, calculating and obtaining the two-dimensional coordinates of each jack in the breadboard area image;

[0010] S4: using the breadboard outline, the jack outline and the electronic component outline, obtaining the segmented image of each electronic component outline; wherein the electronic component outline segmented image includes the electronic component pin part and the electronic component body part;

[0011] S5: Inputting the contour segmentation images of each electronic component into a pre-trained component type classification model to obtain the type of each electronic component;

[0012] Input the sockets within the preset range of each electronic component contour segmentation image into a pre-trained socket state classification model to obtain the socket state: wherein the socket state includes being truly inserted and not inserted;

[0013] S6: Establish a connection relationship between the end area of ​​the pin part of each electronic component and the coordinates of the nearest actual inserted socket to obtain the circuit connection relationship of each electronic component.

[0014] Preferably, breadboard outlines, jack outlines and electronic component outlines are distinguished based on the number of outlines and the outline length.

[0015] Preferably, S3 further comprises the following steps:

[0016] Select all the jack contours in the binary breadboard area image, and calculate the centroids of all the jack contours to obtain the first point set;

[0017] Select all the jack contours in the binary template image, and calculate the centroids of all the jack contours to obtain a second point set;

[0018] Based on the two-dimensional coordinates of the four corners of the binary breadboard area image, the length of the template image and the width of the template image, a perspective transformation matrix is ​​obtained; wherein, a polygon fitting convex hull is performed on the breadboard contour to obtain the two-dimensional coordinates of the four corners of the binary breadboard area image;

[0019] The template image is transformed to the breadboard area using the perspective transformation matrix to obtain the initial two-dimensional coordinates of each socket in the breadboard area image;

[0020] The ICP algorithm is used to match the first point set and the second point set to obtain the offset and rotation of the entire point set;

[0021] The initial two-dimensional coordinates are optimized using the offset and the rotation amount to obtain the two-dimensional coordinates of each socket in the breadboard area image.

[0022] Preferably, S4 further comprises the following steps:

[0023] Segment the breadboard area image based on the Segment Anything model to obtain a preliminary segmentation result;

[0024] Based on the breadboard outline, the socket outline and the electronic component outline, each electronic component outline segmentation image is screened out from the preliminary segmentation result.

[0025] Preferably, S5 further comprises the following steps:

[0026] The contour segmentation images of each electronic component are input into the parameter recognition model to obtain the parameters of each electronic component.

[0027] On the other hand, the present invention also provides an intelligent recognition system for a breadboard circuit, comprising a breadboard area image acquisition unit, a contour extraction unit, a jack coordinate acquisition unit, an electronic component contour segmentation image acquisition unit, a component type classification model, a jack state classification model, and a circuit connection relationship acquisition unit;

[0028] The breadboard area image acquisition unit is used to acquire a breadboard image and extract a breadboard area;

[0029] The contour extraction unit is used to perform contour extraction after binarizing the breadboard area image to obtain the breadboard contour, the jack contour and the electronic component contour;

[0030] The socket coordinate acquisition unit is used to calculate and obtain the two-dimensional coordinates of each socket in the breadboard area image using the center of gravity of the socket outline;

[0031] The electronic component outline segmentation image acquisition unit is used to obtain the outline segmentation images of each electronic component by using the breadboard outline, the jack outline and the electronic component outline; wherein the electronic component outline segmentation image includes the electronic component pin part and the electronic component body part;

[0032] The component type classification model is used to identify the type of each electronic component in each electronic component contour segmentation image;

[0033] The socket state classification model is used to identify the socket state of the socket within the preset range of each electronic component contour segmentation image; wherein the socket state includes being truly inserted and not inserted;

[0034] The circuit connection relationship acquisition unit is used to establish a connection relationship between the end area of ​​the pin portion of each electronic component and the coordinates of the nearest actual inserted socket, so as to obtain the circuit connection relationship of each electronic component.

[0035] Preferably, breadboard outlines, jack outlines and electronic component outlines are distinguished based on the number of outlines and the outline length.

[0036] Preferably, the jack coordinate acquisition unit is used to select all jack contours in the binary breadboard area image, and calculate the centroids of all jack contours to obtain a first point set;

[0037] Select all the jack contours in the binary template image, and calculate the centroids of all the jack contours to obtain a second point set;

[0038] Based on the two-dimensional coordinates of the four corners of the binary breadboard area image, the length of the template image and the width of the template image, a perspective transformation matrix is ​​obtained; wherein, a polygon fitting convex hull is performed on the breadboard contour to obtain the two-dimensional coordinates of the four corners of the binary breadboard area image;

[0039] The template image is transformed to the breadboard area using the perspective transformation matrix to obtain the initial two-dimensional coordinates of each socket in the breadboard area image;

[0040] The ICP algorithm is used to match the first point set and the second point set to obtain the offset and rotation of the entire point set;

[0041] The initial two-dimensional coordinates are optimized using the offset and the rotation amount to obtain the two-dimensional coordinates of each socket in the breadboard area image.

[0042] Preferably, the electronic component contour segmentation image acquisition unit includes a Segment Anything model and a preliminary segmentation result screening unit;

[0043] The Segment Anything model is used to segment the breadboard area image to obtain a preliminary segmentation result;

[0044] The preliminary segmentation result screening unit is used to screen out individual electronic component outline segmentation images from the preliminary segmentation results based on the breadboard outline, the socket outline and the electronic component outline.

[0045] Preferably, the above-mentioned intelligent identification system also includes a parameter identification model;

[0046] The parameter recognition model is used to recognize the parameters of each electronic component in the electronic component contour segmentation image.

[0047] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses an intelligent breadboard circuit recognition method and system, which can automatically identify the circuit built on the breadboard, making it convenient for test personnel to quickly determine whether the built circuit is correct. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0049] Figure 1 A flow chart of an intelligent identification method for a breadboard circuit provided by the present invention;

[0050] Figure 2 The training data set of the jack status classification model provided by the embodiment of the present invention includes images;

[0051] Figure 3 A breadboard image provided by an embodiment of the present invention;

[0052] Figure 4 A breadboard area image provided by an embodiment of the present invention;

[0053] Figure 5 An image of a convex hull for polygon fitting provided by an embodiment of the present invention;

[0054] Figure 6 The template image provided in the embodiment of the present invention is an image after perspective transformation;

[0055] Figure 7 Comparison images before and after ICP optimization provided by the embodiment of the present invention;

[0056] Figure 8 The component type classification model provided in the embodiment of the present invention can classify various electronic components. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, an embodiment of the present invention discloses an intelligent identification method for a breadboard circuit, comprising the following steps:

[0059] S1: Get the breadboard image (such as Figure 3 ) and extract the breadboard area to obtain the breadboard area image (as shown Figure 4 shown);

[0060] In one embodiment, breadboard outlines, socket outlines, and electronic component outlines are differentiated based on the number of outlines and the length of the outlines.

[0061] S2: Binarizing the breadboard area image and performing contour extraction to obtain the breadboard contour, the jack contour and the electronic component contour;

[0062] It can be understood that: breadboard outlines, jack outlines and electronic component outlines are distinguished based on the number of outlines and outline lengths; the longest outline is the breadboard outline; the outlines with the largest number and a certain range of outline length difference are the jack outlines; the rest are electronic component outlines.

[0063] S3: using the center of gravity of the jack outline, calculating and obtaining the two-dimensional coordinates of each jack in the breadboard area image;

[0064] In one embodiment, if Figure 5-Figure 7 As shown, S3 further includes the following steps:

[0065] Select all the jack contours in the binary breadboard area image, and calculate the centroids of all the jack contours to obtain the first point set;

[0066] Select all the jack contours in the binary template image, and calculate the centroids of all the jack contours to obtain a second point set;

[0067] Based on the two-dimensional coordinates of the four corners of the binary breadboard area image, the length of the template image and the width of the template image, a perspective transformation matrix is ​​obtained; wherein, a polygon fitting convex hull is performed on the breadboard contour to obtain the two-dimensional coordinates of the four corners of the binary breadboard area image;

[0068] The template image is transformed to the breadboard area using the perspective transformation matrix to obtain the initial two-dimensional coordinates of each socket in the breadboard area image;

[0069] The ICP algorithm is used to match the first point set and the second point set to obtain the offset and rotation of the entire point set;

[0070] The initial two-dimensional coordinates are optimized using the offset and the rotation amount to obtain the two-dimensional coordinates of each socket in the breadboard area image.

[0071] S4: using the breadboard outline, the jack outline and the electronic component outline, obtaining the segmented image of each electronic component outline; wherein the electronic component outline segmented image includes the electronic component pin part and the electronic component body part;

[0072] In one embodiment, S4 further comprises the following steps:

[0073] Segment the breadboard area image based on the Segment Anything model to obtain a preliminary segmentation result;

[0074] Based on the breadboard outline, the socket outline and the electronic component outline, each electronic component outline segmentation image is screened out from the preliminary segmentation result.

[0075] S5: Inputting the contour segmentation images of each electronic component into a pre-trained component type classification model to obtain the type of each electronic component;

[0076] It is understood that the types of electronic components include electrolytic capacitors, wires, resistors, transistors, diodes, ceramic / monolithic capacitors (such as Figure 8 shown) etc.

[0077] In one embodiment, S5 further comprises the following steps:

[0078] The contour segmentation images of each electronic component are input into the parameter recognition model to obtain the parameters of each electronic component.

[0079] Specifically: the parameter identification of each electronic component includes resistor identification, electrolytic capacitor identification, ceramic / monolithic capacitor identification, diode identification, etc.: the corresponding parameter identification models include resistor parameter identification model, electrolytic capacitor parameter identification model, ceramic / monolithic capacitor parameter identification model, diode parameter identification model, etc.;

[0080] 1) Resistance identification

[0081] The resistance parameter recognition model is obtained by training based on the labeled resistance data set;

[0082] Specifically: the color ring regions of the resistor image in the resistor dataset are segmented (each region contains a color ring), and the color rings of each region are labeled with colors;

[0083] The resistor parameter recognition model can be used to segment the resistor color wheel to obtain segmented images of different color wheel areas; color recognition can be performed on the segmented images of different color wheel areas to obtain the colors of different color wheel areas, and the corresponding resistance values ​​can be calculated based on the colors of different color wheel areas.

[0084] 2) Electrolytic capacitor identification

[0085] The electrolytic capacitor parameter recognition model can be used to identify the text on the main part of the electrolytic capacitor, such as 220uF. At the same time, the positive and negative poles of the two pins can be determined based on the position of the white area of ​​the electrolytic capacitor in the main part.

[0086] 3) Identification of ceramic / monolithic capacitors

[0087] The ceramic / monolithic capacitor parameter recognition model can recognize the text on the main part of the ceramic / monolithic capacitor.

[0088] 4) Diode identification

[0089] The diode parameter recognition model can segment the diode image, segment the black area, and determine the positive and negative poles of the diode.

[0090] Input the sockets within the preset range of each electronic component contour segmentation image into a pre-trained socket state classification model to obtain the socket state: wherein the socket state includes being truly inserted and not inserted;

[0091] The training data set of the jack status classification model of the present invention includes images of the jack being actually inserted (such as Figure 2 Figure a), images of components passing through the jack but not inserted (such as Figure 2 b in Figure 2), the uninserted and unobstructed jack image (e.g. Figure 2 (c in Figure ).

[0092] S6: Establish a connection relationship between the end area of ​​the pin part of each electronic component and the coordinates of the nearest actual inserted socket to obtain the circuit connection relationship of each electronic component.

[0093] The end regions of the pin portions of each electronic component of the present invention are obtained based on the following method:

[0094] Obtaining and saving pixel points of the pin part of the electronic component in the electronic component contour segmentation image, and fitting the pixel points to obtain a fitting curve;

[0095] By obtaining the pixel point corresponding to the starting point of the fitting curve, the end area of ​​the pin part of each electronic component can be obtained.

[0096] S7: Input the circuit connection relationship of each electronic component into SPice in the form of a netlist for circuit verification to determine whether the circuit connection on the breadboard is correct.

[0097] The embodiment of the present invention also provides an intelligent recognition system for a breadboard circuit, comprising a breadboard area image acquisition unit, a contour extraction unit, a jack coordinate acquisition unit, an electronic component contour segmentation image acquisition unit, a component type classification model, a jack state classification model, and a circuit connection relationship acquisition unit;

[0098] The breadboard area image acquisition unit is used to acquire a breadboard image and extract a breadboard area;

[0099] The contour extraction unit is used to perform contour extraction after binarizing the breadboard area image to obtain the breadboard contour, the jack contour and the electronic component contour;

[0100] The socket coordinate acquisition unit is used to calculate and obtain the two-dimensional coordinates of each socket in the breadboard area image using the center of gravity of the socket outline;

[0101] The electronic component outline segmentation image acquisition unit is used to obtain the outline segmentation images of each electronic component by using the breadboard outline, the jack outline and the electronic component outline; wherein the electronic component outline segmentation image includes the electronic component pin part and the electronic component body part;

[0102] The component type classification model is used to identify the type of each electronic component in each electronic component contour segmentation image;

[0103] The socket state classification model is used to identify the socket state of the socket within the preset range of each electronic component contour segmentation image; wherein the socket state includes being truly inserted and not inserted;

[0104] The circuit connection relationship acquisition unit is used to establish a connection relationship between the end area of ​​the pin portion of each electronic component and the coordinates of the nearest actual inserted socket, so as to obtain the circuit connection relationship of each electronic component.

[0105] In one embodiment, breadboard outlines, socket outlines, and electronic component outlines are differentiated based on the number of outlines and the length of the outlines.

[0106] In one embodiment, the jack coordinate acquisition unit is used to select all jack contours in the binary breadboard area image and calculate the centroids of all jack contours to obtain a first point set;

[0107] Select all the jack contours in the binary template image, and calculate the centroids of all the jack contours to obtain a second point set;

[0108] Based on the two-dimensional coordinates of the four corners of the binary breadboard area image, the length of the template image and the width of the template image, a perspective transformation matrix is ​​obtained; wherein, a polygon fitting convex hull is performed on the breadboard contour to obtain the two-dimensional coordinates of the four corners of the binary breadboard area image;

[0109] The template image is transformed to the breadboard area using the perspective transformation matrix to obtain the initial two-dimensional coordinates of each socket in the breadboard area image;

[0110] The ICP algorithm is used to match the first point set and the second point set to obtain the offset and rotation of the entire point set;

[0111] The initial two-dimensional coordinates are optimized using the offset and the rotation amount to obtain the two-dimensional coordinates of each socket in the breadboard area image.

[0112] In one embodiment, the electronic component contour segmentation image acquisition unit includes a Segment Anything model and a preliminary segmentation result screening unit;

[0113] The Segment Anything model is used to segment the breadboard area image to obtain a preliminary segmentation result;

[0114] The preliminary segmentation result screening unit is used to screen out individual electronic component outline segmentation images from the preliminary segmentation results based on the breadboard outline, the socket outline and the electronic component outline.

[0115] In one embodiment, the intelligent identification system further includes a parameter identification model;

[0116] The parameter recognition model is used to recognize the parameters of each electronic component in the electronic component contour segmentation image.

[0117] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0118] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent identification method for a breadboard circuit, characterized in that: The following steps are involved: S1: Acquire a breadboard image and extract the breadboard area to obtain a breadboard area image; S2: Binarizing the breadboard area image and performing contour extraction to obtain the breadboard contour, the jack contour and the electronic component contour; S3: using the center of gravity of the jack outline, calculating and obtaining the two-dimensional coordinates of each jack in the breadboard area image; S4: using the breadboard outline, the jack outline and the electronic component outline, obtaining the segmented image of each electronic component outline; wherein the electronic component outline segmented image includes the electronic component pin part and the electronic component body part; S5: Inputting the contour segmentation images of each electronic component into a pre-trained component type classification model to obtain the type of each electronic component; Input the sockets within the preset range of each electronic component contour segmentation image into a pre-trained socket state classification model to obtain the socket state: wherein the socket state includes being truly inserted and not inserted; S6: Establish a connection relationship between the end area of ​​the pin part of each electronic component and the coordinates of the nearest actual inserted socket to obtain the circuit connection relationship of each electronic component.

2. The intelligent identification method of a breadboard circuit according to claim 1, characterized in that: Differentiate between breadboard outlines, socket outlines, and electronic component outlines based on the number of outlines and outline length.

3. The intelligent identification method of a breadboard circuit according to claim 1, characterized in that: S3 further comprises the following steps: Select all the jack contours in the binary breadboard area image, and calculate the centroids of all the jack contours to obtain the first point set; Select all the jack contours in the binary template image, and calculate the centroids of all the jack contours to obtain a second point set; Obtain a perspective transformation matrix based on the two-dimensional coordinates of the four corners of the binary breadboard area image, the length of the template image, and the width of the template image; Among them, the convex hull of the breadboard outline is fitted with a polygon to obtain the two-dimensional coordinates of the four corners of the binary breadboard area image; The template image is transformed to the breadboard area using the perspective transformation matrix to obtain the initial two-dimensional coordinates of each socket in the breadboard area image; The ICP algorithm is used to match the first point set and the second point set to obtain the offset and rotation of the entire point set; The initial two-dimensional coordinates are optimized using the offset and the rotation amount to obtain the two-dimensional coordinates of each socket in the breadboard area image.

4. The intelligent identification method of a breadboard circuit according to claim 1, characterized in that: S4 further comprises the following steps: Segment the breadboard area image based on the Segment Anything model to obtain a preliminary segmentation result; Based on the breadboard outline, the socket outline and the electronic component outline, each electronic component outline segmentation image is screened out from the preliminary segmentation result.

5. The intelligent identification method of a breadboard circuit according to claim 1, characterized in that: S5 further comprises the following steps: The contour segmentation images of each electronic component are input into the parameter recognition model to obtain the parameters of each electronic component.

6. An intelligent recognition system for breadboard circuits, characterized in that: It includes a breadboard area image acquisition unit, a contour extraction unit, a socket coordinate acquisition unit, an electronic component contour segmentation image acquisition unit, a component type classification model, a socket state classification model, and a circuit connection relationship acquisition unit; The breadboard area image acquisition unit is used to acquire a breadboard image and extract a breadboard area; The contour extraction unit is used to perform contour extraction after binarizing the breadboard area image to obtain the breadboard contour, the jack contour and the electronic component contour; The socket coordinate acquisition unit is used to calculate and obtain the two-dimensional coordinates of each socket in the breadboard area image using the center of gravity of the socket outline; The electronic component contour segmentation image acquisition unit is used to obtain the contour segmentation images of each electronic component by using the breadboard contour, the jack contour and the electronic component contour; wherein the electronic component contour segmentation image includes the electronic component pin part and the electronic component body part; The component type classification model is used to identify the type of each electronic component in each electronic component contour segmentation image; The socket state classification model is used to identify the socket state of the socket within the preset range of each electronic component contour segmentation image; wherein the socket state includes being truly inserted and not inserted; The circuit connection relationship acquisition unit is used to establish a connection relationship between the end area of ​​the pin part of each electronic component and the coordinates of the nearest actual inserted socket, so as to obtain the circuit connection relationship of each electronic component.

7. The intelligent recognition system for breadboard circuit according to claim 6, characterized in that: Differentiate between breadboard outlines, socket outlines, and electronic component outlines based on the number of outlines and outline length.

8. The intelligent recognition system for breadboard circuit according to claim 6, characterized in that: The jack coordinate acquisition unit is used to select all jack contours in the binary breadboard area image and calculate the centroids of all jack contours to obtain a first point set; Select all the jack contours in the binary template image, and calculate the centroids of all the jack contours to obtain a second point set; Based on the two-dimensional coordinates of the four corners of the binary breadboard area image, the length of the template image and the width of the template image, a perspective transformation matrix is ​​obtained; wherein, a polygon fitting convex hull is performed on the breadboard contour to obtain the two-dimensional coordinates of the four corners of the binary breadboard area image; The template image is transformed to the breadboard area using the perspective transformation matrix to obtain the initial two-dimensional coordinates of each socket in the breadboard area image; The ICP algorithm is used to match the first point set and the second point set to obtain the offset and rotation of the entire point set; The initial two-dimensional coordinates are optimized using the offset and the rotation amount to obtain the two-dimensional coordinates of each socket in the breadboard area image.

9. The intelligent recognition system for breadboard circuit according to claim 6, characterized in that: The electronic component contour segmentation image acquisition unit includes a Segment Anything model and a preliminary segmentation result screening unit; The Segment Anything model is used to segment the breadboard area image to obtain a preliminary segmentation result; The preliminary segmentation result screening unit is used to screen out individual electronic component outline segmentation images from the preliminary segmentation results based on the breadboard outline, the socket outline and the electronic component outline.

10. The intelligent recognition system for breadboard circuit according to claim 6, characterized in that: Also included are parameter identification models; The parameter recognition model is used to recognize the parameters of each electronic component in the electronic component contour segmentation image.

Images