A method, apparatus, computing device, and storage medium for wire identification.

Abstract

This invention discloses a method, apparatus, computing device, and storage medium for wire identification. The method includes: acquiring an image of a wire to be identified; preprocessing the wire image to obtain a target image; inputting the target image into a trained convolutional neural network model for feature extraction to obtain the key point coordinates and categories of the wire, the affinity matrix between key points, and the bidirectional connection vector field between key points; based on the key point coordinates and categories, using a matching algorithm to perform pairwise matching of key points to obtain a first matching result between key points; confirming the first matching result based on the affinity matrix between key points to obtain a second matching result between key points; and confirming the second matching result based on the bidirectional connection vector field between key points to obtain multiple sets of associated key points, with each set of key points identifying a wire. This scheme can improve the accuracy of wire identification in complex scenes.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and specifically to a wire recognition method, apparatus, computing device, and storage medium. Background Technology

[0002] In electrical experiments, it's necessary to determine the connections between components using wires. However, due to the length, crisscrossing nature, and complex wiring of these wires, distinguishing each individual wire can be challenging. Common solutions to this problem include using object detection or semantic segmentation methods to study the relationships between wires and thus differentiate them. However, object detection is prone to errors when wires are close together and fails to distinguish them when they intersect. While semantic segmentation can address the issue of close wires, it performs poorly in cases of intersecting or overlapping wires.

[0003] Therefore, there is a need to provide a wire identification method that can accurately distinguish complex wires in order to solve the problems existing in the above-mentioned prior art. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a wire identification method, apparatus, computing device, and storage medium that overcomes or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a method for identifying a conductor is provided. In this method, an image of the conductor to be identified is first acquired, and then preprocessed to obtain a target image. Subsequently, the target image is input into a trained convolutional neural network model for feature extraction, yielding the coordinates and categories of key points, the affinity matrix between key points, and the bidirectional connection vector field between key points. Next, based on the key point coordinates and categories, a matching algorithm is used to perform pairwise matching of key points, obtaining a first matching result. Then, the first matching result is confirmed based on the affinity matrix between key points, yielding a second matching result. Finally, the second matching result is confirmed based on the bidirectional connection vector field between key points, resulting in multiple sets of associated key points. Since two points can determine a line segment, each set of key points can identify a conductor.

[0006] By detecting the position and category of the wire endpoints and predicting and matching the relationships between them, complex wires can be identified without being limited by factors such as wire length, color, and position, thus improving the accuracy and efficiency of wire identification.

[0007] Optionally, in the method according to the present invention, the key points can be divided into a first set of points and a second set of points based on the category of the key points; then, the points in the first set of points and the points in the second set of points are matched pairwise based on the Hungarian matching algorithm to obtain the first matching result between the key points.

[0008] This step can initially perform pairwise matching of key points of different categories. The first matching result is the connection relationship between key points of different categories obtained by the matching algorithm.

[0009] Optionally, in the method according to the present invention, in order to further confirm the matching results obtained by the matching algorithm, the first matching results can be filtered based on the similarity score between every two key points in the affinity matrix, and each group of key points with a similarity score greater than a predetermined threshold can be used as the second matching result; otherwise, key points with a similarity score less than the predetermined threshold can be re-matched.

[0010] Optionally, in the method according to the present invention, to further confirm the matching results, multiple relation vectors between two key points can be determined through a bidirectional connection vector field; then, the average value of the projections of all relation vectors on the connection line between the two key points is determined, and the average value is used as the relation vector field score; the relation vector field scores of the key points in the second matching result are sorted; key points with relation vector field scores greater than a preset threshold are identified as associated key points, and key points with relation vector field scores less than the preset threshold are re-matched. This can further improve the accuracy of key point matching.

[0011] Optionally, in the method according to the present invention, the convolutional neural network model includes a backbone network, a keypoint detection network head connected to the backbone network, an affinity prediction network head, and a vector field prediction network head. The keypoint detection network, the affinity prediction network, and the vector field prediction network are connected in parallel. The backbone network is adapted to extract features from the target image to obtain a feature map. The keypoint detection network is adapted to determine the coordinates and categories of keypoints in the feature map. The affinity prediction network is adapted to predict the affinity matrix between keypoints in the feature map. The vector field prediction network is adapted to predict the bidirectional relationship vector field between keypoints in the feature map.

[0012] The structure of a convolutional neural network can be adjusted according to the complexity of the image to be recognized. The backbone can use mainstream residual neural networks, deep convolutional neural networks, lightweight neural networks, etc.

[0013] Optionally, in the method according to the present invention, in order to train the convolutional neural network, multiple training images can be acquired and preprocessed; then, the endpoint coordinates, endpoint categories, and relationships between endpoints of the wires in each preprocessed training image are labeled; the training images and the labeled endpoint coordinates, endpoint categories, and relationships between endpoints are input into a pre-built convolutional neural network model for training, thereby obtaining the trained convolutional neural network model.

[0014] Optionally, in the method according to the present invention, the preprocessing of the image may include any one or more of grayscale transformation, contrast enhancement, rotation, cropping, and normalization.

[0015] According to another aspect of the present invention, a wire identification device is provided. The device includes an acquisition module, a preprocessing module, a feature extraction module, a matching module, a first confirmation module, and a second confirmation module. The acquisition module acquires an image of the wire to be identified. The preprocessing module preprocesses the wire image to obtain a target image. The feature extraction module inputs the target image into a trained convolutional neural network model for feature extraction. The keypoints are detected by a keypoint detection network (head) to obtain the coordinates and categories of the wires; the affinity prediction network (head) predicts the affinity matrix between the keypoints; and the vector field prediction network (head) predicts the bidirectional connection vector field between the keypoints. The matching module performs pairwise matching of the keypoints based on their coordinates and categories using a matching algorithm to obtain a first matching result. The first confirmation module confirms the first matching result based on the affinity matrix between the keypoints to obtain a second matching result. The second confirmation module confirms the second matching result based on the bidirectional connection vector field between the keypoints, obtaining multiple sets of associated keypoints, each set of keypoints identifying a wire.

[0016] According to another aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the methods described above.

[0017] According to another aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the method described above.

[0018] According to the present invention, key point localization and relationship clustering are used instead of the method of localization by detection boxes in semantic segmentation and target detection schemes. The wire identification is based on the relationship between the two ends of the wire, which does not depend on the connection relationship between the wires, and is not limited by the wire color, length, intersection, overlap, shape, size, etc. between the wires, and can accurately identify complex wires.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0021] Figure 1 A schematic flowchart of a wire identification method according to an embodiment of the present invention is shown;

[0022] Figure 2 A structural diagram of a computing device 200 according to an embodiment of the present invention is shown;

[0023] Figure 3 A flowchart of a wire identification method 300 according to an embodiment of the present invention is shown;

[0024] Figure 4 A schematic diagram of a first point set and a second point set according to an embodiment of the present invention is shown;

[0025] Figure 5 A schematic diagram of the structure of a wire identification device 500 according to an embodiment of the present invention is shown. Detailed Implementation

[0026] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0027] Due to factors such as varying wire lengths and colors, wire intersections and overlaps in electrical experiments, and interference from factors like lighting and occlusion during image capture, wire recognition methods based on object detection and semantic segmentation generally perform poorly. To address these issues, this paper proposes a wire recognition method based on the relationship between the two endpoints of a wire. This method is independent of the wire's length, color, and position, thus improving the accuracy of wire recognition.

[0028] This scheme first uses a convolutional neural network model for iterative training to obtain the positions, categories, and relationships between the wire endpoints. Then, based on the endpoint categories, the endpoints are divided into two independent sets: one set for current inflow points and the other for current outflow points. Matching results are obtained by pairwise matching of points in each set. Finally, affinity calculations and vector field calculations are used to further determine the matching results of the wire endpoints until all endpoints achieve optimal matching. The wires connecting the endpoints are then determined based on these optimal matching results.

[0029] Figure 1 A flowchart of a wire identification method according to an embodiment of the present invention is shown. Figure 1As shown, the first step is to input the image to be identified. The image of the conductor to be identified undergoes preprocessing such as grayscale transformation, image enhancement, and normalization. Then, it is input into a trained convolutional neural network model. Through three head network branches connected to the backbone network, keypoint location and category feature maps, affinity matrix feature maps between keypoints, and bidirectional connection vector field feature maps between keypoints are obtained. The keypoint coordinates and category feature maps are labeled with the location and category of the keypoints (different symbols are used for different current flow directions; for example, current inflow points are labeled with ×, current inflow points with ◎, or other symbols for category differentiation; this scheme does not limit this). Keypoints of different categories can be divided into two sets: the first set and the second set. These two sets satisfy the bipartite graph condition (two independent subsets, and each edge in the graph has two vertices belonging to both independent subsets). Then, a matching algorithm is used to perform pairwise matching of each keypoint in the two sets to obtain the matching results between each keypoint. Then, based on the affinity matrix between keypoints output by the model, the matching results are further confirmed. Specifically, the first matching result is filtered based on the similarity score between every two keypoints in the affinity matrix. Keypoints with similarity scores greater than a predetermined threshold are used as the second matching result; otherwise, keypoints with similarity scores less than the predetermined threshold are re-matched. The association between keypoints is confirmed based on the bidirectional connection vector field output by the model. Keypoints that do not simultaneously satisfy both the positive and negative conditions of the vector field are returned for re-matching. Two keypoints that simultaneously satisfy both the positive and negative conditions are confirmed as associated keypoints, ultimately resulting in multiple associated sets of keypoints, with a connecting line between each set. After multiple iterations, each set of points has a matching score. Scores less than 0.5 can be matched multiple times, ultimately resulting in multiple definite sets of points, with each keypoint having only one corresponding connection relationship.

[0030] This wire identification method can be executed in a computing device. Figure 2 A structural diagram of a computing device 200 according to an embodiment of the present invention is shown. Figure 2 As shown, in the basic configuration 202, the computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 can be used for communication between the processors 204 and the system memory 206.

[0031] Depending on the desired configuration, processor 204 can be any type of processor, including but not limited to: microprocessor (μP), microcontroller (μC), digital information processor (DSP), or any combination thereof. Processor 204 may include one or more levels of cache such as L1 cache 210 and L2 cache 212, processor core 214, and registers 216. Example processor core 214 may include an arithmetic logic unit (ALU), floating-point unit (FPU), digital signal processing core (DSP core), or any combination thereof. Example memory controller 218 may be used with processor 204, or in some implementations, memory controller 218 may be an internal part of processor 204.

[0032] Depending on the desired configuration, system memory 206 can be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. Physical memory in a computing device typically refers to volatile RAM, and data on a disk needs to be loaded into physical memory before it can be read by processor 204. System memory 206 may include operating system 220, one or more applications 222, and program data 224. Application 222 is actually a set of program instructions that instruct processor 204 to perform corresponding operations. In some embodiments, application 222 may be arranged to execute instructions on the operating system by one or more processors 204 using program data 224. Operating system 220 may be, for example, Linux, Windows, etc., and includes program instructions for handling basic system services and performing hardware-dependent tasks. Application 222 includes program instructions for implementing various user-desired functions, and application 222 may be, for example, a browser, instant messaging software, software development tools (such as integrated development environments (IDEs), compilers, etc.), but is not limited to these. When application 222 is installed on computing device 200, a driver module can be added to operating system 220.

[0033] When the computing device 200 starts up, the processor 204 reads the program instructions of the operating system 220 from the memory 206 and executes them. The application 222 runs on the operating system 220 and uses the interfaces provided by the operating system 220 and the underlying hardware to implement various functions expected by the user. When the user starts the application 222, the application 222 is loaded into the memory 206, and the processor 204 reads and executes the program instructions of the application 222 from the memory 206.

[0034] The computing device 200 also includes a storage device 232, which includes a removable storage device 236 and a non-removable storage device 238, both of which are connected to the storage interface bus 234.

[0035] The computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via a bus / interface controller 230. Example output devices 242 include a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices such as displays or speakers via one or more A / V ports 252. Example peripheral interfaces 244 may include a serial interface controller 254 and a parallel interface controller 256, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 258. Example communication devices 246 may include a network controller 260, which may be arranged to facilitate communication with one or more other computing devices 262 via a network communication link through one or more communication ports 264.

[0036] A network communication link can be an example of a communication medium. A communication medium can typically be embodied in a modulated data signal, such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A “modulated data signal” can be a signal whose data set, or whose modifications, can be encoded with information within the signal. As a non-limiting example, a communication medium can include wired media such as wired networks or leased lines, and various wireless media including sound, radio frequency (RF), microwave, infrared (IR), or other wireless media. The term “computer-readable medium” as used herein can include both storage media and communication media.

[0037] The computing device 200 also includes a storage interface bus 234 connected to the bus / interface controller 230. The storage interface bus 234 is connected to a storage device 232 adapted for data storage. Example storage device 232 may include removable storage 236 (e.g., CD, DVD, USB flash drive, removable hard disk, etc.) and non-removable storage 238 (e.g., hard disk drive HDD, etc.). In the computing device 200 according to the invention, application 222 includes multiple program instructions for executing method 300.

[0038] Figure 3A schematic flowchart of a wire identification method 300 according to an embodiment of the present invention is shown. Figure 3 As shown, method 300 begins with step S310, acquiring an image of the wire to be identified. The original image to be identified can be an image received by the computing device from another device, such as a camera or mobile phone. The image to be identified can be an image of a circuit structure with wires already connected in an electrical experiment. To verify the effectiveness of this method, images of wires with complex wiring, varying lengths, and overlapping or intersecting can be selected as the images to be identified.

[0039] Then, step S320 is executed to preprocess the guide wire image to obtain the target image. Preprocessing can include digitization, image denoising, grayscale conversion, and normalization of the guide wire image to be identified. This preprocessing can clarify the contours or lines of the image, selectively enhance or suppress image information to improve the visual effect, and better adapt the image information to the structure of the neural network, facilitating data extraction or image information recognition, thus improving the accuracy of image recognition.

[0040] Then, step S330 is executed, where the target image is input into the trained convolutional neural network model for feature extraction, obtaining the keypoint coordinates and categories of the wires, the affinity matrix between keypoints, and the bidirectional connection vector field between keypoints. In one embodiment of the present invention, the pre-constructed convolutional neural network structure may include a backbone network and a keypoint detection network head, an affinity prediction network head, and a vector field prediction network head connected sequentially to the backbone network. The keypoint detection network, affinity prediction network, and vector field prediction network are connected in parallel. The backbone network extracts feature information from the target image for use by subsequent networks; the head network is the network that obtains the output content of the network and makes predictions using the previously extracted feature information. The backbone network framework can use any one or more of the network structures such as AlexNet, VGGNet, ResNet, and MobileNet, and includes at least convolutional layers, pooling layers, and fully connected layers. The backbone network extracts features from the target image to obtain a feature map. The keypoint detection network determines the coordinates and categories of keypoints in the feature map. The affinity prediction network predicts the affinity matrix between keypoints in the feature map, and the vector field prediction network predicts the bidirectional relationship vector field between keypoints in the feature map. To train the pre-built convolutional neural network, multiple training images are acquired and preprocessed, including grayscale transformation, contrast enhancement, rotation, cropping, and normalization. Annotation tools are used to label the coordinates and categories of the endpoints of each wire in each training image. Then, the endpoint coordinates, endpoint categories, and relationships between endpoints in each preprocessed training image are labeled. The training images and the labeled endpoint coordinates, endpoint categories, and relationships are input into the pre-built convolutional neural network model. Supervised training is performed through three network branches: keypoint prediction supervision, keypoint affinity supervision, and keypoint vector field supervision. When the relationship between two points is known during the training of the keypoint affinity prediction network, an affinity algorithm between the two points can be established. After model training, the affinity between the two points can be predicted. When training a vector field prediction network, if the relationship between two points is known, a virtual straight line can be established between the two points a and b. The virtual straight line is expanded to obtain a region, which is the vector field range n. Within the range n, the distance between a and b is segmented in a direction, thus obtaining two vector fields a->b and b->a.

[0041] Finally, the detection performance of the convolutional neural network model is evaluated using a validation set, and the model is optimized. For example, during the training of the keypoint detection network, the coordinates of the keypoints on the guide wire can be determined by minimizing the L2 loss function between the predicted and actual keypoints. The L2 loss function is:

[0042]

[0043]

[0044]

[0045] Among them, w p Here, J represents the bias coefficient, and C represents the set of horizontal coordinates and vertical coordinates. To predict the x-coordinate of key points, To predict the ordinate of key points, The x-coordinate of the actual key point The vertical coordinate of the actual key point. The confidence error of the x-axis of the key point. Let be the confidence error of the keypoint's ordinate, and f be the keypoint confidence error. Finally, the trained convolutional neural network model is used as the final model. The target image is input into the trained convolutional neural network model for feature extraction, resulting in the keypoint coordinates and categories of the guide wire, the affinity matrix between keypoints, and the bidirectional connection vector field between keypoints.

[0046] Next, step S340 is executed. Based on the keypoint coordinates and categories, a matching algorithm is used to perform pairwise matching of the keypoints to obtain the first matching result between the keypoints. The first matching result is the maximum matching result calculated by the Hungarian algorithm. Since the keypoint feature map output by the model contains the location and category information of the keypoints, including current inflow points and current outflow points, the keypoints can be divided into two point sets according to the category information. These two point sets meet the conditions of a bipartite graph. Figure 4 A schematic diagram of a first point set and a second point set according to an embodiment of the present invention is shown. Figure 4 As shown, the first set of points represents current inflow points, and the second set represents current outflow points; or the first set represents current outflow points, and the second set represents current inflow points. The matching result can be obtained by pairwise matching of points in the two sets using the maximum flow matching algorithm or the Hungarian matching algorithm. Taking the Hungarian algorithm as an example, for a bipartite graph formed by two set of points, the maximum matching can be obtained using the following steps:

[0047] 1) Initialize the vertex numbers, let n = 1;

[0048] 2) If n is greater than the number of vertices N, end; otherwise, proceed to step 3).

[0049] 3) Use depth-first search to find an augmenting path to vertex n. If found, augment the vertex; otherwise, the vertex is not saturated, n = n + 1, and return to step 2.

[0050] Then, step S340 is executed to confirm the first matching result based on the affinity matrix between key points, thereby obtaining the second matching result between key points. To further confirm the matching result obtained in step S330, the first matching result can be filtered based on the similarity score between every two key points in the affinity matrix. Each pair of key points with a similarity score greater than a predetermined threshold is taken as the second matching result; otherwise, key points with a similarity score less than the predetermined threshold are re-matched.

[0051] The affinity matrix contains the similarity score between every two keypoints. In one embodiment of the invention, the similarity between two keypoints is calculated using the negative of the distance; that is, the closer the distance, the greater the similarity. The similarity between points i and j in the affinity matrix, or similarity matrix S, is the negative of the distance between the two points. However, the numbers on the main diagonal represent the similarity between a point and itself, but cannot be directly represented by 0. According to the algorithm requirements, the values ​​s(k,k) on the main diagonal are generally called bias parameters, which are usually equal for all k, and are taken as the median of all numbers not on the main diagonal.

[0052] Finally, step S350 is executed to confirm the second matching result based on the bidirectional connection vector field between key points, thereby obtaining multiple sets of associated key points, and determining a wire between each set of key points.

[0053] In one embodiment of the present invention, vector field calculation can be performed on the second matching result obtained in step S340. Multiple relationship vectors between two key points are determined using the bidirectional connection vector field output by the model. Then, the average value of the projections of all relationship vectors onto the line connecting the two key points is determined, and this average value is used as the relationship vector field score. Specifically, when the direction of the connection between the two key points is consistent, the corresponding relationship vector field score is 90 or higher; when the directions are orthogonal or there is a certain deviation, the score is 0 or lower than 90. The key points in the second matching result are ordered according to their relationship vector field scores; key points with relationship vector field scores greater than a preset threshold are selected as associated key points. That is, if both the forward direction (a points to b) and the reverse direction (b points to a) are satisfied, the association between the two points can be confirmed. For points whose matching relationship cannot be determined, the process returns to step S330 for rematching. Finally, multiple sets of associated key points are obtained, and a wire is determined between each set of key points.

[0054] Figure 5 A schematic diagram of the structure of a wire identification device 500 according to an embodiment of the present invention is shown. Figure 5As shown, the device includes an acquisition module 510, a preprocessing module 520, a feature extraction module 530, a matching module 540, a first confirmation module 550, and a second confirmation module 560. The acquisition module 510 acquires an image of the guide wire to be identified. The preprocessing module 520 preprocesses the guide wire image to obtain a target image. The feature extraction module 530 inputs the target image into a trained convolutional neural network model for feature extraction, obtaining the keypoint coordinates and categories of the guide wire, the affinity matrix between keypoints, and the bidirectional connection vector field between keypoints. The matching module 540 uses a matching algorithm to perform pairwise matching of keypoints based on their coordinates and categories, obtaining a first matching result between keypoints. The first confirmation module 550 confirms the first matching result based on the affinity matrix between keypoints, obtaining a second matching result between keypoints. The second confirmation module 560 confirms the second matching result based on the bidirectional connection vector field between keypoints, obtaining multiple sets of associated keypoints, with each set of keypoints identifying a guide wire.

[0055] The above scheme identifies wires based on the relationship between the two ends of the wires. It does not depend on the connection between the wires, nor is it limited by the wire color, length, intersection, overlap, shape, or size. It can accurately identify complex wires.

[0056] It should be noted that the wire identification method provided in this solution is also applicable to the identification of other linear objects, such as roads and network connections, further enhancing the beneficial effects of this solution.

[0057] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0058] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0059] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0060] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0061] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0062] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0063] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0064] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative rather than restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A wire identification method, suitable for execution in a computing device, characterized in that, The method includes: Obtain the image of the conductor to be identified; The conductor image is preprocessed to obtain the target image; The target image is input into a trained convolutional neural network model for feature extraction to obtain the key point coordinates and categories of the wire, the affinity matrix between key points, and the bidirectional connection vector field between key points. Based on the categories of key points, the key points are divided into a first set of key points and a second set of key points; wherein, the first set of key points includes current inflow points and the second set of key points includes current outflow points. The Hungarian matching algorithm is used to match points in the first set of points and points in the second set of points pairwise to obtain the first matching result between key points. The first matching result is confirmed based on the affinity matrix between key points, and a second matching result between key points is obtained; The second matching result is confirmed based on the bidirectional connection vector field between key points. For key points that do not simultaneously satisfy the positive and negative conditions of the vector field, the two-point matching is repeated. Two key points that simultaneously satisfy the positive and negative conditions of the vector field are confirmed as associated key points to obtain multiple sets of associated key points. A wire is determined between each set of key points.

2. The method according to claim 1, characterized in that, The step of confirming the first matching result based on the affinity matrix between key points to obtain the second matching result between key points includes: The first matching result is filtered based on the similarity score between each pair of key points in the affinity matrix. Each pair of key points with a similarity score greater than a predetermined threshold is taken as the second matching result; otherwise, key points with a similarity score less than the predetermined threshold are re-matched.

3. The method according to claim 2, characterized in that, The step of confirming the second matching result based on the bidirectional connection vector field between key points to obtain multiple sets of associated key points includes: The bidirectional connection vector field is used to determine multiple relationship vectors between two key points; Determine the average value of all the relationship vectors projected onto the line connecting the two key points, and use the average value as the relationship vector field score; Sort the relation vector field scores of the key points in the second matching results; Keypoints with a relation vector field score greater than a preset threshold are used as associated keypoints, while keypoints with a relation vector field score less than the preset threshold are re-matched.

4. The method according to claim 1, characterized in that, The convolutional neural network model includes a backbone network, a keypoint detection network connected to the backbone network, an affinity prediction network, and a vector field prediction network. The keypoint detection network, affinity prediction network, and vector field prediction network are connected in parallel. The backbone network is adapted to extract features from the target image to obtain a feature map, the key point detection network is adapted to determine the coordinates and categories of key points in the feature map, the affinity prediction network is adapted to predict the affinity matrix between key points in the feature map, and the vector field prediction network is adapted to predict the bidirectional relationship vector field between key points in the feature map.

5. The method according to claim 1, characterized in that, Prior to the step of acquiring the image of the conductor to be identified, the method further includes: Acquire multiple training images; The multiple training images are preprocessed; The endpoint coordinates, endpoint categories, and relationships between endpoints of the wires in each preprocessed training image are labeled. The training image and the labeled endpoint coordinates, endpoint categories, and relationships between endpoints are input into a pre-built convolutional neural network model for training, resulting in a trained convolutional neural network model.

6. The method according to claim 5, characterized in that, The preprocessing includes any one or more of the following: grayscale transformation, contrast enhancement, rotation, cropping, and normalization.

7. A wire identification device, characterized in that, The device includes: The acquisition module is suitable for acquiring images of the conductor to be identified. The preprocessing module is adapted to preprocess the conductor image to obtain the target image; The feature extraction module is adapted to input the target image into a trained convolutional neural network model for feature extraction, and obtain the key point coordinates and categories of the wire, the affinity matrix between key points, and the bidirectional connection vector field between key points; The matching module is suitable for dividing key points into a first set of points and a second set of points based on the category of key points; wherein the first set of points includes current inflow points and the second set of points includes current outflow points; and matching points in the first set of points and points in the second set of points pairwise based on the Hungarian matching algorithm to obtain the first matching result between key points. The first confirmation module is adapted to confirm the first matching result based on the affinity matrix between key points, and obtain a second matching result between key points; The second confirmation module is adapted to confirm the second matching result based on the bidirectional connection vector field between key points. For key points that do not simultaneously satisfy the positive and negative conditions of the vector field, the two-point matching is returned and re-performed. Two key points that simultaneously satisfy the positive and negative conditions of the vector field are confirmed as associated key points to obtain multiple sets of associated key points, and a wire is determined between each set of key points.

8. A computing device, comprising: At least one processor; and A memory storing program instructions configured to be executed by the at least one processor, the program instructions including instructions for performing the method as described in any one of claims 1-6.

9. A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-6.

Images