A wire tracking recognition method based on gradient transformation

Through the wire track recognition method based on gradient transformation, the difficulty of wire identification and connection judgment in physical circuit experiments is solved, and the complete extraction of wires and connection relationship judgment is realized, which improves the accuracy and reliability of recognition.

CN114565929BActive Publication Date: 2024-11-12NANJING HONGSONG INFORMATION TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210134424.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-14
Publication Date
2024-11-12
Estimated Expiration
2042-02-14

AI Technical Summary

Technical Problem

In physical circuit experiments, there are difficulties in identifying and connecting wires, especially due to the influence of the light environment, the wire extraction and tracking are difficult to effectively solve.

Method used

Using the wire tracking recognition method based on gradient transformation, the video image is read frame by frame, denoising and edge profile detection are performed, the wire profile is extracted, and the target recognition model positioning electrode is combined to perform wire tracking and direction tracking, and the path is optimized. Select to determine the connection status of the equipment at both ends of the wire.

Benefits of technology

The complete extraction and connection relationship of the wire is achieved, and the connection relationship between the devices at both ends of the wire can be accurately determined in the image, improving the accuracy and reliability of wire recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114565929B_ABST
    Figure CN114565929B_ABST
Patent Text Reader

Abstract

The present invention relates to a method for wire tracing and identification based on gradient transformation, and the steps are as follows: S1 reads a video frame image; S2 wire extraction: performs denoising on the video frame image, and then performs edge contour detection to extract the wire in the video frame image to obtain a wire image; S3 wire tracing: performs target detection according to a target recognition model to obtain the position coordinates of all device electrodes, realizes electrode positioning, removes the electrodes in the wire image according to the electrode positioning, and then performs wire tracing and direction tracing to obtain a wire path; S4 device connection judgment: according to the wire path obtained in step S3, selects two endpoints constituting the wire path, judges the contact between the coordinates of the two endpoints and the position coordinates of the device electrodes, and thus judges the devices connected at both ends of the wire. The method can completely extract the wire to be identified, and performs tracing according to the actual connection direction of the wire, so as to determine the connection relationship of the devices at both ends of the wire from the image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, and in particular relates to a wire tracing recognition method based on gradient transformation, which is used to solve the problem of identifying the direction of wires in physical circuits. Background Art

[0002] With the rapid development of artificial intelligence technology, more and more intelligent products have been launched and widely used in various industries. In recent years, the core technology of artificial intelligence has begun to be applied in the field of education, and has gradually achieved great development. Various educational products have emerged in an endless stream, and the accompanying artificial intelligence services can be seen everywhere. For example: the automated review service for test papers and homework, which not only abandons the high error rate and inefficiency brought by the traditional review method, but also replaces the traditional review method to a certain extent. At present, with the gradual rise of the footsteps of educational reform, the educational reform that keeps pace with the times has prompted artificial intelligence to develop in a more favorable direction. Artificial intelligence is no longer limited to the simple field of image processing technology, but has turned to the deeper field of video processing technology, such as: the automated review service for experiments, which can process and analyze video content and give review results.

[0003] As a new review method, automatic video review brings together a variety of image processing technologies. However, there are also some technical barriers in the actual application process. For example, in physical circuit experiments, it is necessary to determine the specific devices connected to the two ends of the wire. However, there are many complex situations in the actual connection of the wire, and the detection of the wire is easily affected by the environment such as light, which makes it difficult to extract and track the wire. In order to be able to completely extract the wire from the image and determine the actual connection status of the devices at both ends of the wire, it is particularly important to use a combination of traditional and deep methods. Summary of the invention

[0004] The technical problem to be solved by the present invention is to propose a wire tracing recognition method based on gradient transformation for the recognition of wires in images, which is mainly used to solve the problem of connection judgment of wires between devices in physical circuit experiments.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: the wire tracing identification method based on gradient transformation specifically comprises the following steps:

[0006] S1 reads video frame images: reads video information frame by frame and obtains video frame images;

[0007] S2 wire extraction: performing denoising on the video frame image obtained in step S1, and then performing edge contour detection to extract wires in the video frame image to obtain a wire image;

[0008] S3 wire tracking: perform target detection according to the target recognition model to obtain the position coordinates of all device electrodes, realize electrode positioning, remove the electrodes in the wire image extracted in step S2 according to the electrode positioning, and then perform wire tracking and direction tracking to obtain the wire path;

[0009] S4 device connection judgment: according to the wire path obtained in step S3, two endpoints constituting the wire path are selected, and the contact between the coordinates of the two endpoints and the position coordinates of the device electrodes is judged, so as to judge the devices connected to the two ends of the wire.

[0010] Using the above technical solution, firstly, the contour detection technology based on gradient transformation is used to extract the wire contour in each frame of the image; secondly, all wires are extracted according to the general contour of the wire, and then the wires are refined; then, the wires are traced from the device electrodes to obtain multiple wire connection paths between the device electrodes; finally, different path results are optimized and selected to determine the devices actually connected between the wires. This wire tracing identification method based on gradient transformation can completely extract the wires to be identified, and trace them according to the actual connection direction of the wires, so as to determine the connection relationship of the devices at both ends of the wires from the image.

[0011] As a preferred technical solution of the present invention, in step S1, the streaming media protocol (RTSP) is used to pull the video stream or directly read the video file in mp4 format. All video frame images are read as three-channel input, denoted as: src, and its function is expressed as: f(x, y).

[0012] As a preferred technical solution of the present invention, the specific steps of step S2 are:

[0013] S21: Use a general bilateral filtering or Gaussian filtering algorithm to perform denoising on the video frame image, suppress image noise, and remove noise interference in the image;

[0014] S22: performing enhancement processing on the edge contour of the wire to obtain target extraction of the highlight part in the video frame image;

[0015] S23: performing edge contour detection on the video frame image of the highlight part extracted in step S22;

[0016] S24: After all target contours in the video frame image are detected, the detected image contours are screened to obtain complete extraction of the wires in the video frame image, thereby obtaining a wire image, which is recorded as: srcLineImg image.

[0017] As a preferred technical solution of the present invention, the specific steps of enhancing the edge profile of the wire in step S22 are:

[0018] S221: First, use the Laplace operator to perform second-order gradient derivation on the image, as shown in formula (1),

[0019]

[0020] S222: The gradient image after derivation is then subjected to highlight enhancement processing, and color filtering is performed through the three channels of RGB, so as to achieve target extraction of the highlight part of the image. The corresponding filtering value of the R channel color is greater than 200; the corresponding filtering value of the G channel color is greater than 210, and the corresponding filtering value of the B channel color is greater than 220.

[0021] As a preferred technical solution of the present invention, the specific steps of performing edge contour detection in step S23 are:

[0022] S231: The detection starts from the upper left corner of the image, and traverses the first non-zero pixel value point in the image by row and column, recorded as: p(x, y), and assigns it a label label, and then pushes all foreground pixels adjacent to the pixel into the stack;

[0023] S232: Pop the top pixel of the stack, assign a label to the top pixel of the stack, and then push all foreground pixels adjacent to the top pixel of the stack into the stack. Repeat this process until the stack is empty, and the connected area of ​​edge detection is obtained.

[0024] As a preferred technical solution of the present invention, the specific steps of step S3 are:

[0025] S31: train the experimental equipment samples according to the target recognition neural network (such as Yolo series network, CenterNet network, etc.) to obtain the target recognition model, and then obtain the position coordinates of all equipment electrodes through the target recognition model, where the electrode types are divided into two categories, one is the wiring electrode and the other is the non-wiring electrode. The coordinates of all wiring electrodes are saved in the list P, which is recorded as: P = [((x 0 ,y 0 ),(x' 0 ,y' 0 )),…,((x n ,y n ),(x' n ,y' n ))]; where (x 0 ,y 0 ) represents the upper left corner of the electrode column positioning frame, (x' 0 ,y' 0 ) represents the lower right corner of the electrode column positioning frame;

[0026] S32: traverse the electrode coordinate list P, and enlarge the electrode coordinates from the srcLineImg image in the x and y directions respectively to ensure that the electrode positioning frame can contact the wire; preferably, enlarge 10-20 pixels;

[0027] S33: Binarize the srcLineImg image, extract the central axis of the connected pixels of the srcLineImg image according to the judgment condition, and loop through all non-zero pixels in the entire srcLineImg image to obtain the extracted centerline skeleton image;

[0028] S34: Traverse the electrode coordinate list P, and enlarge the electrode coordinates from the midline skeleton image in the x and y directions respectively. Here, it is preferred to enlarge by 10 to 20 pixels to ensure that the electrode positioning frame can contact the wire; all contact point pixel coordinates are marked as: B = [(x a0 ,y b0 ),(x a1 ,y b1 )…,(x an ,y bn )]; where (x a0 ,y b0 ) represents a contact point B 0 Coordinates of. For example: (B 0 ,…,B′ i ) indicates that the starting coordinate point B 0 To the end coordinate point B' i The pixel path composed of the pixels may represent a complete wire or a part of a wire.

[0029] S35: traverse the contact point coordinate list B, take each contact point as the starting point of the wire tracing, use the breadth-first search traversal algorithm to traverse all paths, and then perform path screening to obtain the wire path.

[0030] As a preferred technical solution of the present invention, the specific steps of step S33 are: binarizing the srcLineImg image, setting the values ​​of all non-zero pixels to 1, and then extracting the central axis of the connected pixels of the entire image based on the values ​​of each pixel and the surrounding 8 adjacent pixels as the judgment condition. The entire extraction process is expanded by traversing all non-zero point pixels with a 3*3 convolution kernel, and the pixel matrix corresponding to the convolution kernel is expressed as follows:

[0031]

[0032] Among them, p 1 is the pixel value of the center point. The default setting is: 1 means retaining the pixel point; when condition 1 or condition 2 is met at the same time, p 1The value of is set to 0, indicating that the pixel is deleted; all non-zero pixels in the entire image are looped through to obtain the extracted midline skeleton image, recorded as: skeletonLineImg image;

[0033] As a preferred technical solution of the present invention, the judgment condition in step S33 includes judgment condition 1 and judgment condition 2, and the judgment condition 1 is:

[0034] (1-1)

[0035] (1-2)p i p i+1 (i=2,3,…,8), satisfying p i p i+1 The number of values ​​equal to "01" is 1;

[0036] (1-3)p 2 p 4 p 6 =0;

[0037] (1-4)p 4 p 6 p 8 =0;

[0038] The second judgment condition is:

[0039] (2-1)

[0040] (2-2)p i p i+1 (i=2,3,…,8), satisfying p i p i+1 The number of values ​​equal to "01" is 1;

[0041] (2-3)p 2 p 4 p 8 =0;

[0042] (2-4)p 2 p 6 p 8 =0.

[0043] As a preferred technical solution of the present invention, the screening conditions for path screening in step S35 include screening condition 1 and screening condition 2, wherein the screening condition 1 is whether the number of pixel points constituting the path meets the set threshold G; the screening condition 2 is whether the change of the coordinate slope between adjacent pixel points is consistent; specifically, the following steps are included:

[0044] S351: Count the number of pixels contained in each path in the list R, denoted as: S = [s 0,s 1 ,…,s n ],(n∈N * ), if the corresponding s i <G(i=1,…,n), then set the index value at the corresponding position to 0, otherwise, set it to 1, and save the index list as: I;

[0045] S352: Based on the short-distance invariance principle, the coordinates of the adjacent pixels (x ai ,y bi ) is used to calculate the slope, and the coordinates are calculated as shown in formula (2):

[0046]

[0047] Calculate the coordinate slope of all pixel points in the path, recorded as:

[0048] K a =[k 0 ,k 1 ,…,k b ], (a∈0,1,…,n; b∈i,j,…,n);

[0049] S353: Combined with the result of the path pixel number threshold screening in the screening condition 1, the first round of path optimization selection is performed and the optimization result is retained, which is recorded as: R A =(R·I T ) T ;

[0050] Among them, the non-zero items represent the retained paths; assuming that the paths retained after optimization are: R A =[0,0,…,(B 0 , ..., B' j ),0,…(B 0 ,…,B' n-1 ),0], and then traverse the path list R A For all non-zero sub-elements in , a general dynamic programming algorithm is used to solve their longest common subsequence. The solution of the path between two non-zero sub-elements is shown in formula (3);

[0051] LCS=max{LCS(r a ,r b )},(a,b∈j,…,n-1;a≠b) (3)

[0052] Among them, r j and r n-1 Respectively represent the path sequence (B 0 , ..., B' j ) and (B 0 ,…,B' n-1);Determine the path r according to the obtained maximum common subsequence a and r b and intersection B LCS-1 , and then use the coordinate slope K of all points on the path a =[k 0 ,k 1 ,…,k a ] and K b =[k 0 ,k 1 ,…,k b ]; respectively compare K a and K b Intersection point B LCS-1 The slope change after the corresponding index position LCS-1, if the slopes of multiple adjacent pixel coordinates are similar, the corresponding path is selected as the final wire path.

[0053] As a preferred technical solution of the present invention, the convolution kernel size of the Gaussian filter used in step S21 can be set to: 3*3, wherein the standard deviation of the filter convolution kernel in the X-axis and Y-axis directions is set, and the standard deviation size is set to: 3~5. The purpose of image blurring and image smoothing can be basically achieved through Gaussian filtering; the conditions for image contour screening in step S24 include: (1) whether the number of contour points meets the threshold C; (2) whether the area of ​​the contour meets the threshold S; (3) whether the circumscribed rectangle of the contour meets the threshold B; (4) whether the aspect ratio of the contour meets the threshold T.

[0054] Compared with the prior art, the present invention has the following beneficial effects: first, the contour detection technology based on gradient transformation is used to extract the contour of the wire in each frame of the image; second, all the wires are extracted according to the general contour of the wire, and then the wires are refined; then, the wires are traced from the device electrodes to obtain multiple wire connection paths between the device electrodes; finally, different path results are optimized and selected to determine the devices actually connected between the wires. The wire tracing identification method based on gradient transformation can completely extract the wire to be identified, and trace it according to the actual connection direction of the wire, so as to determine the connection relationship of the devices at both ends of the wire from the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The technical solution of the present invention is further described below in conjunction with the accompanying drawings:

[0056] Figure 1 It is a flow chart of the wire tracking identification method based on gradient transformation of the present invention;

[0057] Figure 2 is an effect diagram of complete extraction of the wire obtained in step S2 of the wire tracing identification method based on gradient transformation of the present invention;

[0058] Figure 3 It is an effect diagram of the midline skeleton image obtained in step S33 of the wire tracing identification method based on gradient transformation of the present invention. DETAILED DESCRIPTION

[0059] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The embodiments are only used to explain the present invention and do not limit the protection scope of the present invention.

[0060] Example: Figure 1 As shown, the wire tracking identification method based on gradient transformation specifically includes the following steps:

[0061] S1 reads video frame images: reads video information frame by frame and obtains video frame images; in the step S1, a streaming media protocol (such as RTSP) is used to pull the video stream or directly read the video file in mp4 format. All video frame images are read as three-channel input, denoted as src, and its function is expressed as f(x, y);

[0062] S2 wire extraction: performing denoising on the video frame image obtained in step S1, and then performing edge contour detection to extract wires in the video frame image, to obtain a wire image, which is recorded as: srcLineImg image;

[0063] The specific steps of step S2 are:

[0064] S21: Use a general bilateral filtering or Gaussian filtering algorithm to perform denoising on the video frame image, suppress image noise, and remove noise interference in the image;

[0065] The convolution kernel size of the Gaussian filter used in step S21 can be set to: 3*3, wherein in order to completely retain the detail information of the wire part, the standard deviation of the filter convolution kernel in the X-axis and Y-axis directions is set, and the standard deviation size is set to: 3-5; the purpose of image blurring and image smoothing can be basically achieved through Gaussian filtering;

[0066] S22: In order to completely extract all wires from the source image, it is necessary to enhance the edge contours of the wires. Therefore, the edge contours of the wires are enhanced to obtain target extraction of the highlighted part in the video frame image.

[0067] The specific steps of enhancing the edge profile of the wire in step S22 are:

[0068] S221: First, use the Laplace operator to perform second-order gradient derivation on the image, as shown in formula (1),

[0069]

[0070] S222: Highlight enhancement processing is then performed on the gradient image after derivation, and color filtering is performed through the three channels of RGB respectively, so as to achieve target extraction of the highlight part in the image; wherein the filtering corresponding value of the R channel color is greater than 200; the filtering corresponding value of the G channel color is greater than 210, and the filtering corresponding value of the B channel color is greater than 220.

[0071] S23: performing edge contour detection on the video frame image of the highlight part extracted in step S22;

[0072] The specific steps of edge contour detection in step S23 are:

[0073] S231: The detection starts from the upper left corner of the image, and traverses the first non-zero pixel value point in the image by row and column, recorded as: p(x, y), and assigns it a label label, and then pushes all foreground pixels adjacent to the pixel into the stack;

[0074] S232: Pop the top pixel of the stack, assign a label to the top pixel of the stack, and then push all foreground pixels adjacent to the top pixel of the stack into the stack. Repeat this process until the stack is empty, and the connected area of ​​edge detection is obtained.

[0075] S24: After all target contours in the video frame image are detected, the detected image contours are screened. The conditions for image contour screening include: (1) whether the number of contour points meets the threshold C; (2) whether the area of ​​the contour meets the threshold S; (3) whether the circumscribed rectangle of the contour meets the threshold B; (4) whether the aspect ratio of the contour meets the threshold T, etc. All thresholds are empirical values, and finally the complete extraction of the wire in the image is achieved, such as Figure 2 As shown;

[0076] S3 wire tracking: Target detection is performed according to the target recognition model to obtain the position coordinates of all device electrodes, and electrode positioning is achieved. According to the electrode positioning, the electrodes in the wire image extracted in step S2 are removed, and then a path search algorithm is used to perform wire tracking and direction tracking to obtain the wire path;

[0077] The specific steps of step S3 are:

[0078] S31: According to the existing target recognition neural network, such as Yolo series network, CenterNet network, etc., the experimental equipment samples are trained to obtain the target recognition model, and then the position coordinates of all equipment electrodes are obtained through the target recognition model. The electrode types are divided into two categories, one is the wiring electrode and the other is the non-wiring electrode. The coordinates of all wiring electrodes are saved in the list P, which is recorded as: P = [((x 0 ,y 0 ),(x'0 ,y' 0 )),…,((x n ,y n ),(x' n ,y' n ))]; where (x 0 ,y 0 ) represents the upper left corner of the electrode column positioning frame, (x' 0 ,y' 0 ) represents the lower right corner of the electrode column positioning frame;

[0079] S32: traverse the electrode coordinate list P, and enlarge the electrode coordinates from the srcLineImg image in the x and y directions respectively. It is recommended to enlarge by 10 to 20 pixels to ensure that the electrode positioning frame can contact the wire;

[0080] S33: Binarize the srcLineImg image, extract the central axis of the connected pixels of the srcLineImg image according to the judgment condition, then refine and scale the image, loop through all non-zero pixels in the entire srcLineImg image, and obtain the extracted centerline skeleton image;

[0081] The specific steps of step S33 are: binarize the srcLineImg image, set all non-zero pixel values ​​to 1, and then extract the central axis of the connected pixels of the entire image based on the values ​​of each pixel and the surrounding 8 adjacent pixels as the judgment condition. The entire extraction process is expanded by traversing all non-zero point pixels with a 3*3 convolution kernel. The pixel matrix corresponding to the convolution kernel is expressed as follows:

[0082]

[0083] Among them, p 1 is the pixel value of the center point. The default setting is: 1, which means retaining the pixel point; when condition 1 or condition 2 is met at the same time, p 1 The value of is set to 0, indicating that the pixel is deleted; all non-zero pixels in the entire image are looped through to obtain the extracted midline skeleton image, recorded as: skeletonLineImg image; Figure 3 As shown;

[0084] The judgment conditions in step S33 include judgment condition 1 and judgment condition 2, and the judgment condition 1 is:

[0085] (1-1)

[0086] (1-2)p i p i+1 (i=2,3,…,8), satisfying p i pi+1 The number of values ​​equal to "01" is 1;

[0087] (1-3)p 2 p 4 p 6 =0;

[0088] (1-4)p 4 p 6 p 8 =0;

[0089] The second judgment condition is:

[0090] (2-1)

[0091] (2-2)p i p i+1 (i=2,3,…,8), satisfying p i p i+1 The number of values ​​equal to "01" is 1;

[0092] (2-3)p 2 p 4 p 8 =0;

[0093] (2-4)p 2 p 6 p 8 =0.

[0094] S34: Traverse the electrode coordinate list P, and enlarge the electrode coordinates from the midline skeleton image in the x and y directions respectively to ensure that the electrode positioning frame can contact the wire; all contact point pixel coordinates are marked as: B = [(x a0 ,y b0 ),(x a1 ,y b1 )…,(x an ,y bn )]; where (x a0 ,y b0 ) represents a contact point B 0 Coordinates of; such as: (B 0 ,…,B′ i ) indicates that the starting coordinate point B 0 To the end coordinate point B' i The pixel path composed of the image may represent a complete wire or a part of a wire;

[0095] S35: traverse the contact point coordinate list B, take each contact point as the starting point of the wire tracing, use the breadth-first search traversal algorithm to traverse all paths, and then perform path screening to obtain the wire path;

[0096] Contact point B 0 As an example, starting from the coordinates, we can get the entire path composed of the coordinates of consecutive pixel points, which is recorded as: R = [(B 0 ,…,B′ i ),(B 0 ,…,B' j ),…,(B 0 ,…,B' n )],(i,j,…,n∈N * ); each element in the list R represents a path, such as: (B 0 ,…,B′ i ) indicates that the starting coordinate point B 0 To the end coordinate point B' i The pixel point path composed of the plurality of pixels may represent a complete wire or a part of the wire. In order to exclude the path irrelevant to the actual wire, it is necessary to add a path screening condition. The path screening condition includes screening condition 1 and screening condition 2. The screening condition 1 is whether the number of pixel points constituting the path meets the set threshold G. The screening condition 2 is whether the change of the coordinate slope between adjacent pixel points is consistent. Specifically, the following steps are included:

[0097] S351: Count the number of pixels contained in each path in the list R, denoted as: S = [s 0 ,s 1 ,…,s n ],(n∈N * ), if the corresponding s i <G(i=1,…,n), then set the index value at the corresponding position to 0, otherwise, set it to 1, and save the index list as: I;

[0098] S352: Based on the continuity of pixels, in order to ensure the correctness of the wire direction, the short-distance invariance principle is used as a basis for judgment, and the coordinates of the adjacent pixels (x ai ,y bi ) is used to calculate the slope, and the coordinates are calculated as shown in formula (2):

[0099]

[0100] Calculate the coordinate slope of all pixel points in the path, recorded as:

[0101] K a =[k 0 ,k 1 ,…,k b ], (a∈0,1,…,n; b∈i,j,…,n);

[0102] S353: Combined with the result of the path pixel number threshold screening in the screening condition 1, the first round of path optimization selection is performed and the optimization result is retained, which is recorded as: R A =(R·I T ) T ;

[0103] Among them, the non-zero items represent the retained paths; assuming that the paths retained after optimization are: R A =[0,0,…,(B 0 , ..., B' j ),0,…(B 0 ,…,B' n-1 ),0], and then traverse the path list R A For all non-zero sub-elements in , a general dynamic programming algorithm is used to solve their longest common subsequence. The solution of the path between two non-zero sub-elements is shown in formula (3);

[0104] LCS=max{LCS(r a ,r b )},(a,b∈j,…,n-1;a≠b) (3)

[0105] Among them, r j and r n-1 Represent the path sequence (B 0 , ..., B' j ) and (B 0 ,…,B' n-1 );Determine the path r according to the obtained maximum common subsequence a and r b and intersection B LCS-1 , and then use the coordinate slope K of all points on the path a =[k 0 ,k 1 ,…,k a ] and K b =[k 0 ,k 1 ,…,k b ]; respectively compare K a and K b Intersection point B LCS-1 The slope change after the corresponding index position LCS-1, if the slopes of multiple adjacent pixel coordinates are similar, the corresponding path is selected as the final wire path.

[0106] S4 Device connection determination: According to the wire path obtained by the path search algorithm in step S3, select the two endpoints that constitute the wire path, which are recorded as: (x a0 ,y b0 ) and (x an ,ybn ); then determine the contact between the two endpoint coordinates and the position coordinate frame of the relevant device electrode, and the coordinates of the upper left corner and the lower right corner of the electrode frame can be enlarged by 5 to 10 pixels, such as: ((x 0 +5,y 0 +5),(x' 0 +5,y' 0 +5)), if x 0 +5<=x a0 <=x' 0 +5 and y 0 +5<=y a0 <=y' 0 +5, meet the contact point (x a0 ,y b0 ) The condition within the electrode frame can directly determine that one end of the wire is connected to the electrode; similarly, when it is determined that the contact point at the other end of the wire is connected to other electrodes, the actual device connected to both ends of the wire can be finally determined.

[0107] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A wire tracking identification method based on gradient transformation, characterized in that: The specific steps include: S1 reads video frame images: reads video information frame by frame and obtains video frame images; S2 wire extraction: performing denoising on the video frame image obtained in step S1, and then performing edge contour detection to extract wires in the video frame image to obtain a wire image; S3 wire tracking: perform target detection according to the target recognition model to obtain the position coordinates of all device electrodes, realize electrode positioning, remove the electrodes in the wire image extracted in step S2 according to the electrode positioning, and then perform wire tracking and direction tracking to obtain the wire path; S4 device connection determination: according to the wire path obtained in step S3, two endpoints constituting the wire path are selected, and the contact between the coordinates of the two endpoints and the position coordinates of the device electrodes is determined, so as to determine the devices connected to the two ends of the wire; The specific steps of step S3 are: S31: According to the target recognition neural network, the experimental equipment samples are trained to obtain the target recognition model, and then the position coordinates of all equipment electrodes are obtained through the target recognition model. The electrode types are divided into two categories, one is the wiring electrode and the other is the non-wiring electrode. The coordinates of all wiring electrodes are saved in the list P, which is recorded as: P = [((x0, y0), (x'0, y'0)),…, ((x n ,y n ),(x' n ,y' n ))]; where (x0, y0) represents the upper left corner of the electrode column positioning frame, and (x'0, y'0) represents the lower right corner of the electrode column positioning frame; S32: traverse the electrode coordinate list P, and enlarge the electrode coordinates from the srcLineImg image in the x and y directions respectively to ensure that the electrode positioning frame can contact the wire; S33: Binarize the srcLineImg image, extract the central axis of the connected pixels of the srcLineImg image according to the judgment condition, and loop through all non-zero pixels in the entire srcLineImg image to obtain the extracted centerline skeleton image; S34: Traverse the electrode coordinate list P, and enlarge the electrode coordinates from the midline skeleton image in the x and y directions respectively to ensure that the electrode positioning frame can contact the wire; all contact point pixel coordinates are marked as: B = [(x a0 ,y b0 ),(x a1 ,y b1 )…,(x an ,y bn )]; where (x a0 ,y b0 ) represents the coordinates of a contact point B0; S35: traverse the contact point coordinate list B, take each contact point as the starting point of the wire tracing, use the breadth-first search traversal algorithm to traverse all paths, and then perform path screening to obtain the wire path.

2. The wire tracing identification method based on gradient transformation according to claim 1, characterized in that: In the step S1, a streaming media protocol is used to pull the video stream or directly read the video file in mp4 format. All video frame images are read as three-channel input, denoted as: src, and its function is expressed as: f(x, y).

3. The wire tracing identification method based on gradient transformation according to claim 1, characterized in that: The specific steps of step S2 are: S21: Use a general bilateral filtering or Gaussian filtering algorithm to perform denoising on the video frame image, suppress image noise, and remove noise interference in the image; S22: performing enhancement processing on the edge contour of the wire to obtain target extraction of the highlight part in the video frame image; S23: performing edge contour detection on the video frame image of the highlight part extracted in step S22; S24: After all target contours in the video frame image are detected, the detected image contours are screened to obtain complete extraction of the wires in the video frame image, thereby obtaining a wire image, which is recorded as: srcLineImg image.

4. The method for wire tracing identification based on gradient transformation according to claim 3, characterized in that: The specific steps of enhancing the edge profile of the wire in step S22 are: S221: First, use the Laplace operator to perform second-order gradient derivation on the image, as shown in formula (1), S222: Highlight enhancement processing is then performed on the derived gradient image, and color filtering is performed through the RGB three channels respectively, so as to achieve target extraction of the highlight part in the image.

5. The method for wire tracing identification based on gradient transformation according to claim 4, characterized in that: The specific steps of edge contour detection in step S23 are: S231: The detection starts from the upper left corner of the image, and traverses the first non-zero pixel in the image by rows and columns, denoted as: p(x, y), and assigns it a label label, and then pushes all foreground pixels adjacent to the pixel into the stack; S232: Pop the top pixel of the stack, assign a label to the top pixel of the stack, and then push all foreground pixels adjacent to the top pixel of the stack into the stack. Repeat this process until the stack is empty, and the connected area of ​​edge detection is obtained.

6. The wire tracing identification method based on gradient transformation according to claim 1, characterized in that: The specific steps of step S33 are: binarize the srcLineImg image, set all non-zero pixel values ​​to 1, and then extract the central axis of the connected pixels of the entire image based on the values ​​of each pixel and the surrounding 8 adjacent pixels as the judgment condition. The entire extraction process is expanded by traversing all non-zero point pixels with a 3*3 convolution kernel. The pixel matrix corresponding to the convolution kernel is expressed as follows: Among them, p1 is the pixel value of the center point, and the default setting is: 1, indicating that the pixel point is retained; when condition 1 or condition 2 is met at the same time, the value of p1 is set to: 0, indicating that the pixel point is deleted; all non-zero pixels in the entire image are looped through to obtain the extracted midline skeleton image, which is recorded as: skeletonLineImg image.

7. The method for wire tracing identification based on gradient transformation according to claim 6, characterized in that: The judgment conditions in step S33 include judgment condition 1 and judgment condition 2, and the judgment condition 1 is: (1-1) (1-2)p i p i+1 (i=2,3,…,8), satisfying p i p i+1 The number of values ​​equal to "01" is 1; (1-3)p2p4p6=0; (1-4)p4p6p8=0; The second judgment condition is: (2-1) (2-2)p i p i+1 (i=2,3,…,8), satisfying p i p i+1 The number of values ​​equal to "01" is 1; (2-3)p2p4p8=0; (2-4)p2p6p8=0.

8. The method for wire tracing identification based on gradient transformation according to claim 6, characterized in that: The screening conditions for path screening in step S35 include screening condition 1 and screening condition 2, wherein the screening condition 1 is whether the number of pixel points constituting the path meets a set threshold value G; The second screening condition is whether the changes in the coordinate slopes of adjacent pixel points are consistent; specifically, it includes the following steps: S351: Count the number of pixels contained in each path in the list R, denoted as: S = [s0, s1, ..., s n ],(n∈N * ), if the corresponding s i <G(i=1,…,n), then set the index value at the corresponding position to 0, otherwise, set it to 1, and save the index list as: I; S352: Based on the short-distance invariance principle, the coordinates of the adjacent pixels (x ai ,y bi ) is used to calculate the slope, and the coordinates are calculated as shown in formula (2): Calculate the coordinate slope of all pixel points in the path, recorded as: K a =[k0,k1,…,k b ],(a∈0,1,…,n;b∈i,j,…,n); S353: Combined with the result of the path pixel number threshold screening in the screening condition 1, the first round of path optimization selection is performed and the optimization result is retained, which is recorded as: R A =(R·I T ) T ; Among them, the non-zero items represent the retained paths; assuming that the paths retained after optimization are: R A =[0,0,...,(B0,...,B' j ),0,...(B0,…,B' n-1 ),0], and then traverse the path list R A For all non-zero sub-elements in , a general dynamic programming algorithm is used to solve their longest common subsequence. The solution of the path between two non-zero sub-elements is shown in formula (3); LCS=max{LCS(r a ,r b )},(a,b∈j,…,n-1;a≠b) (3) Among them, r j and r n-1 Respectively represent the path sequence (B0, ..., B' j ) and (B0,…,B' n-1 );Determine the path r according to the obtained maximum common subsequence a and r b and intersection B LCS-1 , and then use the coordinate slope K of all points on the path a =[k0,k1,…,k a ] and K b =[k0,k1,…,k b ]; respectively compare K a and K b Intersection point B LCS-1 The slope change after the corresponding index position LCS-1, if the slopes of multiple adjacent pixel coordinates are similar, the corresponding path is selected as the final wire path.

9. The method for wire tracing identification based on gradient transformation according to claim 3, characterized in that: The convolution kernel size of the Gaussian filter used in step S21 can be set to: 3*3, wherein the standard deviation of the filter convolution kernel in both the X-axis and Y-axis directions is set, and the standard deviation size is set to: 3-5; the conditions for image contour screening in step S24 include: (1) whether the number of contour points meets the threshold C; (2) whether the area of ​​the contour meets the threshold S; (3) whether the circumscribed rectangle of the contour meets the threshold B; (4) whether the aspect ratio of the contour meets the threshold T.

Citation Information

Patent Citations

  • Corner-based aerial target detection method

    WO2021098163A1