A multi-class contour tracking method, device and medium for multi-class images

Abstract

The present invention discloses a multi-class contour tracking method, device and medium for multi-class images, which relates to the technical field of industrial production contour tracking. The method includes a segmentation and assignment process, a starting point acquisition process, a contour point tracking process, and a termination execution process. In the embodiments of the present invention, only one scan of the multi-class images is required to quickly and accurately obtain the information of the inner and outer contours of all target objects of different classes in the images, so that the inclusion, overlap and other relationships between different class connected domains will not be lost, the accuracy of the contour information can be guaranteed, and at the same time, the time and space complexity of the algorithm are low, which is suitable for devices with low computing power and is convenient to be transplanted to existing low-power industrial devices.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial production contour tracking, and particularly relates to a multi-class contour tracking method, device and medium for multi-class images. Background Art

[0002] Contour tracking, sometimes also called boundary tracking, is one of the basic techniques in image processing. Its purpose is to obtain the contour information of the target object in an image. Based on the contour information, the geometric features of the target object, such as angles, areas, perimeters, curvatures, centers, eccentricities, and projections, can be calculated more accurately. Contour tracking algorithms are the basis for image processing such as image compression, object shape representation, object recognition, and contour-based region analysis, and have a wide range of industrial applications.

[0003] With the continuous development of science and technology, the image sizes in image processing are getting larger and larger, the number of target object categories extracted from images is increasing, and the accuracy requirements are getting higher and higher. In recent years, image segmentation technology based on deep learning has made it easier and more refined to segment images into different categories, and thus the applications of image processing are more extensive. There are several important characteristics in industrial vision detection application scenarios: high real-time requirements, high running stability requirements, high accuracy requirements, low device power consumption requirements, and cost sensitivity. Here, the real-time requirements not only refer to the running speed requirements of the algorithm itself, but also include the low latency requirements for data transmission. For example, the transmission of image data needs to be completed within 1 ms. Due to the installation space limitation of industrial production lines, long-term uninterrupted operation, and cost sensitivity, it is a great challenge to the performance of the algorithm.

[0004] When the existing multi-class image tracking algorithms are applied to industrial application scenarios, if parallel computing is used, the information transmission of nodes will inevitably cause delays during program operation, and it cannot well meet the real-time and low-latency detection requirements in industrial applications. At the same time, considering the cost investment in industrial applications, there are often great limitations in the performance of devices, and in most cases, there are no conditions for parallel computing. However, the tracking methods of existing algorithms have obvious defects on serial machines: redundant computational complexity and large memory overhead. Therefore, when applying the contour tracking algorithm to industrial production lines, the existing algorithms, especially the contour tracking algorithms for multi-class images, are difficult to meet the requirements of online real-time and accurate detection. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a multi-class contour tracking method, device and medium for multi-class images, which can quickly and accurately track complex contour information with only one scan of multi-class images, can distinguish contour categories, can quickly and accurately perform multi-class image contour tracking, and is easy to be applied to low-computing-power devices in industrial production.

[0006] In a first aspect, the present invention provides a multi-class contour tracking method for multi-class images, including: a segmentation and assignment process, a starting point acquisition process, a contour point tracking process, and a termination execution process;

[0007] The segmentation and assignment process includes: performing image segmentation on the image to obtain multi-class images, dividing the pixel points of the multi-class images into background pixels and at least one nth-class pixel point, where n is an integer, and the pixels around the border of the multi-class images are always background pixels; assigning the first set value to the background pixels, and respectively assigning the (n + 1)th set value to the nth-class pixel points;

[0008] The starting point acquisition process includes: performing pixel-by-pixel scanning on the assigned multi-class images in the order from top to bottom and from left to right; when the starting point of the outer contour or the starting point of the inner contour is found by comparing the values of two adjacent pixel points on the left and right, the scanning is interrupted, and the found starting point of the outer contour or the starting point of the inner contour is used as the tracking starting point for contour tracking;

[0009] The contour point tracking process includes: detecting the pixel points in the specified direction around the starting point in the specified order, and taking the first found same-class pixel point e as the tracking termination point of the contour. If no pixel point e that meets the conditions is found, it means that the tracking starting point is an isolated point. After performing the operation of setting the pixel value of the tracking starting point to a negative class value, resume scanning from the right pixel point, and return to the starting point acquisition process;

[0010] After finding the tracking termination point, starting from the tracking starting point, based on the chain code, detect the 8 pixel points around the current contour point b one by one in the predefined order. Starting from the pixel point at the 0 position, detect the pixel points in sequence until the same-class pixel point of point b is detected, which means finding the next contour point; then take the next contour point as the current contour point b and continue to detect the pixel points in sequence. Set the pixel values of all contour points to negative class values, and modify the flag variable of the pixel points on the right side of the contour points that have been detected to the set value, so that they cannot become the tracking starting points of other contours; loop execution. When tracing from the tracking termination point to the tracking starting point, terminate the tracking of the current contour and obtain the complete contour, then resume scanning from the right pixel point of the contour starting point, return to the starting point acquisition process, and use the same method to find the tracking starting points of other contours and perform contour tracking;

[0011] The termination execution process includes: when scanning to the last pixel point of the multi-class images, end the scanning and obtain the contour tracking result.

[0012] Further, in the starting point acquisition process, comparing the values of two adjacent pixel points on the left and right to find the starting point of the outer contour or the starting point of the inner contour specifically includes:

[0013] Determine two consecutive pixel points p i,j and p i,j-1 or p i,j and p i,j+1 whether the values satisfy the following conditions:

[0014] Condition 1: T i,j ≠T i,j-1 and f i,j > 0;

[0015] Condition 2: T i,j ≠T i,j+1 and f i,j > 0;

[0016] Condition 3: T i,j ≠T i,j+1 and p i,j is not flag - point;

[0017] where T i,j is the category value of the current pixel point p i,j , which is a non - negative value and does not change; f i,j is the pixel value of p i,j ; flag - point means that the value of the flag variable is not the initial value;

[0018] When only Condition 1 is satisfied, the pixel point p i,j is the starting point of the outer contour; when Condition 2 or Condition 3 is satisfied, the pixel point p i,j is the starting point of the inner contour; when both Condition 1 and Condition 2 are satisfied, the pixel point p i,j is only used as the starting point of the outer contour.

[0019] Furthermore, in the process of obtaining the starting point, when finding the starting point of the inner contour or the outer contour, if the contour starting point is the starting point of the outer contour, set the p - code of this pixel point to 7, if the contour starting point is the starting point of the inner contour, set the p - code of this pixel point to 3, and the p - code is the current chain code, representing the direction from the previous contour point to the current contour point.

[0020] Furthermore, in the process of tracking the contour points, sequentially detect the pixel points to the right, lower - right, lower, lower - left, and left of this starting point, and take the first similar pixel point e found as the tracking termination point of this contour; when tracking from one contour point to the tracking starting point, determine whether this contour point is the tracking termination point. If it is not the tracking termination point, continue tracking the current contour. If it is the tracking termination point, terminate the tracking of the current contour.

[0021] Second aspect, the present invention provides a multi-class contour tracking device for multi-class images, including: a segmentation and assignment module, a starting point acquisition module, a contour point tracking module, and a termination execution module;

[0022] The segmentation and assignment module is used to perform image segmentation on the image to obtain multi-class images, divide the pixel points of the multi-class images into background pixels and at least one nth-class pixel point, where n is a real number, and the pixels around the border of the multi-class images are always background pixels; assign the background pixels with a first set value, and assign the nth-class pixel points with an (n + 1)th set value respectively;

[0023] The starting point acquisition module is used to perform pixel-by-pixel scanning on the multi-class images after assignment in the order from top to bottom and from left to right; when the starting point of the outer contour or the starting point of the inner contour is found by comparing the values of two adjacent left and right pixel points, the scanning is interrupted, and the found starting point of the outer contour or the starting point of the inner contour is used as the tracking starting point for contour tracking;

[0024] The contour point tracking module is used to detect the pixel points in the specified direction around the starting point in the specified order, and take the first same-class pixel point e found as the tracking termination point of the contour. If no pixel point e that meets the conditions is found, it means that the tracking starting point is an isolated point. After performing the operation of setting the pixel value of the tracking starting point to a negative class value, resume scanning from the right pixel point;

[0025] After the tracking termination point is found, starting from the tracking starting point, based on the chain code, 8 pixel points around the current contour point b are detected one by one in the predefined order. Starting from the pixel point at the 0 position, the pixel points are detected in sequence until the same-class pixel point of point b is detected, which means the next contour point is found; then the next contour point is used as the current contour point b to continue detecting the pixel points in sequence. Set the pixel values of all contour points to negative class values, and modify the flag variable of the pixel points on the right side of the contour points that have been detected to a set value, so that they cannot become the tracking starting points of other contours; loop execution. When the tracking starting point is traced from the tracking termination point, terminate the tracking of the current contour and obtain the complete contour, then resume scanning from the right pixel point of the contour starting point, return to the starting point acquisition module, and find the tracking starting points of other contours and perform contour tracking in the same way;

[0026] The termination execution module is used to end the scanning when the last pixel point of the multi-class images is scanned, and obtain the contour tracking result.

[0027] Further, in the starting point acquisition module, comparing the values of two adjacent left and right pixel points to find the starting point of the outer contour or the starting point of the inner contour specifically includes:

[0028] Judge two consecutive pixel points pi,j With p i,j-1 Or p i,j With p i,j+1 Does the value of satisfy the following conditions:

[0029] Condition 1: T i,j ≠T i,j-1 , and f i,j > 0;

[0030] Condition 2: T i,j ≠T i,j+1 , and f i,j > 0;

[0031] Condition 3: T i,j ≠T i,j+1 , and p i,j Is not flag - point;

[0032] Among them, T i,j Is the category value of the current pixel point p i,j , which is a non - negative value and will not change; f i,j Is the pixel value of p i,j ; flag - point indicates that the value of the flag variable is not the initial value;

[0033] When only Condition 1 is satisfied, the pixel point p i,j Is the starting point of the outer contour; when Condition 2 or Condition 3 is satisfied, the pixel point p i,j Is the starting point of the inner contour; when both Condition 1 and Condition 2 are satisfied, the pixel point p i,j Is only used as the starting point of the outer contour.

[0034] Furthermore, in the starting point acquisition module, when finding the starting point of the inner contour or the outer contour, if the contour starting point is the outer contour starting point, set the p - code of this pixel point to 7, if the contour starting point is the inner contour starting point, set the p - code of this pixel point to 3, and the p - code is the current chain code, indicating the direction from the previous contour point to the current contour point.

[0035] Furthermore, in the contour point tracking module, sequentially detect the pixel points to the right, right - lower, lower, left - lower, and left of this starting point, and take the first same - type pixel point e found as the tracking termination point of this contour; when tracing from a contour point to the tracking starting point, determine whether this contour point is the tracking termination point. If it is not the tracking termination point, continue the tracking of the current contour. If it is the tracking termination point, terminate the tracking of the current contour.

[0036] In a third aspect, the present invention provides a computer - readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0037] The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0038] 1. Only by scanning multiple types of images once, the information of the inner and outer contours of all target objects of different categories in the images can be obtained quickly and accurately. Therefore, the inclusion, overlap, and other relationships between different category connected regions will not be lost, which can ensure the accuracy of the contour information. At the same time, the time and space complexity of the algorithm are low, which is suitable for devices with low computing power and is convenient to be transplanted to existing low-power industrial devices.

[0039] 2. After a contour point is tracked, the pixel value of this point is set to a negative category value, so as to distinguish the tracked contour points and avoid tracking the same contour repeatedly.

[0040] 3. By using the flag variable marking method, it is ensured that this algorithm will not lose any contour, nor will it track the same contour repeatedly, and it can distinguish different categories.

[0041] The above description is only an overview of the technical solutions of the present invention. In order to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are given below. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be further described below with reference to the accompanying drawings in conjunction with the embodiments.

[0043] Figure 1 It is a schematic diagram of four-connectivity and eight-connectivity of the connectivity relationship in the embodiments of the present invention;

[0044] Figure 2 It is a pixel schematic diagram of multiple types of images in the embodiments of the present invention;

[0045] Figure 3 It is a schematic diagram of the original camera image / binary image / multiple types of images of the defect detection image of the cable;

[0046] Figure 4 It is a schematic diagram of the hole connected region and the inner contour in the embodiments of the present invention;

[0047] Figure 5 It is a schematic diagram of the chain code in the prior art;

[0048] Figure 6 It is a schematic diagram of 8 different detection sequences of contour tracking in the embodiments of the present invention;

[0049] Figure 7 For Figure 2 It is a result pixel schematic diagram after the outer contour of the 1-connected domain of the pixel schematic diagram is completely tracked;

[0050] Figure 8 It is a schematic flowchart of the method in the first embodiment of the present invention;

[0051] Figure 9 It is a schematic structural diagram of the device in the second embodiment of the present invention. Detailed implementation manners

[0052] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the accompanying drawings and specific implementation manners. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, and should not be construed as a limitation of the present invention.

[0054] The general idea of the technical solutions in the embodiments of the present invention is as follows:

[0055] In order to solve the problem that it is difficult for existing contour tracking algorithms to achieve online real-time and accurate detection of multiple types of images, the present invention proposes an efficient and accurate multi-class contour tracking method for multiple types of images. This method only needs to scan the original multiple types of images once to obtain the contour information of all categories. The multi-class contour tracking algorithm of the present invention has low time and space complexity, is suitable for devices with low computing power, and is convenient to be transplanted to existing low-power industrial devices.

[0056] Consider an image with a size of H×W, and the pixel point is denoted as pi ,j , where (i, j) is the coordinate of a pixel, representing the position of the pixel in the i-th row and j-th column of the image, and the position of the upper left corner of the image is (1, 1). f i,j represents the pixel value of the pixel point p i,j . There are two types of connectivity relationships involved in the present invention, namely four-connectivity and eight-connectivity, as shown in Figure 1 . Some other definitions are as follows.

[0057] Definition 1 (multiple types of images): Each pixel point in the image has its unique belonging category, denoted by T i,j as the category value of the pixel point p i,j , 0≤T i,j ≤255, and pixel points with the same category value are called homogeneous pixel points. Pixel points with a category value of 0 are called background pixels, and the rest of the pixel points are called target object pixels. In an initial multiple types of image, the pixel value of each pixel is equal to its category value. Figure 2A pixel schematic diagram of a multi-class image is described. Each square represents a pixel, and pixels with a pixel value of 0 represent the background. The remaining four represent four categories, denoted by 1, 2, 3, and 4 respectively. Contour tracking is to find the contours of all regions with the same number.

[0058] The multi-class image is the image that the present invention needs to directly perform contour tracking on, which is very different from the binary image. To better display the multi-class image, when it is necessary to show the multi-class image, different colors are assigned to pixels of different classes for distinction, which is called the multi-class image schematic diagram. Figure 3 A camera original image of a cable in industrial defect detection and its corresponding binary image and multi-class image schematic diagram are shown. Figure 3 (a) is the camera original image, Figure 3 (b) is Figure 3 (a)'s corresponding binary image, Figure 3 (c) is Figure 3 (a)'s multi-class image schematic diagram obtained after image segmentation, and each color represents a category.

[0059] Definition 2 (K-connected domain): Under the premise of four (eight) connectivity, all mutually connected pixels of the same class form a four (eight) connected K-connected domain, where the value of K is the common class value of the pixels that make up the connected domain, 0 ≤ K ≤ 255. In particular, if a non-zero pixel has no neighboring pixels of the same class, this pixel is called an isolated point and is regarded as an independent K-connected domain. Taking Figure 2 as an example, for the pixels with a class value of 1 in the figure, if the eight-connected method is used as the premise, they form a 1-connected domain; if the four-connected method is used as the premise, they form two 1-connected domains. Therefore, it is stipulated that the background pixels are always connected in a four-connected manner to form a background connected domain. For a given eight-connected non-zero K-connected domain S, the outermost pixels in the connected domain S are called outer contour pixels, and the set of all outer contour pixels in S constitutes the outer contour of S.

[0060] Definition 3 (enclosure): For two given connected domains S1 and S2, where S1 is an eight-connected K1-connected domain and S2 is a four-connected K2-connected domain, K1 ≠ K2, K1 ≠ 0. First, at least one pixel in S2 has a common side with a certain pixel in S1. Second, starting from any pixel in S2 and extending along the four-connected direction as a path to the border of the entire image, all paths will pass through pixels in S1, then it is called that S1 encloses S2.

[0061] Definition 4 (Hole Connected Region and Inner Contour): When an eight-connected non-zero K1-connected region S1 encloses a four-connected K2-connected region S2, S2 and all other four-connected connected regions S3, S4, ……, S n form a hole region H1. Then, H1 and all other four-connected connected regions S n+1 , ……, S m form a hole region H2, and so on. Eventually, a non-expandable hole region H t will definitely be obtained; otherwise, it will conflict with the enclosure relationship. Among them, the category values of all four-connected connected regions S i are all different from S1, i≠1. At this time, H t is called a hole connected region of S1, and the set of all pixel points in S1 that have a common edge with this hole connected region constitutes an inner contour of S1. Figure 4 shows Figure 2 all the hole connected regions and inner contours in the schematic diagram. The shaded part in the figure represents the hole connected region, and the pixel points with negative pixel values represent the inner contour pixel points. Figure 4 (a) and Figure 4 (b) show two different inner contours of 1-connected regions, Figure 4 (c) shows the inner contour of a 3-connected region.

[0062] According to the above definition, it can be seen that the contour pixel points form a closed contour in an eight-connected manner, and the background pixel points are not contour pixel points. The goal of the multi-class contour tracking algorithm proposed in the present invention is to quickly and accurately obtain the information of the inner and outer contours of all target objects of different classes in an image after a single scan of a multi-class image. By using this, more accurate information about defects can be obtained in industrial vision inspection.

[0063] The multi-class contour tracking algorithm proposed in the present invention mainly consists of three parts. The first part is how to determine the starting point of contour tracking. In the subsequent description of the present invention, the concept of a point is equivalent to the concept of a pixel. The second part is the tracking strategy, that is, how to track from the current contour point to the next contour point. The last part is to determine the tracking termination condition to end the current tracking and obtain a complete contour. The scanning order of the image is from top to bottom and from left to right, scanning pixel by pixel. When scanning to pixel p i,j and finding that it is the starting point of a certain contour, the scanning is interrupted, and then contour tracking starts. When the tracking meets the termination condition, the current tracking ends, and the scanning resumes from pixel point p i,j+1 . When the scanning reaches the lower right corner of the image, the algorithm terminates. Without loss of generality, the present invention assumes that the pixels around the border of the processed multi-class image are always background pixels.

[0064] By comparing the values of two adjacent pixels on the left and right, a new contour can be found. To avoid re-tracking the same contour, some special marking methods are usually used to distinguish the already tracked contour points. The present invention proposes a new marking method: when a contour point p x,y has been tracked, the pixel value of this point is set to a negative class value, that is, f x,y =-T x,y . At the same time, an additional bool-type flag variable is set for the contour point p x,y , with an initial value of 0. When the pixel point p x,y+1 on the right side of this contour point is detected during the tracking process, it means that this point cannot become the starting point of tracking other contours. Therefore, the value of the flag variable of this contour point p x,y is set to 1 for distinction. The contour point with the flag variable value of 1 is called a "flag-point". This marking method ensures that this algorithm will not lose any contour, will not re-track the same contour, and can distinguish different classes.

[0065] During the scanning process, if the values of two consecutive pixel points p i,j and p i,j-1 or p i,j and p i,j+1 meet one of the following change conditions, then the pixel point p i,j is the starting point of tracking a contour. Where f i,j is the pixel value of p i,j , and it will change between positive and negative values during the tracking process. T i,j is the class value of the current pixel point p i,j , which is a non-negative value and will not change.

[0066] Condition 1: T i,j ≠T i,j-1 , and f i,j >0; then p i,j is the starting point of the outer contour.

[0067] Condition 2: T i,j ≠T i,j+1 , and f i,j >0; then p i,j is the starting point of the inner contour.

[0068] Condition 3: T i,j ≠T i,j+1 , and p i,j is not a flag-point; then p i,j is the starting point of the inner contour.

[0069] When both Condition 1 and Condition 2 are satisfied, the pixel point p i,jOnly as the starting point of the outer contour.

[0070] During the process of tracking from the current contour point to the next contour point, the tracker detects pixel points one by one according to a predefined order. The tracking algorithm of the present invention is based on chain codes. For example, Figure 5 the direction codes shown are chain codes. Since each pixel has at most 8 neighbors, it is sufficient to use 3 bits to indicate the direction code of the next contour pixel.

[0071] The detection order adopted in the embodiments of the present invention is as shown in Figure 6 Each square represents a pixel, b is the current contour point, a is the previous contour point, and the current chain code p-code represents the direction from the previous contour point to the current contour point, denoted as a→b, and its value corresponds to the chain code shown in Figure 5 The numbers 0→6 (0→5) in Figure 6 represent the order in which the tracker detects pixel points. Each time a new contour point is tracked, according to the different p-codes, the corresponding order diagram is found. The tracker starts from the pixel point at position 0 and sequentially detects pixel points in the order of 0→6 (0→5) until a pixel point of the same type as point b is detected, which means the next contour point has been found. The p-code is then updated. In addition, it has been proven that the pixel point identifying the x position must not be a pixel point of the same type as contour point b. Otherwise, according to the counterclockwise detection order, for the previous contour point a, the current contour point should be the pixel point at the x position, rather than point b. Since there is no previous contour point for the contour starting point, it is stipulated that at the start of tracking the outer contour starting point and the inner contour starting point, the p-code is set to the default values 7 and 3 respectively to ensure the normal start of contour tracking.

[0072] The specific tracking strategy is as follows: Whenever a new contour pixel point b i,j is detected, the pixel value of this point is set to the negative class value, that is, f i,j =-T i,j , and then according to the value of the current chain code p-code, the detection is carried out according to the corresponding detection order in Figure 6 until the next contour point is found and the p-code is updated. During this process, if the pixel point p i,j+1 has been detected, the value of the flag variable of the current contour point b i,j is changed to 1. It should be noted that when the value of the p-code is 2 or 3, since the pixel point p i,j on the right side of the current contour point b i,j+1 cannot be the next contour point, so at this time, the value of the flag variable of the current contour point b i,j also needs to be changed to 1, indicating that its right side pixel point has been detected. This cycle continues until a complete contour is tracked.

[0073] The contour tracking process of the overall multi-class images is shown in the following embodiments.

[0074] Embodiment 1

[0075] This embodiment provides a multi-class contour tracking method for multi-class images. As Figure 7 shown, it includes: a segmentation and assignment process, a starting point acquisition process, a contour point tracking process, and a termination execution process;

[0076] The segmentation and assignment process includes: performing image segmentation on the image to obtain multi-class images, dividing the pixel points of the multi-class images into background pixels and at least one nth-class pixel point, where n is an integer, and the pixels around the border of the multi-class images are always background pixels (a circle of background pixels can be added around the multi-class images, or the pixels around the border can be directly set as background pixels); assigning the first set value to the background pixels, and respectively assigning the (n + 1)th set value to the nth-class pixel points (for example, Figure 2 if there are four types of pixel points other than the background pixels, the background pixels are assigned 0, and the other 4 types of pixel points are respectively assigned 1, 2, 3, and 4);

[0077] The starting point acquisition process includes: scanning the assigned multi-class images pixel by pixel in the order from top to bottom and from left to right (setting this scanning order is for the convenience of describing the orientation of each pixel point and its surrounding pixel points in the subsequent method, and should not be construed as a limitation of the present invention; the scanning order can also be changed to other orders, such as from bottom to top, from right to left, or any order combination that can ensure that the pixels are scanned one by one in sequence, but the corresponding point comparison method, positioning method, and the order of the chain code need to be adjusted accordingly); when the outer contour starting point or the inner contour starting point is found by comparing the values of two adjacent pixel points on the left and right, the scanning is interrupted, and the found outer contour starting point or inner contour starting point is used as the tracking starting point for contour tracking;

[0078] The contour point tracking process includes: detecting the pixel points in the specified direction around the starting point in the specified order, and taking the first same-class pixel point e found as the tracking termination point of the contour. If no pixel point e that meets the conditions is found, it means that the tracking starting point is an isolated point. After performing the operation of setting the pixel value of the tracking starting point to a negative class value, the scanning is resumed from the right pixel point, and the starting point acquisition process is returned;

[0079] When the tracking termination point is found, starting from the tracking start point, 8 pixel points around the current contour point b are detected one by one according to the predefined order based on the chain code. Starting from the pixel point at position 0, the pixel points are detected in sequence until a pixel point of the same type as point b is detected, which means the next contour point is found; then the next contour point is used as the current contour point b to continue detecting the pixel points in sequence. Set the pixel values of all contour points to negative class values, and modify the flag variable of the detected contour point of the pixel point on the right side of the contour point to a set value (if there are other pixel points detected as pixel points of the same type before the pixel point on the right side, the status of the pixel point on the right side is undetected), making it impossible to be the tracking start point of other contours (it can only not be used as the tracking start point, but can still be used as a contour point); execute in a loop. When tracing from the tracking termination point to the tracking start point, terminate the tracking of the current contour and obtain the complete contour, then resume scanning from the pixel point on the right side of the contour start point, return to the start point acquisition process, and find the tracking start points of other contours and perform contour tracking in the same way;

[0080] The termination execution process includes: when scanning to the last pixel point of multiple types of images, end the scanning and obtain the contour tracking result.

[0081] In a possible implementation manner, in the start point acquisition process, compare the values of two adjacent pixel points on the left and right to find the outer contour start point or the inner contour start point. Specifically, it includes:

[0082] Judge whether the values of two consecutive pixel points p i,j and p i,j-1 or p i,j and p i,j+1 meet the following conditions:

[0083] Condition 1: T i,j ≠T i,j-1 , and f i,j >0;

[0084] Condition 2: T i,j ≠T i,j+1 , and f i,j >0;

[0085] Condition 3: T i,j ≠T i,j+1 , and p i,j is not flag-point;

[0086] Among them, T i,j is the class value of the current pixel point p i,j , which is a non-negative value and will not change; f i,j is the pixel value of p i,j ; flag-point means that the value of the flag variable is not the initial value;

[0087] When only condition 1 is satisfied, pixel p i,j is the starting point of the outer contour; when condition 2 or condition 3 is satisfied, pixel p i,j is the starting point of the inner contour; when both condition 1 and condition 2 are satisfied, pixel p i,j is only used as the starting point of the outer contour.

[0088] In a possible implementation, during the starting point acquisition process, when finding the starting point of the inner contour or the outer contour, if the contour starting point is the starting point of the outer contour, set the p-code of this pixel to 7, and if the contour starting point is the starting point of the inner contour, set the p-code of this pixel to 3. The p-code is the current chain code, indicating the direction from the previous contour point to the current contour point.

[0089] In a possible implementation, during the contour point tracking process, sequentially detect the pixels to the right, lower right, below, lower left, and left of this starting point, and take the first similar pixel e found as the tracking termination point of this contour; when tracing from a contour point to the tracking starting point, determine whether this contour point is the tracking termination point. If it is not the tracking termination point, continue the tracking of the current contour. If it is the tracking termination point, terminate the tracking of the current contour.

[0090] In a specific embodiment, when scanning and finding the tracking starting point p i,j of the contour, pause the scanning, and sequentially detect the pixels to the right, lower right, below, lower left, and left of this starting point, that is, in the order of Figure 5 where the chain code directions are 0, 7, 6, 5, 4, sequentially detect the pixels around p i,j . The first similar pixel e of p i,j found is called the tracking termination point of this contour. After recording the tracking termination point of the contour, the tracker starts to perform one-by-one contour point tracking from the starting point until the tracker traces from the termination point e back to the starting point p i,j . Then, terminate the tracking of the current contour to obtain a complete contour, and then resume scanning from the right pixel p i,j+1 . If no pixel e that meets the conditions is found, it means that this tracking starting point is an isolated point. After performing the operation of f i,j =-T i,j , resume scanning from the right pixel p i,j+1 .

[0091] Taking the pixel schematic diagram shown in Figure 2 as an example, it can be determined that pixel p 2,4 is the tracking starting point of an outer contour, and pixel e 2,5is its corresponding tracking termination point. Applying the contour tracking method of the present invention, after the first contour is completely tracked, the result is as Figure 8 shown. In Figure 8 , the contour tracking starts from p 2,4 and finally returns to p 2,5 from e 2,4 , obtaining a complete contour, that is, the outer contour of the 1-connected domain. Figure 8 The pixel values of all the tracked contour points in

[0092] become negative values. The contour points circled by circles indicate that the contour points are flag-points.

[0093] Embodiment 2

[0094] In this embodiment, a multi-class contour tracking device for multi-class images is provided, as Figure 9 shown, including: a segmentation and assignment module, a starting point acquisition module, a contour point tracking module, and a termination execution module;

[0095] The segmentation and assignment module is used to perform image segmentation on the image to obtain multi-class images, divide the pixel points of the multi-class images into background pixels and at least one nth-class pixel point, where n is a real number, and the pixels around the border of the multi-class images are always background pixels; assign the background pixels to the first set value and assign the nth-class pixel points to the (n + 1)th set value respectively;

[0096] The starting point acquisition module is used to perform pixel-by-pixel scanning on the assigned multi-class images in the order from top to bottom and from left to right; when the starting point of the outer contour or the starting point of the inner contour is found by comparing the values of two adjacent pixels on the left and right, the scanning is interrupted, and the found starting point of the outer contour or the starting point of the inner contour is used as the tracking starting point for contour tracking;

[0097] The contour point tracking module is used to detect the pixel points in the specified direction around the starting point in the specified order, and take the first same-class pixel point e found as the tracking termination point of the contour. If no pixel point e that meets the conditions is found, it means that the tracking starting point is an isolated point. After performing the operation of setting the pixel value of the tracking starting point to a negative class value, the scanning resumes from the right pixel point;

[0098] After finding the tracking termination point, starting from the tracking starting point, based on the chain code, 8 pixel points around the current contour point b are detected one by one according to the predefined order. Starting from the pixel point at position 0, the pixel points are detected in sequence until a pixel point of the same type as point b is detected, which means finding the next contour point; then the next contour point is used as the current contour point b to continue detecting pixel points in sequence. The pixel values of all contour points are set to negative class values, and the flag variable of the detected contour point of the pixel point on the right side of the contour point is modified to a set value, making it impossible to become the tracking starting point of other contours; loop execution. When tracing from the tracking termination point to the tracking starting point, terminate the tracking of the current contour and obtain the complete contour, then resume scanning from the pixel point on the right side of the contour starting point, return to the starting point acquisition module, and use the same method to find the tracking starting points of other contours and perform contour tracking;

[0099] The termination execution module is used to end the scan and obtain the contour tracking result when the last pixel point of multiple types of images is scanned.

[0100] In a possible implementation manner, in the starting point acquisition module, the outer contour starting point or the inner contour starting point is found by comparing the values of two adjacent pixel points on the left and right, specifically including:

[0101] Judge whether the values of two consecutive pixel points p i,j and p i,j-1 or p i,j and p i,j+1 meet the following conditions:

[0102] Condition 1: T i,j ≠T i,j-1 , and f i,j >0;

[0103] Condition 2: T i,j ≠T i,j+1 , and f i,j >0;

[0104] Condition 3: T i,j ≠T i,j+1 , and p i,j is not flag-point;

[0105] Among them, T i,j is the class value of the current pixel point p i,j , which is a non-negative value and will not change; f i,j is the pixel value of p i,j ; flag-point means that the value of the flag variable is not the initial value;

[0106] When only condition 1 is met, the pixel point p i,jis the starting point of the outer contour; when condition 2 or condition 3 is satisfied, the pixel point p i,j is the starting point of the inner contour; when both condition 1 and condition 2 are satisfied, the pixel point p i,j is only used as the starting point of the outer contour.

[0107] In a possible implementation, in the starting point acquisition module, when finding the starting point of the inner contour or the starting point of the outer contour, if the contour starting point is the starting point of the outer contour, the p-code of this pixel point is set to 7, and if the contour starting point is the starting point of the inner contour, the p-code of this pixel point is set to 3. The p-code is the current chain code, indicating the direction from the previous contour point to the current contour point.

[0108] In a possible implementation, in the contour point tracking module, the pixel points to the right, lower right, below, lower left, and left of this starting point are sequentially detected, and the first same-kind pixel point e found is used as the tracking termination point of this contour; when tracing from one contour point to the tracking starting point, it is judged whether the contour point is the tracking termination point. If it is not the tracking termination point, the tracking of the current contour continues. If it is the tracking termination point, the tracking of the current contour is terminated.

[0109] Since the device introduced in the second embodiment of the present invention is the device adopted for implementing the method of the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of this device, so it will not be elaborated here. Any device adopted for the method of the first embodiment of the present invention belongs to the scope protected by the present invention.

[0110] Based on the same inventive concept, this application provides a storage medium corresponding to the first embodiment, as detailed in the third embodiment.

[0111] Embodiment Three

[0112] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, any implementation manner in the first embodiment can be realized.

[0113] Since the computer-readable storage medium introduced in this embodiment is the computer-readable storage medium adopted for implementing the method in the first embodiment of this application, based on the method introduced in the first embodiment of this application, those skilled in the art can understand the specific implementation manner of the computer-readable storage medium in this embodiment and its various variations. Therefore, how this computer-readable storage medium realizes the method in the embodiments of this application will not be introduced in detail here. As long as the computer-readable storage medium adopted by those skilled in the art for implementing the method in the embodiments of this application belongs to the scope protected by this application.

[0114] The present invention focuses on the field of product vision inspection in industrial intelligent manufacturing, which has its own particularities: high real-time requirements, high operating stability requirements, high accuracy requirements, low latency, low device power consumption requirements, and cost sensitivity. The present invention proposes a new multi-class contour tracking algorithm that can directly perform contour tracking on multi-class images. Only by scanning the image once can the contours of all classes be accurately obtained.

[0115] The multi-class contour tracking algorithm of the present invention is also applicable when processing binary images, and even has better efficiency than some traditional single-class tracking algorithms. When processing multi-class images, the efficiency is significantly improved. This algorithm can achieve an operating speed in the single-digit millisecond level when processing small-size multi-class images, and the operating speed can also be maintained below 50 milliseconds when processing larger-size multi-class images. And this is all under the premise that the code has not been optimized. Therefore, it can be expected that this algorithm can meet the real-time requirements in industrial applications after optimization. In addition, the multi-class algorithm of the present invention has low time and space complexity, is suitable for devices with low computing power, and is convenient to be transplanted to existing low-power industrial devices. These characteristics well meet the requirements for algorithm performance in industrial vision inspection applications.

[0116] The embodiment of the present invention only needs to scan the multi-class image once to quickly and accurately obtain the information of the inner and outer contours of all different-class target objects in the image, so that the inclusion, overlap and other relationships between different-class connected regions will not be lost, which can ensure the accuracy of the contour information. At the same time, the algorithm has low time and space complexity, is suitable for devices with low computing power, and is convenient to be transplanted to existing low-power industrial devices; when a contour point is tracked, the pixel value of this point is set to a negative class value, so as to distinguish the tracked contour points and avoid repeated tracking of the same contour; the marking method of the flag variable ensures that this algorithm will not lose any contour, will not repeat tracking the same contour, and can distinguish different classes.

[0117] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0119] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0121] Although the specific embodiments of the present invention have been described above, those skilled in the art of this technology should understand that the specific embodiments we described are illustrative rather than used to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should all be covered by the scope protected by the claims of the present invention.

Claims

1. A multi-class contour tracking method for multi-class images, characterized in that, Including: A segmentation assignment process, a starting point acquisition process, a contour point tracking process, and a termination execution process; The segmentation assignment process includes: performing image segmentation on an image to obtain multiple types of images, dividing the pixel points of the multiple types of images into background pixels and at least one type-n pixel point, where n is an integer, and the pixels around the border of the multiple types of images are always background pixels; assigning the background pixels to a first set value, and respectively assigning the type-n pixel points to a (n + 1)-th set value; The starting point acquisition process includes: performing pixel-by-pixel scanning on the multiple types of images after assignment in the order from top to bottom and from left to right; when finding an outer contour starting point or an inner contour starting point by comparing the values of two adjacent left and right pixel points, the scanning is interrupted, and the found outer contour starting point or inner contour starting point is used as a tracking starting point for contour tracking; The contour point tracking process includes: detecting the pixel points in the specified directions around the starting point in the specified order, and taking the first same-type pixel point e found as the tracking termination point of the contour. If no pixel point e that meets the conditions is found, it means that the tracking starting point is an isolated point. After performing the operation of setting the pixel value of the tracking starting point to a negative class value, the scanning is resumed from the right pixel point, and the starting point acquisition process is returned; After finding the tracking termination point, starting from the tracking starting point, based on the chain code, 8 pixel points around the current contour point b are detected one by one in the predefined order. Starting from the pixel point at the 0 position, the pixel points are detected in sequence until the same-type pixel point of point b is detected, which means that the next contour point is found; then the next contour point is used as the current contour point b to continue detecting the pixel points in sequence. The pixel values of all contour points are set to negative class values, and the flag variable of the contour point whose pixel point on the right has been detected is modified to a set value so that it cannot become the tracking starting point of other contours; this is looped. When the tracking starting point is traced from the tracking termination point, the tracking of the current contour is terminated and a complete contour is obtained. Then the scanning is resumed from the right pixel point of the contour starting point, and the starting point acquisition process is returned. The tracking starting points of other contours are found and contour tracking is performed in the same way; The termination execution process includes: when scanning to the last pixel point of the multiple types of images, the scanning ends, and the contour tracking result is obtained.

2. The method according to claim 1, wherein: In the starting point acquisition process, finding the outer contour starting point or the inner contour starting point by comparing the values of two adjacent left and right pixel points specifically includes: Determine two consecutive pixel points p i,j and p i,j-1 or p i,j and p i,j+1 whether the values satisfy the following conditions: Condition 1: T i,j ≠ T i,j-1 and f i,j > 0; Condition 2: T i,j ≠ T i,j+1 , and f i,j > 0; Condition 3: T i,j ≠ T i,j+1 and p i,j is not a flag-point; Among them, T i,j is the category value of the current pixel point p i,j , which is a non-negative value and will not change; f i,j is the pixel value of p i,j ; flag-point indicates that the value of the flag variable is not the initial value; When only condition 1 is satisfied, pixel point p i,j is the starting point of the outer contour; when condition 2 or condition 3 is satisfied, pixel point p i,j is the starting point of the inner contour; when both condition 1 and condition 2 are satisfied, pixel point p i,j is only used as the starting point of the outer contour.

3. The method according to claim 1 or 2, characterized in that: In the starting point acquisition process, when finding the inner contour starting point or the outer contour starting point, if the contour starting point is an outer contour starting point, the p-code of the pixel point is set to 7, and if the contour starting point is an inner contour starting point, the p-code of the pixel point is set to 3. The p-code is the current chain code, indicating the direction from the previous contour point to the current contour point.

4. The method according to claim 1, wherein: In the contour point tracking process, the pixel points on the right, lower right, lower, lower left, and left of the starting point are detected in sequence, and the first same-type pixel point e found is used as the tracking termination point of the contour; When tracing from a contour point to the tracking starting point, determine whether the contour point is a tracking termination point. If it is not a tracking termination point, continue the tracking of the current contour. If it is a tracking termination point, terminate the tracking of the current contour.

5. A multi-class contour tracking device for multi-class images, characterized in that, Including: a segmentation and assignment module, a starting point acquisition module, a contour point tracking module, and a termination execution module; The segmentation and assignment module is used to perform image segmentation on an image to obtain multiple types of images, divide the pixel points of the multiple types of images into background pixels and at least one type-n pixel point, where n is a real number, and the pixels around the border of the multiple types of images are always background pixels; assign the first set value to the background pixels and assign the (n + 1)-th set value to the type-n pixel points respectively; The starting point acquisition module is used to perform pixel-by-pixel scanning on the assigned multiple types of images in the order from top to bottom and from left to right; when finding an outer contour starting point or an inner contour starting point by comparing the values of two adjacent left and right pixel points, the scanning is interrupted, and the found outer contour starting point or inner contour starting point is used as the tracking starting point for contour tracking; The contour point tracking module is used to detect the pixel points in the specified direction around the starting point in the specified order, and take the first same-type pixel point e found as the tracking termination point of the contour. If no pixel point e that meets the conditions is found, it means that the tracking starting point is an isolated point. After performing the operation of setting the pixel value of the tracking starting point to a negative class value, resume scanning from the right pixel point; After finding the tracking termination point, based on the chain code, detect the 8 pixel points around the current contour point b one by one from the tracking starting point in the predefined order. Starting from the pixel point at the 0 position, detect the pixel points in sequence until a same-type pixel point of point b is detected, which means finding the next contour point; then take the next contour point as the current contour point b and continue to detect the pixel points in sequence. Set the pixel values of all contour points to negative class values, and modify the flag variable of the detected contour points of the pixel points on the right side of the contour points so that they cannot become the tracking starting points of other contours; execute in a loop. When tracing from the tracking termination point to the tracking starting point, terminate the tracking of the current contour and obtain a complete contour, then resume scanning from the right pixel point of the contour starting point, return to the starting point acquisition module, and find the tracking starting points of other contours and perform contour tracking in the same way; The termination execution module is used to end the scanning and obtain the contour tracking result when scanning to the last pixel point of the multiple types of images.

6. The device according to claim 5, wherein: In the starting point acquisition module, finding the outer contour starting point or the inner contour starting point by comparing the values of two adjacent left and right pixel points specifically includes: Determine two consecutive pixel points p i,j and p i,j-1 or p i,j and p i,j+1 whether the values satisfy the following conditions: Condition 1: T i,j ≠ T i,j-1 and f i,j > 0; Condition 2: T i,j ≠ T i,j+1 , and f i,j > 0; Condition 3: T i,j ≠ T i,j+1 and p i,j is not a flag-point; Among them, T i,j is the category value of the current pixel point p i,j , which is a non - negative value and will not change; f i,j is the pixel value of p i,j ; flag - point indicates that the value of the flag variable is not the initial value; When only condition 1 is satisfied, pixel p i,j is the starting point of the outer contour; when condition 2 or condition 3 is satisfied, pixel p i,j is the starting point of the inner contour; when both condition 1 and condition 2 are satisfied, pixel p i,j is only used as the starting point of the outer contour.

7. The device according to claim 5 or 6, characterized in that: In the starting point acquisition module, when finding the inner contour starting point or the outer contour starting point, if the contour starting point is an outer contour starting point, set the p-code of the pixel point to 7. If the contour starting point is an inner contour starting point, set the p-code of the pixel point to 3. The p-code is the current chain code, indicating the direction from the previous contour point to the current contour point.

8. The device according to claim 5, characterized in that In the contour point tracking module, the pixel points to the right, lower right, below, lower left, and left of the starting point are sequentially detected, and the first homogeneous pixel point e found is used as the tracking termination point of the contour; When a contour point is traced back to the tracking starting point, it is determined whether the contour point is a tracking termination point. If it is not a tracking termination point, the tracking of the current contour continues; if it is a tracking termination point, the tracking of the current contour is terminated.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images