An image edge detection and tracking method

Abstract

This invention relates to an edge detection and tracking method. The edge detection stage includes color grayscale conversion, Gaussian filtering, generation of gradient magnitude matrix, generation of gradient magnitude histogram and calculation of threshold. The edge tracking stage includes initial point selection, edge concatenation, removal of redundant points, and edge verification. The input to this invention is a color photograph image, and the output is a set of contour line information contained in the image. It is particularly suitable for real-time algorithms in embedded hardware development and can be applied to practical scenarios such as educational robots, pattern recognition, and autonomous driving.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and image signal processing, specifically involving edge detection and tracking technology. Background Technology

[0002] The present invention addresses the scenario of digital image processing on low-cost embedded hardware. The technical process involved is divided into two main stages: edge detection and edge tracing / tracking (also known as boundary / contour tracing when the edge is a closed curve).

[0003] Edge detection and edge tracking can extract object contour information from the original image. Compared to the original input pixel matrix, this contour information is more structured, occupies less storage space, and is a versatile image content feature representation. Therefore, this method can be used for image preprocessing in many application-level image analysis, recognition, and understanding algorithms. However, most methods are not used in real-time scenarios or require high-performance hardware. This invention aims to expand the application scope of the algorithm and improve real-time performance while maintaining output accuracy as much as possible. Therefore, minimizing the computational storage and time resources consumed in the computation process is crucial.

[0004] The task involved in edge detection is to classify pixels in an image into edge points representing edges, or other non-edge pixels. The basic principle is that after converting the image to grayscale, areas where the brightness signal changes abruptly often correspond to the edges of objects represented in the image.

[0005] Using this principle, a naive algorithm for image edge detection can be constructed:

[0006] Using a specific pixel as the center, an orthogonal coordinate system is selected. First-order difference operations are performed on the brightness values ​​of adjacent pixels in two orthogonal directions to obtain a discrete gradient vector of image brightness. The L2 norm of this vector (…) The larger the norm (also known as the Euclidean norm), the more drastic the change in brightness signal near that pixel, and this is used to determine whether the center pixel is an edge pixel.

[0007] After completing the image edge detection described above, the gradient information of each pixel in the initial input image will be obtained. For two-dimensional images, gradient information can be represented in various ways, such as the horizontal / vertical components in a Cartesian coordinate system, or the magnitude / angle components in polar coordinates.

[0008] The obtained gradient information can then be used for various preprocessing steps, such as "non-maximum suppression": setting the gradient values ​​of pixels with relatively small gradient values ​​around pixels with large gradient magnitudes to zero, in order to "refine" the edges and avoid detecting edges that are too wide or whose "ridge" positions are difficult to determine.

[0009] The task involved in edge tracking is to select an initial point based on the gradient information obtained above, using a specific strategy. Starting from this point, and using information from other edge points in the neighborhood through a specific selection metric, the edge pixels are connected in the direction given by the selection metric to outline a continuous contour curve and recover the actual edges of objects in the original image as accurately as possible. During this process, the gradient values ​​of pixels identified as edge points are set to zero to avoid repeated tracking. Simultaneously, pixels near these edge-identified pixels also need to be set to zero using a specific strategy to prevent the mistracking of multiple parallel edges for a wide single edge.

[0010] Methods based on the above principles are generally categorized as edge tracking methods based on gradient information. However, simply applying the gradient principle has significant limitations.

[0011] For example, due to the limitations of various chemical photosensitive materials (such as film) or electronic photosensitive sensors (such as CCD and CMOS), images acquired from the real world through methods such as shooting and scanning contain random and irregular noise. In addition, some lossy compression algorithms (such as the discrete cosine transform used in the JPEG format) also introduce compression noise, which manifests as high-frequency noise in the discretized image pixel matrix.

[0012] Performing brightness value difference operations near high-frequency noise points can also yield larger result values. However, because the naive method relies solely on the gradient magnitude to determine whether a pixel is an edge pixel, it is prone to misdetecting high-frequency noise points that are not edge points as edge points. In other words, the naive gradient method must be paired with complementary techniques that reduce high-frequency noise interference to truly meet real-world application requirements.

[0013] The Canny edge detection algorithm is a relatively mature and high-performance method based on this principle. However, it has high requirements for storage and computing resources, and a threshold that needs to be manually determined as a parameter is required during algorithm execution. If the threshold is not selected appropriately, there may be too many false edge points or the edges may not be detected at all.

[0014] Currently, in academia and industry, besides the various methods developed based on local brightness change rates mentioned above, there are also methods constructed from other theories such as level sets, wavelet transforms, machine learning (including support vector machines), neural networks, deep convolutional neural networks, and mathematical morphology. A common characteristic of level set, wavelet transform, and mathematical morphology methods is their high demand for computational or storage resources.

[0015] Machine learning and artificial intelligence methods can reduce the amount of computation when actually deploying hardware through an offline training-online prediction process and by setting up dedicated chips in the hardware. However, the overall deployment cost is still much higher than that of traditional low-power chips and microcontrollers. Moreover, their working principle lacks an intuitive theoretical explanation. When encountering scenarios where the algorithm output is poor, it is necessary to spend a lot of time and effort to re-prepare the dataset for the corresponding scenario. Furthermore, since there are usually many parameters to adjust, a large amount of offline training computation is also required.

[0016] In summary, for the scenarios targeted by this invention, basing the invention on the gradient principle remains a more advantageous choice. The significant features of this invention lie in how to select the starting pixel for edge tracking, the selection criteria for including or excluding pixels during the process of concatenating pixel sets into a curve, and the methods for avoiding floating-point operations, division, and other complex calculations in the computational steps.

[0017] The beneficial effects of this invention are: it proposes a complete set of edge recognition and tracking methods that are intuitive, easy to interpret, low-cost, have low hardware performance requirements, and offer good real-time performance. This significantly lowers the technical threshold for edge extraction and tracking, facilitating the expansion of practical application scenarios. While reducing costs, the detection accuracy it provides can meet the requirements of most educational robots, service robots, and autonomous vehicles in their respective application scenarios. When deployed on high-performance hardware, its real-time performance is also superior to existing algorithms, providing higher detection speeds. Summary of the Invention

[0018] To address the problem that existing technologies and theories are difficult to deploy on low-power embedded electronic chips and microcontrollers in practical production and daily life applications, or that they cannot meet real-time requirements on general conventional hardware, this invention provides an image edge detection and tracking algorithm with low computational power requirements and small storage capacity requirements.

[0019] (I) Technical Solution

[0020] This invention accepts a color photograph image as input and outputs a set of contour information contained in the image. Each contour corresponds to information such as the coordinates of each pixel on the contour within the image and whether the contour is closed. The technical solution includes two stages: edge detection and edge tracking.

[0021] The specific steps in the edge detection process include:

[0022] a) Input a color image by streaming or reading a file, and use the color-to-grayscale conversion formula to calculate the grayscale value corresponding to each color pixel in the image to obtain a grayscale image;

[0023] b) Apply Gaussian filtering to the obtained grayscale image to weaken or even eliminate the influence of high-frequency noise in the image and obtain a smooth grayscale image;

[0024] c) For the smoothed grayscale image obtained after Gaussian filtering, calculate the Cartesian coordinate components of the first-order gradient around each pixel and calculate their magnitudes. It stores only the amplitude. After calculating for all pixels, it processes the amplitude through multiplication to bring it into the range of legal grayscale values ​​under the quantization bit depth selected by the user, and generates a gradient amplitude matrix.

[0025] d) Generate a gradient magnitude histogram based on the gradient magnitude matrix, and calculate the edge gradient magnitude threshold based on the proportional threshold of pixels representing the edge;

[0026] The edge tracing process tracks each edge individually. The specific tracing steps for each edge are as follows:

[0027] e) Initial point selection: Starting from the top left corner of the gradient magnitude matrix generated in the edge detection stage, traverse to the right and down. Points with gradient magnitudes greater than the edge gradient value threshold are taken as the initial points of the edge and become the "front edge points" of the edge.

[0028] f) Edge “concatenation”: Starting from the “front edge point”, select the pixel with the largest gradient magnitude greater than the edge gradient value threshold in the neighborhood of the current “front edge point” as the next round of “front edge point”. At the same time, the next round of “front edge point” is “concatenated” to the currently tracked edge.

[0029] g) "Sweep away" redundant points: adopt the "wingspan" pixel zeroing strategy, set the gradient magnitude corresponding to the point that needs to be "sweeped away" to zero, and decrement the count of the corresponding gradient magnitude in the corresponding interval of the histogram by 1, and increment the number of pixels contained in the current edge by 1.

[0030] h) The “chaining” and “sweeping” operations are repeated until no point with a gradient magnitude greater than the edge gradient value threshold can be selected in the neighborhood, at which point the edge ends.

[0031] i) Edge verification: Based on the conditions that an edge should meet, decide whether to include the currently detected edge in the output or exclude it from the output;

[0032] j) If the traversal has reached the lower right corner of the gradient grayscale image matrix, then the tracing ends.

[0033] Preferably, the color-to-grayscale conversion in step a) of the edge detection process uses the formula... Perform the conversion.

[0034] Preferably, in the edge detection step d), the edge gradient magnitude threshold is the left boundary value of the corresponding brightness interval slot whose cumulative sum of frequencies calculated from the low brightness interval slot in the histogram exceeds [total number of non-zero gradient magnitude points × (100% - user-selected high confidence ratio threshold)].

[0035] Preferably, in step g) of the edge tracking process, the points that need to be "sweeped" in the "wingspan" pixel zeroing strategy are two neighboring pixels that form a 135° angle between the current "leading point" and the next round "leading point".

[0036] Preferably, in step i) of the edge tracking process, the condition that the edge should meet is: calculate whether the total number of pixels contained in the currently tracked edge is greater than or equal to the set minimum number of pixels contained in the edge; if it meets the condition, the current edge is valid and the 8 connected points around the current point are set to zero; if it does not meet the condition, the current edge is invalid and the information of the current edge is cleared from memory.

[0037] (ii) Beneficial effects

[0038] 1. This invention only needs to utilize the magnitude information of the gradient, without storing the angle information of the gradient or the component values ​​in the horizontal / vertical direction, thus saving the storage space required in the edge extraction process;

[0039] 2. This invention introduces a histogram-based edge tracking initial point selection strategy, which, compared with other edge tracking methods based on gradient information, can eliminate irrelevant pixels at an earlier stage of the algorithm.

[0040] 3. In the process of implementing the present invention, the proportion threshold of the pixels representing the edge can be adjusted according to the specific output accuracy requirements to reduce the instruction execution cycle required for the algorithm to run (i.e., faster single-frame analysis speed).

[0041] 4. In the process of implementing the present invention, different candidate neighborhood ranges can be switched, such as 4-connection / 8-connection mode. In the 4-connection mode, by reducing the candidate domain, although pixels that are more in line with the tracking conditions may be missed, the instruction cycle required for program operation can be reduced (that is, faster single-frame analysis speed can be achieved).

[0042] 5. This method has lower hardware performance requirements when deployed on embedded or low-power devices. Even when the hardware lacks floating-point or division capabilities, it can complete edge tracking of a single frame image in fewer instruction cycles. Furthermore, for practical applications, the information content and quality of the output are comparable to similar algorithms, meeting the needs of real-world production and daily life. When deployed on high-performance hardware, its real-time performance is superior to existing algorithms, providing higher detection speeds. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the data representation format of the "color image" described in this invention in a computer, wherein (1a) shows the definition of the coordinates and coordinate growth direction of the color image, as well as the indexing method of the pixels; (1b) shows two possible storage methods of a single pixel in a computer, the top row shows the storage method in which each pixel occupies 24 bits (3 bytes) of storage space, and the bottom row shows the storage method in which each pixel occupies 16 bits (2 bytes) of storage space.

[0044] Figure 2 This is a schematic diagram of the overall process and input / output structure of the present invention.

[0045] Figure 3 The diagram shows the candidate range of the neighborhood, where (3a) shows the 4 connected points in the "4 connected" mode and (3b) shows the 8 connected points in the "8 connected" mode.

[0046] Figure 4 A schematic diagram of a "wingspan" type pixel zeroing strategy.

[0047] Figure 5 The image is the input image for the exemplary embodiment of the present invention, "Edge Recognition-Based Classification of Preschool Cards".

[0048] Figure 6 The output after edge tracking is completed for the implementation demonstration (after cropping and inversion, the original desktop part of the photo that was not output is removed, and the gradient magnitude is larger in the darker and darker areas in the image).

[0049] Figure 7This is an example of a card design that can be successfully distinguished by this invention. In the figure, compared with the reference card, the other cards all have different extracted edge features. The differences shown by the cards from left to right are: different number of edges of the main pattern, different single and double line styles of the edge frame, and different closure of the edge frame. Detailed Implementation

[0050] The objects and functions of the present invention, as well as the methods for achieving these objects and functions, are explained below with reference to the accompanying drawings and exemplary embodiments. However, the present invention is not limited to the exemplary embodiments disclosed below; it can be implemented in various forms. A more detailed description of specific embodiments of the invention can be made with reference to the accompanying drawings showing the outputs of the calculation process involved in this exemplary embodiment.

[0051] This embodiment utilizes the C programming language for edge extraction and tracking, and can be deployed on various hardware, including low-power embedded chips. An exemplary embodiment uses children's flashcards as an example. These flashcards can be printed on various flat media such as ordinary paper or plastic, and can be printed in black and white or in color, but the size should be large enough to be clearly captured by a camera. The flashcards include a main image and a border.

[0052] The image obtained from photographing the card is represented in computer memory with each pixel occupying multiple independent channels. This can be, but is not limited to, a three-channel format (red, green, blue); or a three-channel format (hue, saturation, brightness). The quantization bit depth can also be selected based on actual needs during image storage, generally using one of the following two methods (e.g., ...). Figure 1 (as shown)

[0053] (1) When high image quality is required, a quantization bit depth of 8 bits (1 byte) is generally used. Each channel on a single pixel has 256 levels of distinction, and the corresponding signal value is generally from 0 to 255. In this case, a pixel needs to occupy 3 bytes (24 bits) of storage space, which can represent more than 16.77 million different colors;

[0054] (2) On embedded devices, the RGB565 format can be used, using 2 bytes (16 bits) to represent colors. The R and B channels each have 32 levels of distinction, and the G channel has 64 bits of distinction, which can represent a total of 65,536 different colors.

[0055] The goal of this algorithm embodiment is to classify children's flashcards. The input is a color photograph of a children's flashcard, such as... Figure 5 As shown. The specific process includes the following steps:

[0056] (1). The image is stored in the cache, occupying a total number of bytes = (image height (number of rows) × image width (number of columns) × number of bytes per pixel). Since the total number of bytes determines the storage space occupied and the overall time complexity of the operation, for images taken at ultra-high resolution, they should be downsampled to a suitable resolution before being input into this algorithm;

[0057] (2). Convert the color image to grayscale. Let R represent the red channel value, G represent the green channel value, B represent the blue channel value, and I represent the brightness grayscale value. , that is Using the left shift of integer data types ( Equivalent to multiplying by 2), right shift ( This is equivalent to dividing by 2), optimizing computational overhead and facilitating deployment on low-power embedded devices that lack floating-point arithmetic circuits or divider circuits. In scenarios with ample computing resources, the formula is generally used. By performing calculations, the program's perception of image scenes becomes closer to that of the human eye, which can further improve the output effect of the algorithm.

[0058] (3). Select a reasonable two-dimensional Gaussian convolution kernel with an appropriate window size using the parameters (in this example, a radius of 2 and a window size of [missing information] are used). Gaussian filtering is applied to the image using a Gaussian convolution kernel.

[0059] (4) Utilizing various two-dimensional convolution kernels for calculating gradients (such as the Robert operator) / Prewitt operator / Sobel operator / (etc.), calculate the Cartesian coordinate components of the first-order gradient around each pixel. Then calculate its amplitude. Only the amplitude is stored. After calculating for all pixels, the amplitude is mapped to the valid range of pixel grayscale values ​​(the quantization bit depth of the pixel grayscale value is selected according to actual needs; it is generally 0-255 in an 8-bit grayscale image and 0-1023 in a 10-bit grayscale image). This exemplary embodiment validates the value by right shifting. When computing resources are sufficient, a method of normalizing after counting the maximum value can be used.

[0060] (5) Calculate the histogram of gradient grayscale. Excluding statistical information where the gradient grayscale is 0, and guided by the high confidence ratio threshold selected by the user, calculate the gradient magnitude threshold that can be considered as an edge based on the information given by the histogram. For example, if there are 10,000 pixels with values ​​greater than 0 in the gradient grayscale image, and the user selects a high confidence ratio threshold of 15%, then the gradient grayscale value threshold is the grayscale value corresponding to the slot when the accumulated values ​​exceed 10,000 × (100% - 15%) = 8,500 pixels, calculated by accumulating the values ​​from the low slot to the high slot according to the histogram.

[0061] (6). At this point, edge tracking begins. The initial preceding tracking start stack is empty. If starting from the top left pixel of the image and continuing until the bottom right pixel of the image is reached, if the preceding tracking start stack is not empty, the top of the stack is popped as the initial point (edge ​​front point) of the current round; otherwise, the process proceeds column by column to the right and then row by row downwards, selecting pixels whose gradient gray value is not less than the minimum legal gradient gray value threshold calculated in the previous step as the initial point (edge ​​front point) of edge tracking and pushing them onto the initial preceding tracking start stack.

[0062] Begin tracking a single edge in this round. When the leading edge point of the next edge can be found, perform the following steps (if the bottom right corner of the image has been reached, end the tracking and obtain the edge information):

[0063] 6a) Select the initial pixel point (edge ​​front point) for the next step within the neighboring candidate range. The neighboring candidate range specifically refers to: a "4-connected" range considering only the top, bottom, left, and right pixels; or an "8-connected" range adding the top-left, bottom-left, top-right, and bottom-right pixels to the "4-connected" range (see [link to relevant documentation]). Figure 3 The gradient magnitude of a qualified candidate point should be no less than the edge gradient magnitude threshold given in step (5). If there are multiple qualified candidate points, select the one with the largest gradient magnitude.

[0064] 6b) If the initial pixel point for the next step is successfully selected within the above "4-connected" or "8-connected" candidate range (if selection fails, the current round ends and the next round of tracking begins from step (6)): Figure 4 As shown, the gradient magnitude of two neighboring pixels that form a 135° angle with the line connecting the current initial point and the next initial point is directly set to zero after legalization, to avoid the same pixel or the same edge being tracked repeatedly. Because... Figure 4The relative pixel positions shown resemble the wingspan of an airplane; this zeroing strategy is also known as the "wingspan" zeroing strategy. After completion, the count of the slots in the histogram corresponding to the two points where the "wingspan" is zeroed for the current initial point is decremented by 1, and then the number of pixels contained in the current edge is incremented by 1. Then, starting from the next initial point (edge ​​leading edge point) selected in this step, the tracking returns to step 6a).

[0065] (7) Complete edge tracking and obtain edge information, such as... Figure 7 As shown.

[0066] For this exemplary embodiment, by simply designing the preschool cards appropriately so that different cards have different edge features after edge extraction, the main algorithm of this exemplary embodiment can be used for classification and recognition, and the classification and recognition accuracy is extremely high. By designing several cards, each with different edge features, and deliberately differentiating them in terms of the number of edges of the main pattern, the single or double line style of the edge frame, and the closure of the edge frame, these features can be used as encoding features, and the algorithm can be applied with extremely high accuracy.

[0067] Besides standalone applications, tracking is generally considered an early-stage algorithm in image processing and pattern recognition. Further feature matching of edge information can enable more advanced applications. The output accuracy of this algorithm is generally sufficient to meet the needs of subsequent algorithms in applications such as high-contrast sign recognition (e.g., road traffic signs) and hand gesture recognition.

[0068] The above description is merely a preferred embodiment of the present invention and is used to illustrate the invention, not to limit it. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention; therefore, all equivalent technical solutions also fall within the scope of the invention, and the patent protection scope of the invention should be defined by the claims.

Claims

1. A method of image edge detection and tracking, characterized by It comprises two links of edge detection and edge tracking; The specific process steps of the edge detection link are: a) inputting a color image through streaming or reading a file, calculating the gray scale value corresponding to each color pixel in the image by using a color-to-gray scale conversion formula to obtain a gray scale image; b) performing Gaussian filtering on the obtained gray scale image to weaken or even eliminate the influence of high-frequency noise in the image, and obtaining a smooth gray scale image; c) For the smoothed gray scale image obtained after Gaussian filtering, the components of the right angle of the first order gradient around each pixel are calculated and the magnitude thereof is calculated and only the magnitude is stored. After the calculation for all pixels is completed, the magnitude is processed by multiplication to make it fall within the legal gray scale value range under the quantization bit depth selected by the user, and a gradient magnitude matrix is generated, wherein d x represents the component of the gradient in the horizontal direction, d y represents the component of the gradient in the vertical direction; d) generating a gradient amplitude histogram according to the gradient amplitude matrix, and calculating an edge gradient amplitude threshold value according to a proportion threshold value of pixels representing edges; In the edge tracking link, each edge is tracked in turn; the specific tracking process steps of each edge are: e) initial point selection: starting from the top left corner of the gradient amplitude matrix generated in the edge detection link, traversing to the right and down, and selecting a point with a gradient amplitude greater than the edge gradient value threshold as the initial point of the edge as the "front point" of the edge; f) edge "concatenation": starting from the "front point", selecting the maximum value of the pixels in the neighborhood of the current "front point" as the "front point" of the next round, and the "concatenation" of the next round of "front points" is performed into the current edge; g) "cleaning" redundant points: using a "wing span" pixel zeroing strategy to set the gradient amplitude of the points to be "cleaned" to zero, and reducing the count of the corresponding interval in the histogram by 1 and increasing the number of pixels in the current edge by 1; h) "concatenation" and "cleaning" operations are performed in a loop until no point with a gradient amplitude greater than the edge gradient value threshold can be selected in the neighborhood, and the edge ends; i) edge verification: according to the conditions that the edge should satisfy, deciding whether to include the current detected edge in the output or exclude it from the output; j) if the right bottom corner of the gradient gray scale image matrix has been traversed, the tracking ends.

2. The method of claim 1, further comprising: In the edge detection link, the color-to-gray scale conversion in step a) is performed using the formula I = (R + (G << 1) + B) >> 2, where R represents the red channel value, G represents the green channel value, B represents the blue channel value, and I represents the luminance gray scale value.

3. The method of claim 1, further comprising: In the edge detection link, the edge gradient amplitude threshold value in step d) is the left boundary value of the corresponding luminance interval slot whose cumulative frequency exceeds [total non-zero gradient amplitude point number × (100% - user-selected high confidence proportion threshold)].

4. The method of claim 1, further comprising: In step g) of the edge tracking link, the "wing span" pixel zeroing strategy requires that the points to be "cleaned" are the two adjacent pixel points in the direction of the 135° angle formed by the line connecting the current "front point" and the next "front point".

5. The method of claim 1, further comprising: In step i) of the edge tracking link, the conditions that the edge should satisfy are: calculating whether the total number of pixels contained in the current tracked edge is greater than or equal to a set minimum edge pixel number threshold value: if it is satisfied, the current edge is valid, and the 8-connected points around the current point are set to zero; if it is not satisfied, the current edge is invalid, and the information of the current edge is cleared in the memory.

Images