Image clustering method and electronic device

Abstract

The application relates to the technical field of image processing, and discloses an image clustering method and an electronic device. The image clustering method comprises the following steps: acquiring pixel points of each frame of image; if it is determined that the pixel points are effective pixel points according to the gray values of the pixel points, performing coarse clustering on the image to obtain coarse clustering results, wherein each coarse clustering result corresponds to a clustering number; and performing fine clustering on the coarse clustering results according to the clustering numbers and a first command to obtain fine clustering results. The application can reduce the data amount of processing and save the image processing time.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image clustering method and electronic device. Background Technology

[0002] Region labeling methods are mainly used to extract the desired target regions from images. They play a crucial role in image processing fields such as machine vision and have wide applications in areas such as face recognition, text recognition, fingerprint recognition, and medical image analysis.

[0003] Currently, pixel scanning is commonly used for image region labeling. Pixel scanning treats each pixel as an object, sequentially scanning the entire image, and each operation only considers the 4 or 8-neighborhood of the target pixel. This method is suitable for parallel hardware implementation, but it generates a large number of temporary labels, requiring multiple scans to complete the connection and replacement of equivalent labels. However, caching the entire image and traversing the image multiple times increases the amount of data processed, resulting in longer image processing times. Summary of the Invention

[0004] This application provides an image clustering method and electronic device to reduce the amount of data processed and save image processing time.

[0005] This application provides the following technical solution:

[0006] Firstly, this application provides an image clustering method, including:

[0007] Obtain the pixels of each frame of the image;

[0008] If a pixel is determined to be a valid pixel based on its gray value, then coarse clustering is performed on the image to obtain coarse clustering results, where each coarse clustering result corresponds to a cluster number.

[0009] Based on the cluster number and the first command, the coarse clustering results are subjected to fine clustering to obtain the fine clustering results.

[0010] In some embodiments, valid pixels include pixels with gray values ​​greater than the high-pass filter threshold;

[0011] If a pixel is determined to be a valid pixel based on its grayscale value, then coarse clustering is performed on the image to obtain the coarse clustering results, including:

[0012] Determine whether each valid pixel is the first valid pixel in a frame of an image;

[0013] If the effective pixel is the first effective pixel in a frame of an image, then the first clustering region is determined with the first effective pixel as the center.

[0014] If the valid pixel is not the first valid pixel in a frame, then determine whether the pixel coordinates of the valid pixel are within the first clustering region.

[0015] If the pixel coordinates of a valid pixel are within the first cluster region, then the valid pixel is assigned to the first cluster region.

[0016] After the image traversal is completed, the first coarse clustering result, consisting of all valid pixels within the first clustering region, is cached in the first clustering module.

[0017] In some embodiments, if a pixel is determined to be a valid pixel based on its grayscale value, then coarse clustering is performed on the image to obtain a coarse clustering result, which further includes:

[0018] If the pixel coordinates of a valid pixel are not within the first cluster region, then a second cluster region is determined with the valid pixel as the center, wherein the second cluster region is different from the first cluster region;

[0019] After the image traversal is completed, the second coarse clustering result, consisting of all valid pixels within the second clustering region, is cached in the second clustering module.

[0020] In some embodiments, based on the cluster number and the first command, the coarse clustering results are refined to obtain refined clustering results, including:

[0021] Based on the cluster number and the first command, the coarse clustering results of the corresponding data volume are read in parallel in different clustering modules;

[0022] The image growth operation is performed on each coarse clustering result by the region growing algorithm to obtain the fine clustering result corresponding to each coarse clustering result. Each coarse clustering result and its corresponding fine clustering result have the same cluster number.

[0023] In some embodiments, an image growing operation is performed on each coarse clustering result using a region growing algorithm to obtain a fine clustering result corresponding to each coarse clustering result, including:

[0024] Starting from the first pixel of each coarse clustering result, traverse each coarse clustering result.

[0025] Each time the coarse clustering results are traversed, the step size of the fine clustering side length is increased by one pixel.

[0026] When the pixel coordinates of all pixels in the coarse clustering result are outside the growth coordinate range, the fine clustering operation ends, and the fine clustering result corresponding to each coarse clustering result is obtained. The growth coordinate range is determined by the fine clustering side length.

[0027] In some embodiments, the method further includes:

[0028] According to the transmission sequence, the high-pass filtered image data is written into two first-in-first-out queues in sequence, forming three rows of data;

[0029] Apply a sliding window to every three rows of data to determine the edge attributes of each pixel in the three rows of data.

[0030] During the process of caching each coarse clustering result to each clustering module, edge attributes are placed in the highest bit of the pixel.

[0031] In some embodiments, the method further includes:

[0032] When reading coarse clustering results of corresponding data volume in parallel in different clustering modules, edge attribute judgment is performed on the highest bit of each pixel, and the side length of each pixel is normalized.

[0033] If a pixel is an edge point, then the side length of the pixel is included in the perimeter.

[0034] If a pixel is an interior point, its perimeter remains unchanged, where the perimeter represents the number of edge points;

[0035] The perimeter includes:

[0036]

[0037] in, Indicates the perimeter. Indicates an edge point.

[0038] In some embodiments, each refined clustering result corresponds to an image area and a perimeter, and the method further includes:

[0039] Normalize the area of ​​each pixel, count the number of pixels in each fine clustering result, and use the number of pixels in each fine clustering result as the image area corresponding to each fine clustering result.

[0040] The roundness of each fine clustering result is calculated based on the perimeter and image area.

[0041] If the roundness of the refined clustering result satisfies the first condition, then output the refined clustering result;

[0042] The formula for calculating roundness includes:

[0043]

[0044] in, Indicates roundness, Represents the area of ​​the image. Indicates the perimeter.

[0045] In some embodiments, the roundness of each refined clustering result is calculated based on the perimeter and image area, including:

[0046] The reciprocal of the square of the perimeter is cached in memory according to a preset method;

[0047] Using the square of the actual perimeter as the read address, read the data in the memory to obtain the reciprocal of the square of the actual perimeter;

[0048] Substituting the reciprocal of the square of the actual perimeter into the formula for calculating roundness, the roundness of the refined clustering result is obtained.

[0049] Secondly, this application provides an electronic device, comprising:

[0050] At least one processor; and

[0051] A memory that is communicatively connected to at least one processor; wherein,

[0052] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform an image clustering method such as the first aspect.

[0053] Thirdly, this application provides a non-volatile computer-readable storage medium storing computer-executable instructions for causing an electronic device to perform the image clustering method of the first aspect.

[0054] The beneficial effects of this application are as follows: Unlike existing technologies, this application provides an image clustering method, comprising: acquiring pixels of each frame of an image; if a pixel is determined to be a valid pixel based on its grayscale value, then performing coarse clustering on the image to obtain coarse clustering results, wherein each coarse clustering result corresponds to a clustering number; and performing fine clustering on the coarse clustering results based on the clustering number and a first command to obtain fine clustering results. This application can reduce the amount of data processed and save image processing time. Attached Figure Description

[0055] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0056] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0057] Figure 2 This is a flowchart illustrating an image clustering method provided in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of an original image provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram illustrating a mean filtering of an original image provided in an embodiment of this application;

[0060] Figure 5 yes Figure 2 Detailed flowchart of step S203 in the process;

[0061] Figure 6 This is a schematic diagram of a coarse clustering method provided in an embodiment of this application;

[0062] Figure 7 yes Figure 2 Detailed flowchart of step S204 in the process;

[0063] Figure 8 yes Figure 7 Detailed flowchart of step S242 in the process;

[0064] Figure 9 This is a schematic diagram of a refined clustering method provided in an embodiment of this application;

[0065] Figure 10 This is a schematic diagram of the data flow in image processing provided in an embodiment of this application;

[0066] Figure 11 This is a schematic diagram of an edge detection process provided in an embodiment of this application;

[0067] Figure 12 This is a schematic diagram of a data cache provided in an embodiment of this application;

[0068] Figure 13 This is a schematic diagram of three rows of data provided in an embodiment of this application;

[0069] Figure 14 This is a schematic diagram of a data sliding window provided in an embodiment of this application;

[0070] Figure 15 This is a schematic diagram of a process for solving the perimeter provided in an embodiment of this application;

[0071] Figure 16 This is a schematic diagram of a process for outputting refined clustering results provided in an embodiment of this application;

[0072] Figure 17 yes Figure 16 Detailed flowchart of step S162 in the process;

[0073] Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0074] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on the other element, or one or more intermediate elements may exist between them. When an element is described as "connected to" another element, it can be directly connected to the other element, or one or more intermediate elements may exist between them. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this specification are for illustrative purposes only.

[0075] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0076] The technical solution of this application will be described in detail below with reference to the accompanying drawings:

[0077] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0078] like Figure 1 As shown, the application environment 100 includes an image sensor 10 and an electronic device 20, wherein the image sensor 10 and the electronic device 20 are connected via network communication, wherein the network includes a wired network and / or a wireless network. It is understood that the network includes wireless networks such as 2G, 3G, 4G, 5G, wireless LAN, and Bluetooth, and may also include wired networks such as serial cables and network cables.

[0079] In this embodiment, the image sensor 10 is communicatively connected to the electronic device 20 and is used to send images to the electronic device 20. Specifically, the image sensor 10, also known as a photosensitive element, is a device that converts optical image information into electrical signals, and is specifically used to send the converted image information to the electronic device 20. In this embodiment, the image sensor 10 includes, but is not limited to, a charge-coupled device (CCD) image sensor and a complementary metal-oxide-semiconductor (CMOS) image sensor.

[0080] In this embodiment of the application, the electronic device 20 is communicatively connected to the image sensor 10 and is used to receive images sent by the image sensor 10, or to execute the image clustering method in any of the following embodiments, for example: obtaining the pixels of each frame of the image; if the pixels are determined to be valid pixels based on their grayscale values, then performing coarse clustering on the image to obtain coarse clustering results; and performing fine clustering on the coarse clustering results based on the clustering number and the first command to obtain fine clustering results.

[0081] The electronic device 20 is an electronic device with a certain computing capability. This electronic device 20 includes, but is not limited to, terminals and servers. Terminals include, but are not limited to, various terminals with computing capabilities such as laptops, desktop computers, or mobile devices. Servers include, but are not limited to, tower servers, rack servers, blade servers, and cloud servers. Preferably, the electronic device 20 is an electronic device with a field-programmable gate array (FPGA).

[0082] In this embodiment, if the electronic device 20 is a server, there can be multiple servers, and these multiple servers can form a server cluster. For example, the server cluster includes a first server, a second server, ..., an Nth server. Alternatively, the server cluster can be a cloud computing service center, which includes several servers. Preferably, the electronic device 20 is a cloud server (Elastic Compute Service, ECS) with a field-programmable gate array (FPGA).

[0083] Please see Figure 2 , Figure 2 This is a flowchart illustrating an image clustering method provided in an embodiment of this application;

[0084] The image clustering method is applied to electronic devices, such as terminals and servers. Specifically, the execution subject of the image clustering method is one or at least two processors in the electronic device. Preferably, the processor is a field-programmable gate array (FPGA).

[0085] like Figure 2 As shown, this image clustering method includes:

[0086] Step S201: Obtain the pixels of each frame of the image;

[0087] Specifically, the FPGA acquires the pixels of each frame of the image after high-pass filtering.

[0088] In this embodiment of the application, before obtaining the pixels of each frame of the image, the method further includes:

[0089] Preprocess the image.

[0090] Specifically, image preprocessing includes steps (1) to (3), as follows:

[0091] Step (1): Determine the pixel coordinates corresponding to each pixel point according to the transmission timing of the image sensor to obtain the original image;

[0092] Specifically, the FPGA receives images sent by the image sensor and generates a pixel coordinate system for each frame of the image according to the transmission sequence of the image, thereby determining the pixel coordinates corresponding to each pixel in a frame of the image and obtaining the original image, wherein the original image includes the pixel coordinates corresponding to each pixel.

[0093] Specifically, the FPGA calculates the row and column numbers for a frame of image, and represents the position of a pixel in the pixel coordinate system, i.e., the pixel coordinates, using row and column numbers. Each pixel has unique pixel coordinates.

[0094] Please see Figure 3 , Figure 3 This is a schematic diagram of an original image provided in an embodiment of this application;

[0095] like Figure 3 As shown, the original image is M columns * N rows in size. The original image is located in a pixel coordinate system, where each pixel has a unique pixel coordinate.

[0096] Step (2): Perform mean filtering on the original image through the mean filtering window;

[0097] Specifically, mean filtering is a linear filtering algorithm that replaces the current pixel value with the average of the N*N pixels surrounding the current pixel. By using this method to process every pixel in the image, mean filtering of the entire image can be completed.

[0098] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating a mean filtering of an original image provided in an embodiment of this application;

[0099] like Figure 4 As shown, the size of the mean filtering window is 3*3.

[0100] In this embodiment, by sweeping a 3*3 mean filtering window (also known as a filter window) across the entire original image to perform mean filtering, it is possible to smooth out abrupt and isolated noise in the original image.

[0101] Step (3): Perform high-pass filtering on the image after mean filtering.

[0102] Specifically, a high-pass filter is a filtering method that allows high-frequency signals to pass through normally, while blocking or weakening low-frequency signals below a set threshold. It is mainly used to eliminate low-frequency noise and is also known as a low-cutoff filter.

[0103] In this embodiment, an ideal high-pass filter is used, and the specific formula is as follows:

[0104]

[0105] Where H(u,v) represents the grayscale value of the pixel with pixel coordinates (u,v) after high-pass filtering, D(u,v) represents the original pixel grayscale value of the pixel with pixel coordinates (u,v), and D0 is the grayscale threshold of the high-pass filtering.

[0106] Specifically, if the original pixel gray value of a certain pixel in the original image is greater than the high-pass filter gray value threshold, then the original pixel gray value is output; if the original pixel gray value of a certain pixel in the original image is not greater than the high-pass filter gray value threshold, then the gray value of that pixel is set to zero. The high-pass filter gray value threshold can be set according to the actual situation.

[0107] Step S202: Determine whether each pixel is a valid pixel;

[0108] Specifically, each pixel is determined to be a valid pixel based on its grayscale value. Valid pixels include pixels whose grayscale value is greater than the high-pass filter threshold, that is, pixels whose original pixel grayscale value is greater than the high-pass filter grayscale threshold or pixels whose grayscale value after high-pass filtering is greater than the high-pass filter grayscale threshold.

[0109] Further, if the pixel is a valid pixel, proceed to step S203: perform coarse clustering on the image to obtain the coarse clustering result; if the pixel is not a valid pixel, return to step S202 and continue to determine whether other pixels in this frame of the image are valid pixels.

[0110] Understandably, after determining that a pixel is a valid pixel, the process continues to traverse the frame image to determine whether other pixels are valid pixels. If a pixel is valid, the process proceeds to step S203; otherwise, it returns to step S202. This process continues until every pixel in the frame image has undergone the determination operation of step S202, at which point the traversal stops.

[0111] Step S203: Perform coarse clustering on the image to obtain the coarse clustering results;

[0112] Specifically, each coarse clustering result corresponds to a cluster number.

[0113] For details, please refer to [link / reference]. Figure 5 , Figure 5 yes Figure 2 Detailed flowchart of step S203 in the process;

[0114] In this embodiment, the FPGA includes a coarse clustering module, which is used to obtain the effective pixels of each frame of the image after high-pass filtering, and to perform coarse clustering on the image to obtain the coarse clustering result.

[0115] In this embodiment of the application, the electronic device further includes at least one memory for storing coarse clustering results and fine clustering results. The memory includes, but is not limited to, random access memory. Preferably, the memory is dual-ported RAM (DPRAM).

[0116] Dual-port random access memory (DRAM) is a type of random access memory that allows two devices to access it simultaneously (or almost simultaneously). It has two sets of address buses and two sets of data buses (hence the name "dual-port"), thus allowing access by two devices, whereas general random access memory only has one set of address buses and one set of data buses.

[0117] like Figure 5 As shown, step S203 involves performing coarse clustering on the image to obtain coarse clustering results, including:

[0118] Step S2031: Identify each valid pixel;

[0119] Step S2032: Determine whether each valid pixel is the first valid pixel of a frame image;

[0120] Specifically, during the transmission of images to the coarse clustering module after passing through high-pass filtering, the images are transmitted frame by frame. A frame synchronization signal accompanies the transmission. After the frame synchronization signal reaches the coarse clustering module, the first valid pixel received by the coarse clustering module is the first valid pixel of the current frame.

[0121] Further, if the valid pixel is the first valid pixel in a frame of an image, then proceed to step S2033: determine the first clustering region with the first valid pixel as the center; if the valid pixel is not the first valid pixel in a frame of an image, then proceed to step S2034: determine whether the pixel coordinates of the valid pixel are within the first clustering region.

[0122] Step S2033: Determine the first clustering region centered on the first valid pixel;

[0123] Specifically, if the effective pixel is the first effective pixel in a frame of an image, then the first clustering region is determined with the first effective pixel as the center. The first clustering region is determined by the coarse clustering radius, which is a square region centered on the first effective pixel with a side length equal to the coarse clustering radius. The coarse clustering radius represents the number of pixels and can be set according to the actual situation. It is generally determined based on the image characteristics. For example, the coarse clustering radius can be set to 50, representing 50 pixels.

[0124] Step S2034: Determine whether the pixel coordinates of the valid pixels are within the first clustering region;

[0125] Specifically, if a valid pixel is not the first valid pixel in a frame, then it is determined whether the pixel coordinates of the valid pixel are within the first clustering region.

[0126] Further, if the pixel coordinates of the valid pixel are within the first clustering region, then proceed to step S2035: classify the valid pixel into the first clustering region; if the pixel coordinates of the valid pixel are not within the first clustering region, then proceed to step S2037: determine the second clustering region with the valid pixel as the center.

[0127] Step S2035: Divide the valid pixels into the first clustering region;

[0128] Specifically, if the pixel coordinates of a valid pixel are within the first clustering region, then the valid pixel is assigned to the first clustering region.

[0129] Step S2036: After the image traversal is completed, the first coarse clustering result composed of all valid pixels in the first clustering region is cached in the first clustering module;

[0130] Specifically, the first clustering module is the first dual-port random access memory, and the first clustering module and the first clustering region have the same cluster number, namely number one.

[0131] Step S2037: Determine the second clustering region centered on the effective pixels;

[0132] Specifically, the second clustering region differs from the first clustering region. It is divided after the first clustering region is determined. The second clustering region is a square region centered on the current effective pixel and with a coarse clustering radius as the side length. The coarse clustering radius represents the number of pixels and can be set according to the actual situation. It is generally determined based on the image characteristics. For example, the coarse clustering radius can be set to 50, representing 50 pixels.

[0133] In this embodiment of the application, the coarse clustering radius of the second clustering region is the same as that of the first clustering region. Therefore, the area of ​​the second clustering region is the same as that of the first clustering region, but their positions are different.

[0134] Step S2038: After the image traversal is completed, the second coarse clustering result composed of all valid pixels in the second clustering region is cached in the second clustering module.

[0135] Specifically, the second clustering module is the second dual-port random access memory, and the second clustering module and the second clustering region have the same cluster number, namely number two.

[0136] In this embodiment, each dual-port random access memory (dpram) is parallel, and each clustering region corresponds to a clustering module. The clustering module is a dual-port random access memory, and each dual-port random access memory stores the coarse clustering result composed of all valid pixels in a coarse clustering region.

[0137] Please see Figure 6 , Figure 6 This is a schematic diagram of a coarse clustering method provided in an embodiment of this application;

[0138] like Figure 6 As shown in the figure, the four circles represent parts of the image that have been coarsely clustered, and the dashed boxes in the figure represent the clustering regions determined by the coarse clustering radius.

[0139] Specifically, the high-pass filtered image received by the coarse clustering module is transmitted row by row. The first valid pixel of a frame received by the coarse clustering module is... Figure 6 The point on the circle in the upper right corner of the image that intersects with the two line segments is the first clustering region. After receiving the valid pixel, a square region is drawn with the valid pixel as the center and the coarse clustering radius as the side length. Figure 6 The area enclosed by the dashed box outside the circle in the upper right corner.

[0140] Furthermore, the coarse clustering module continues to receive other valid pixels and determines whether the pixel coordinates of each valid pixel are within the first clustering region, i.e., whether they are within the first clustering region. Figure 6 Within the area enclosed by the dashed box in the upper right corner, if the pixel is within the first clustering region, then the valid pixel will be classified into the first clustering region.

[0141] Furthermore, when the pixel coordinates of the valid pixels received by the coarse clustering module are not within the first clustering region, for example: Figure 6 The point where the circle in the upper left corner intersects with the two line segments is used as the center of a new region, namely the second clustering region, with the coarse clustering radius as the side length. Figure 6 The area enclosed by the dashed box in the upper left corner. It's understandable that this is because the pixel coordinates of the valid pixel point exceed... Figure 6The area enclosed by the dashed box in the upper right corner indicates that this valid pixel belongs to a new coarse clustering region.

[0142] Similarly, when the pixel coordinates of valid pixels received by the coarse clustering module are not in the already labeled regions, i.e., the first and second clustering regions, for example: Figure 6 When the point on the circle in the lower left corner intersects with the two line segments, the pixel coordinates of the effective pixel are not within the area encompassed by the dashed box in the upper left corner (i.e., the first cluster area) and the area encompassed by the dashed box in the upper right corner (i.e., the second cluster area). Therefore, a new area, namely the third cluster area, is divided with the effective pixel as the center.

[0143] This process continues until the entire image has been traversed.

[0144] In this embodiment of the application, the cluster number of each cluster region is sorted according to the time sequence in which the cluster regions were defined, for example: Figure 6 The first cluster region in the upper right corner is the first newly divided cluster region, so it is the first cluster region and its cluster number is one. Figure 6 The second cluster region in the upper left corner is the second newly divided cluster region, therefore it is the second cluster region and its cluster number is number two; Figure 6 The third cluster region in the lower left corner is the third newly divided cluster region, so it is the third cluster region and its cluster number is three.

[0145] In this embodiment, multiple dual-port random access memory (DRAM) instances are instantiated according to performance metrics. These DRAMs operate in parallel. For example, if the performance metrics require support for labeling 32 regions, 32 coarse clustering DRAMs (coarse clustering dprams) are instantiated. Each coarse clustering dpram is a clustering module. Different coarse clustering dprams store the coarse clustering results within different clustering regions obtained from coarse clustering. For example, the coarse clustering results within the first clustering region are written to the first coarse clustering dpram, i.e., the first clustering module; the coarse clustering results within the second clustering region are written to the second coarse clustering dpram, i.e., the second clustering module, and so on.

[0146] In this embodiment, images sent by an image sensor are received, and a pixel coordinate system is generated for each frame of the image according to the transmission sequence of the images. Mean filtering and high-pass filtering are then performed to obtain the pixels of each frame of the image after high-pass filtering. If a pixel is a valid pixel, coarse clustering is performed on the image to obtain the coarse clustering result. This application hides the entire coarse clustering algorithm in the transmission sequence of the image, and can perform clustering and labeling on the received pixels in real time without caching an entire image.

[0147] In this embodiment, the FPGA further includes at least one fine clustering module. The number of fine clustering modules is the same as the number of coarse clustering dpram (i.e., clustering modules). The fine clustering modules are used to perform fine clustering on the coarse clustering results to obtain fine clustering results.

[0148] Step S204: Based on the cluster number and the first command, perform fine clustering on the coarse clustering results to obtain the fine clustering results.

[0149] Specifically, each clustering module (i.e., each coarse clustering dpram) has the same clustering number as each clustering region. The first command is sent from the coarse clustering module to the fine clustering module. The first command includes a data volume indicator, which is the amount of data that the fine clustering module needs to read from the coarse clustering dpram.

[0150] For details, please refer to [link / reference]. Figure 7 , Figure 7 yes Figure 2 Detailed flowchart of step S204 in the process;

[0151] like Figure 7 As shown, step S204: Based on the cluster number and the first command, perform fine clustering on the coarse clustering results to obtain fine clustering results, including:

[0152] Step S241: Based on the cluster number and the first command, read the coarse clustering results of the corresponding data volume in parallel in different clustering modules;

[0153] Specifically, the fine clustering module reads the corresponding amount of coarse clustering results from the clustering module corresponding to the clustering number based on the clustering number and the first command. For example, a dual-port random access memory instantiated with a capacity of 8192, but the data volume indicated in the first command is 4096. The data volume of 4096 indicates that only 4096 valid data were written to the coarse clustering dpram in the previous step. Therefore, the fine clustering module only needs to read 4096 data from the coarse clustering dpram.

[0154] In the embodiments of this application, all fine clustering modules read data from the coarse clustering dpram in parallel: that is, the first fine clustering module reads data from the first coarse clustering dpram, i.e., the first clustering module, the second fine clustering module reads data from the second coarse clustering dpram, i.e., the second clustering module, and so on, with each fine clustering module corresponding one-to-one with the clustering number of the coarse clustering dpram, i.e., the clustering module.

[0155] Step S242: Perform image growth operation on each coarse clustering result using the region growing algorithm to obtain the fine clustering result corresponding to each coarse clustering result.

[0156] Specifically, each coarse clustering result has the same cluster number as its corresponding fine clustering result.

[0157] Specifically, region growing is an image segmentation algorithm. Its basic idea is to merge pixels with similar properties together. For each region, a seed point is first designated as the starting point for growth. Then, the pixels in the surrounding neighborhood of the seed point are compared with the seed point, and pixels with similar properties are merged to continue growing outwards until no pixels meeting the conditions are included. This completes the growth of one region.

[0158] For details, please refer to [link / reference]. Figure 8 , Figure 8 yes Figure 7 Detailed flowchart of step S242 in the process;

[0159] like Figure 8 As shown, step S242 involves performing image growth operations on each coarse clustering result using a region growing algorithm to obtain the fine clustering result corresponding to each coarse clustering result, including:

[0160] Step S2421: Starting from the first pixel of each coarse clustering result, traverse each coarse clustering result;

[0161] Specifically, each fine clustering module reads the data in each coarse clustering dpram, and then, starting from the first pixel of each coarse clustering result, it iterates through the corresponding amount of coarse clustering results in each coarse clustering dpram.

[0162] Step S2422: For each iteration of the coarse clustering results, increase the step size of the fine clustering side by one pixel;

[0163] Specifically, the fine clustering edge length is the edge length of the growth coordinate range in the region growing algorithm. After each traversal of the coarse clustering results of the corresponding amount of data in the coarse clustering dpram, the image in the fine clustering module grows by one pixel, that is, the fine clustering edge length is increased by one pixel. Then the next traversal continues until no pixel coordinates are within the growth coordinate range. The fine clustering edge length is initially zero, and after the first traversal, the fine clustering edge length is one pixel.

[0164] Step S2423: When the pixel coordinates of all pixels in the coarse clustering result are outside the growth coordinate range, the fine clustering operation ends, and the fine clustering result corresponding to each coarse clustering result is obtained.

[0165] Specifically, the growth coordinate range is determined by the side length of the refined clustering, which is a square region with the side length of the refined clustering. When the pixel coordinates of all pixels in the corresponding amount of coarse clustering results in the coarse clustering dpram are outside the growth coordinate range, it means that the remaining points in the coarse clustering dpram that are not included in the growth coordinate range are no longer connected to the points included in the previous growth coordinate range, which means that the region growth condition is no longer met. At this point, it marks the end of the refined clustering operation of a refined clustering module.

[0166] In this embodiment, each fine clustering module corresponds to a fine clustering dpram, and their clustering numbers are the same. After obtaining the fine clustering result corresponding to each coarse clustering result, the fine clustering result is cached in the fine clustering dpram corresponding to the fine clustering module, wherein the number of fine clustering dprams is the same as the number of fine clustering modules.

[0167] It is understandable that in this application, DPRAM is divided into two categories: one is coarse clustering DPRAM, used to store coarse clustering results, and the coarse clustering DPRAMs are parallel to each other; the other is fine clustering DPRAM, used to store fine clustering results, and the fine clustering DPRAMs are also parallel to each other, meaning that each fine clustering module writes back to the fine clustering DPRAM corresponding to its own cluster number. It is also understandable that the cache memory operates using a write-back mechanism.

[0168] In this embodiment, the coarse clustering result may encompass multiple regions with intervals greater than the step size set by the region growth mechanism into a single coarse cluster dpram. Therefore, fine clustering is required to remove discontinuous regions from the coarse clustering result. Here, discontinuity refers to intervals greater than the step size set by the region growth mechanism. Fine clustering involves retaining only the first region among multiple regions with intervals greater than the region growth step size, discarding the subsequent regions. The first region refers to the cluster region to which the first valid pixel in the coarse cluster dpram belongs. Thus, the fine clustering result is only a subset of the coarse clustering result, and therefore needs to be re-cached.

[0169] Please see Figure 9 , Figure 9 This is a schematic diagram of a refined clustering method provided in an embodiment of this application;

[0170] like Figure 9As shown in the figure, the 12 circles represent the same coarse clustering result, and each sub-figure represents one traversal. The area enclosed by the black dashed frame represents the growth coordinate range, i.e., the step size of each growth. The three black solid dots indicate that multiple traversals have been performed, i.e., multiple region growths have occurred. It can be seen that with each traversal of the coarse clustering result, the range of the black dashed frame expands until it encompasses the entire coarse clustering result. This is because with each traversal of the coarse clustering result, the side length of the fine clustering increases by one pixel. The fine clustering operation ends when no pixel coordinates are within the growth coordinate range of the fine clustering. The side length of the fine clustering is the side length of the square region enclosed by the black dashed frame.

[0171] Please see Figure 10 , Figure 10 This is a schematic diagram of the data flow in image processing provided in an embodiment of this application;

[0172] In the embodiments of this application, image processing mainly includes three steps, namely: image preprocessing, coarse clustering and fine clustering, wherein image preprocessing includes: mean filtering and high-pass filtering.

[0173] like Figure 10 As shown, the data flow for image processing is as follows:

[0174] S1: Image sensor;

[0175] Specifically, the FPGA receives images sent by the image sensor and generates a pixel coordinate system for each frame of the image according to the transmission sequence of the image, thereby determining the pixel coordinates corresponding to each pixel in a frame of the image and obtaining the original image.

[0176] S2: Mean filtering;

[0177] Specifically, the original image is subjected to mean filtering through a mean filtering window.

[0178] S3: High-pass filter;

[0179] Specifically, a high-pass filter is applied to the image after mean filtering.

[0180] S4: Coarse clustering;

[0181] Specifically, the FPGA acquires the pixels of each frame of the image after high-pass filtering, determines whether each pixel is a valid pixel, and if the pixel is a valid pixel, performs coarse clustering on the image to obtain the coarse clustering result. The specific method for performing coarse clustering on the image is the same as step S203, and will not be repeated here.

[0182] S5: Coarse clustering Dpram;

[0183] Specifically, the coarse clustering results obtained after coarse clustering are cached in the coarse clustering DPRAM. There are multiple coarse clustering DPRAMs, for example: the first coarse clustering DPRAM 1#, the second coarse clustering DPRAM 2#, the third coarse clustering DPRAM 3#, ..., the (n-2)th coarse clustering DPRAM n-2#, the (n-1)th coarse clustering DPRAM n-1#, and the nth coarse clustering DPRAM n#, where n is a positive integer. The coarse clustering DPRAMs are in parallel.

[0184] S6: Fine clustering;

[0185] Specifically, each fine clustering module reads data from each coarse clustering dpram and performs fine clustering to obtain the corresponding fine clustering result. The specific method of fine clustering is the same as in step S204, and will not be repeated here. There are multiple fine clustering modules, for example: the first fine clustering module 1#, the second fine clustering module 2#, the third fine clustering module 3#, ..., the (n-2)th fine clustering module n-2#, the (n-1)th fine clustering module n-1#, and the nth fine clustering module n#, where n is a positive integer. The fine clustering modules are parallel to each other.

[0186] S7: Dpram, a fine clustering class;

[0187] Specifically, each fine clustering module writes the obtained fine clustering results back to the fine clustering dpram corresponding to its own clustering number. There are multiple fine clustering dprams, for example: the first fine clustering dpram 1#, the second fine clustering dpram 2#, the third fine clustering dpram 3#, ..., the (n-2)th fine clustering dpram n-2#, the (n-1)th fine clustering dpram n-1#, and the nth fine clustering dpram n#, where n is a positive integer. The fine clustering dprams are in parallel.

[0188] In this embodiment, by employing a method of coarse clustering followed by fine clustering, the application can reduce the amount of data processed by fine clustering, reduce image processing time, and complete the clustering algorithm within a 3-clock-cycle delay after image transmission. Moreover, compared to traditional image region labeling techniques, this application does not require caching an entire image for multiple labeling operations, thus saving clustering time.

[0189] Please refer to the following: Figure 11 , Figure 11 This is a schematic diagram of an edge detection process provided in an embodiment of this application;

[0190] In this embodiment, the FPGA also includes two First-In-First-Out (FIFO) queues for buffering the image data after high-pass filtering.

[0191] like Figure 11 As shown, the edge detection process includes:

[0192] Step S1101: According to the transmission timing, write the high-pass filtered image data into two first-in-first-out queues in sequence to form three rows of data;

[0193] Please refer to the following for details. Figure 12 and Figure 13 , Figure 12 This is a schematic diagram of a data cache provided in an embodiment of this application;

[0194] Figure 13 This is a schematic diagram of three rows of data provided in an embodiment of this application;

[0195] like Figure 12 As shown, the two first-in-first-out queues are Fifo0 and Fifo1, respectively. Row0 is the first row of image data after high-pass filtering, and Row1 is the second row of image data after high-pass filtering.

[0196] Specifically, the high-pass filtered image data is transmitted row by row. When the first row of high-pass filtered image data Row0 arrives, it is written into the first first-in-first-out queue Fifo0. When the second row of high-pass filtered image data Row1 arrives, it is written into the second first-in-first-out queue Fifo1, thus buffering two rows.

[0197] like Figure 13 As shown, the first FIFO queue Fifo0 has cached the first row of high-pass filtered image data Row0, and the second FIFO queue Fifo1 has cached the second row of high-pass filtered image data Row1. Together with the third row of high-pass filtered image data Row2, they form a three-row data set.

[0198] Specifically, when the third row of high-pass filtered image data Row2 arrives, the data from the first FIFO queue Fifo0 and the second FIFO queue Fifo1 are read out and combined with the third row of high-pass filtered image data Row2 to form a three-row data set.

[0199] Furthermore, please refer again. Figure 12When the high-pass filtered image data Row3 arrives, the data in the first FIFO queue Fifo0 is discarded, the original data in the second FIFO queue Fifo1 is written into the first FIFO queue Fifo0, and then the high-pass filtered image data Row2 in the third row is written into the second FIFO queue Fifo1.

[0200] In this way, the second row of high-pass filtered image data Row1, which is buffered in the first FIFO queue Fifo0, and the third row of high-pass filtered image data Row2, which is buffered in the second FIFO queue Fifo1, are read out and combined with the fourth row of high-pass filtered image data Row3 to form a new three-row data, until the entire image frame is transmitted.

[0201] In this embodiment, by caching two rows of high-pass filtered image data according to the transmission sequence, and then combining the two cached rows with the current row to form three rows, this application can reduce the occupation of FPGA resources.

[0202] Step S1102: Perform a data sliding window on every three rows of data to determine the edge attributes of each pixel in the three rows of data;

[0203] Specifically, a 3x3 window is used to slide data through every three rows of data. The four neighbors of the center point in the area defined by the window each time are determined to meet the requirements of an edge point. If they meet the requirements, the pixel is determined to be an edge point. If they do not meet the requirements, the pixel is determined to be an interior point. The gray values ​​of the four adjacent pixels above, below, left, and right of the edge point are all 0.

[0204] Please see Figure 14 , Figure 14 This is a schematic diagram of a data sliding window provided in an embodiment of this application;

[0205] like Figure 14 As shown, with the second row of data as the center, a 3x3 window is slid across these three rows of data so that the 3x3 window traverses every pixel in the three rows of data.

[0206] Furthermore, examine the center point (the middle pixel) within the 3x3 window. If the grayscale values ​​of the four adjacent pixels above, below, left, and right of this pixel are not 0, then this pixel is an internal point; if the grayscale values ​​of the four adjacent pixels above, below, left, and right of this pixel are all 0, then this pixel is an edge point.

[0207] Understandably, traditional edge detection typically uses gradient calculation, which requires the computation of various operators. However, the edge detection scheme used in this approach to determine the edge attributes of each pixel only requires placing the pixel at the center of a 3x3 window and determining whether the grayscale of the four adjacent pixels above, below, left, and right of the pixel is 0.

[0208] Step S1103: During the process of caching each coarse clustering result to each clustering module, the edge attribute is placed in the highest bit of the pixel.

[0209] Specifically, when the coarse clustering results are written back to each clustering module (i.e., the coarse clustering dpram), the edge attributes of each pixel are placed in the highest bit of the current point. The highest bit of the data represents the edge attributes of the data; if the pixel is an edge point, the highest bit of the data is 1; if the pixel is an interior point, the highest bit of the data is 0.

[0210] Please refer to the following: Figure 15 , Figure 15 This is a schematic diagram of a process for solving the perimeter provided in an embodiment of this application;

[0211] like Figure 15 As shown, the process for solving the perimeter includes:

[0212] Step S1501: When reading the coarse clustering results of the corresponding amount of data in parallel in different clustering modules, the edge attribute is judged for the highest bit of each pixel point read, and the side length of each pixel point is normalized.

[0213] Specifically, normalizing the side length of each pixel means treating the side length of each pixel as 1.

[0214] Step S1502: Determine whether each pixel is an edge point;

[0215] Specifically, if the highest bit of a pixel is 1, then the pixel is an edge point; if the highest bit of a pixel is 0, then the pixel is an interior point.

[0216] Furthermore, if the pixel is an edge point, proceed to step S1503: include the side length of the pixel in the perimeter; if the pixel is an interior point, proceed to step S1504: keep the perimeter unchanged.

[0217] Step S1503: Include the side length of the pixel in the perimeter.

[0218] Specifically, the perimeter represents the number of edge points, and the perimeter includes:

[0219]

[0220] in, Indicates the perimeter. Indicates an edge point.

[0221] Step S1504: Keep the perimeter unchanged.

[0222] Understandably, if a pixel is an interior point, then that point is not included in the perimeter calculation, and the perimeter result remains unchanged at the current moment.

[0223] It's understandable that the perimeter is the set of all edge points, which are determined during edge calculation. When performing fine clustering on the coarse clustering results, the number of pixels with a highest bit of 1 in the coarse clustering dpram is simply counted when reading the coarse clustering results. This number represents the perimeter of the current image. For example, if a coarse clustering dpram contains 4096 data points, and 1920 of them have a highest bit of 1, then 1920 of these 4096 data points are edge points, and the perimeter in this case is 1920.

[0224] In this embodiment, by normalizing the side length of a single pixel, the number of edge points in the image is made equivalent to the perimeter of the image. This eliminates the need to distinguish between the different units of perimeter and number, thus simplifying the calculation of the image perimeter and reducing the computational load.

[0225] Please refer to the following: Figure 16 , Figure 16 This is a schematic diagram of a process for outputting refined clustering results provided in an embodiment of this application;

[0226] like Figure 16 As shown, the output of the refined clustering results includes:

[0227] Step S161: Normalize the area of ​​each pixel, count the number of pixels in each fine clustering result, and use it as the image area corresponding to each fine clustering result;

[0228] Specifically, normalizing the area of ​​each pixel means treating the area of ​​a single pixel as 1*1=1. The image area is the sum of all pixels in the current region, which is the number of pixels in a fine clustering result. For example, if a fine clustering module removes 8192 data points from the corresponding coarse cluster dpram, the resulting fine clustering result will only include 7680 data points. Then the area of ​​the removed region will be 7680.

[0229] The image area includes:

[0230]

[0231] in, Represents the area of ​​the image. This indicates pixels belonging to the same refined clustering result.

[0232] In this embodiment, by normalizing the area of ​​a single pixel, the number of image pixels is made equivalent to the area of ​​the image, thus eliminating the need to distinguish between the different units of area and number. This application simplifies the calculation of image area and reduces the amount of computation required for image area calculation.

[0233] Step S162: Calculate the roundness of each fine clustering result based on the perimeter and image area;

[0234] Specifically, each fine clustering result corresponds to an image area and a perimeter. The formula for calculating the circularity includes:

[0235]

[0236] in, Indicates roundness, Represents the area of ​​the image. Indicates the perimeter.

[0237] For details, please refer to [link / reference]. Figure 17 , Figure 17 yes Figure 16 Detailed flowchart of step S162 in the process;

[0238] like Figure 17 As shown, step S162: Based on the perimeter and image area, calculate the roundness of each refined clustering result, including:

[0239] Step S1621: Cache the reciprocal of the square of the perimeter in the memory according to a preset method;

[0240] Specifically, the reciprocal of the square of the perimeter is cached in memory according to the rounding method. For example, if the square of the perimeter is a 16-bit number, then the reciprocals of all possible numbers from 0 to 65535 are written into the memory initialization file.

[0241] Step S1622: Using the square of the actual perimeter as the read address, read the data in the memory to obtain the reciprocal of the square of the actual perimeter;

[0242] Specifically, the square of the perimeter corresponding to the actual calculated fine clustering result is used as the read address to read the data in the file containing the reciprocal of the perimeter square in the memory, and thus obtain the reciprocal of the actual perimeter square.

[0243] Step S1623: Substitute the reciprocal of the square of the actual perimeter into the formula for calculating the roundness to obtain the roundness of the fine clustering result.

[0244] Specifically, the reciprocal of the square of the actual perimeter is substituted into the formula for calculating roundness. Multiplying these values ​​yields the roundness of the refined clustering result.

[0245] Understandably, not all FPGAs have a divider, and dividers are not very good at timing convergence. Therefore, in this embodiment, multiplication is used instead of division to solve for roundness, that is, the reciprocal of the square of the actual perimeter is multiplied by... Multiplying these values ​​yields the roundness of the refined clustering result.

[0246] It is understood that the embodiments in this application only illustrate the case of extending the lower bits by 18 bits. The final extension number depends on the required precision of the division operation. Theoretically, the larger the extension width, the smaller the rounding error, that is, the higher the precision of the division operation. The final roundness result needs to be truncated, removing 18 bits. If the roundness is normalized to 1024, then only 8 bits need to be truncated.

[0247] In this embodiment of the application, the reciprocal of the square of the actual perimeter is combined with... Multiplication yields the roundness of the clustering results. This application can replace division with multiplication, solving the problem of conventional FPGAs not having triggers. Furthermore, this application can extend the bit width of the reciprocal of the data to improve accuracy during rounding. The number of bits for the bit width extension mentioned in this application is variable, and different bit widths can be extended for different accuracy requirements.

[0248] Step S163: Determine whether the roundness of each fine clustering result satisfies the first condition;

[0249] Specifically, the first condition includes a roundness threshold range, which is used to identify different types of images.

[0250] In the embodiments of this application, different roundness thresholds can be set according to different image types to output fine clustering results for specific image types.

[0251] Please refer to Table 1, which is a schematic table of the circularity of different image types provided in the embodiments of this application;

[0252]

[0253] Table 1

[0254] As shown in Table 1, the circularity is 1.0 when the image type is circle, 0.79 when the image type is square, and 0.6 when the image type is triangle.

[0255] Furthermore, if a refined clustering result for circles is required, the roundness threshold range should be set to greater than 0.79 and include the value 1.0. For example, the roundness threshold range should be set to greater than 0.85, with the first condition being greater than 0.85. If a refined clustering result for squares is required, the roundness threshold range should be set to less than 1.0, greater than 0.6, and include the value 0.79. For example, the roundness threshold range should be set to greater than 0.7 and less than 0.8, with the first condition being greater than 0.7 and less than 0.8. If a refined clustering result for triangles is required, the roundness threshold range should be set to less than 0.79 and include the value 0.6. For example, the roundness threshold range should be set to less than 0.7, with the first condition being less than 0.7.

[0256] Furthermore, if the roundness of the refined clustering result meets the first condition, the refined clustering result is output; if the roundness of the refined clustering result does not meet the first condition, the process returns to step S163: determine whether the roundness of other refined clustering results meets the first condition.

[0257] Step S164: Output the fine clustering results.

[0258] Specifically, if the roundness of the fine clustering result meets the first condition, then the fine clustering result is output. For example, if the first condition is greater than 0.85, and the roundness of the fine clustering result is greater than 0.85, then the fine clustering result with a circle is output.

[0259] In this embodiment, an image clustering method is provided, comprising: acquiring pixels of each frame of an image; if a pixel is determined to be a valid pixel based on its grayscale value, then performing coarse clustering on the image to obtain coarse clustering results, wherein each coarse clustering result corresponds to a clustering number; and performing fine clustering on the coarse clustering results according to the clustering number and a first command to obtain fine clustering results. By performing coarse clustering on the image to obtain coarse clustering results if a pixel is determined to be a valid pixel based on its grayscale value, wherein each coarse clustering result corresponds to a clustering number, and performing fine clustering on the coarse clustering results according to the clustering number and a first command to obtain fine clustering results, this application can reduce the amount of data processed and save image processing time.

[0260] Please refer to the following: Figure 18 , Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0261] like Figure 18 As shown, the electronic device 180 includes one or more processors 181 and a memory 182. Wherein, Figure 18 Take a processor 181 as an example.

[0262] Processor 181 and memory 182 can be connected via a bus or other means. Figure 18 Taking the example of a connection between China and Israel via a bus.

[0263] The processor 181 is configured to provide computing and control capabilities to control the electronic device 180 to perform corresponding tasks, such as controlling the electronic device 180 to perform an image clustering method in any of the above method embodiments, including: acquiring the pixels of each frame of the image; if the pixel is determined to be a valid pixel based on its grayscale value, then performing coarse clustering on the image to obtain coarse clustering results, wherein each coarse clustering result corresponds to a clustering number; and performing fine clustering on the coarse clustering results based on the clustering number and a first command to obtain fine clustering results.

[0264] If a pixel is determined to be a valid pixel based on its grayscale value, then coarse clustering is performed on the image to obtain coarse clustering results. Each coarse clustering result corresponds to a clustering number. Based on the clustering number and the first command, fine clustering is performed on the coarse clustering results to obtain fine clustering results. This application can reduce the amount of data processed and save image processing time.

[0265] Processor 181 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Preferably, processor 181 is a field-programmable gate array (FPGA).

[0266] Memory 182, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the image clustering method in the embodiments of this application. Processor 181 can implement the image clustering method in any of the above method embodiments by running the non-transitory software programs, instructions, and modules stored in memory 182. Specifically, memory 182 may include volatile memory (VM), such as random access memory (RAM); memory 182 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; memory 182 may also include combinations of the above types of memory. Preferably, memory 182 is dual-port random access memory (DPRAM).

[0267] Memory 182 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 182 may optionally include memory remotely located relative to processor 181, and such remote memory may be connected to processor 181 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0268] One or more modules are stored in memory 182. When executed by one or more processors 181, they perform the image clustering method in any of the above method embodiments, for example, the method described above. Figure 2 The steps shown.

[0269] In this embodiment, the electronic device 180 may also have wired or wireless network interfaces, keyboards, and input / output interfaces for input and output. The electronic device 180 may also include other components for implementing device functions, which will not be elaborated here.

[0270] The electronic devices described in this application exist in various forms, and perform the above-described... Figure 2 The steps shown include, but are not limited to, various electronic devices with computing capabilities such as terminals and servers. Preferably, the electronic device is an electronic device including an FPGA.

[0271] This application also provides a computer-readable storage medium, such as a memory including program code, which can be executed by a processor to perform the image clustering method described in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0272] This application also provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. A processor of an electronic device reads the program code from the computer-readable storage medium and executes the program code to complete the method steps of the image clustering method provided in the above embodiments.

[0273] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0274] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0275] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations as described above in different aspects of this application, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image clustering method, characterized in that, The method includes: Obtain the pixels of each frame of the image; If a pixel is determined to be a valid pixel based on its grayscale value, then coarse clustering is performed on the image to obtain coarse clustering results. Each coarse clustering result corresponds to a clustering number. Valid pixels include pixels with grayscale values ​​greater than the high-pass filter threshold. Based on the clustering number and the first command, the coarse clustering result is subjected to fine clustering to obtain the fine clustering result, wherein the first command is an indication of the amount of data read by the fine clustering module from the coarse clustering module; If a pixel is determined to be a valid pixel based on its grayscale value, then coarse clustering is performed on the image to obtain coarse clustering results, including: Determine whether each of the aforementioned valid pixels is the first valid pixel in a frame of an image; If the effective pixel is the first effective pixel in a frame of an image, then the first clustering region is determined with the first effective pixel as the center. If the effective pixel is not the first effective pixel in a frame of an image, then determine whether the pixel coordinates of the effective pixel are within the first clustering region. If the pixel coordinates of the effective pixel are within the first clustering region, then the effective pixel is assigned to the first clustering region. After the image traversal is completed, the first coarse clustering result composed of all valid pixels in the first clustering region is cached in the first clustering module; If the pixel is determined to be a valid pixel based on its grayscale value, then performing coarse clustering on the image to obtain coarse clustering results further includes: If the pixel coordinates of the effective pixel are not within the first clustering region, then a second clustering region is determined with the effective pixel as the center, wherein the second clustering region is different from the first clustering region; After the image traversal is completed, the second coarse clustering result, composed of all valid pixels within the second clustering region, is cached in the second clustering module.

2. The method according to claim 1, characterized in that, The step of performing fine clustering on the coarse clustering result according to the clustering number and the first command to obtain the fine clustering result includes: Based on the clustering number and the first command, the coarse clustering results of the corresponding data volume are read in parallel in different clustering modules; The image growth operation is performed on each of the coarse clustering results by a region growing algorithm to obtain the fine clustering result corresponding to each coarse clustering result, wherein each coarse clustering result and the corresponding fine clustering result have the same cluster number.

3. The method according to claim 2, characterized in that, The step of performing image growth operations on each of the coarse clustering results using a region growing algorithm to obtain the fine clustering result corresponding to each coarse clustering result includes: Starting from the first pixel of each coarse clustering result, traverse each coarse clustering result. Each time the coarse clustering result is traversed, the step size of the fine clustering side length is increased by one pixel. When the pixel coordinates of all pixels in the coarse clustering result are outside the growth coordinate range, the fine clustering operation ends, and the fine clustering result corresponding to each coarse clustering result is obtained, wherein the growth coordinate range is determined by the fine clustering side length.

4. The method according to any one of claims 1 or 3, characterized in that, The method further includes: According to the transmission sequence, the high-pass filtered image data is written into two first-in-first-out queues in sequence, forming three rows of data; For each of the three rows of data, a data sliding window is used to determine the edge attributes of each pixel in the three rows of data; During the process of caching each coarse clustering result to each clustering module, the edge attribute is placed in the highest bit of the pixel.

5. The method according to claim 4, characterized in that, The method further includes: When reading coarse clustering results of corresponding data volume in parallel in different clustering modules, edge attribute judgment is performed on the highest bit of each pixel point read, and the side length of each pixel point is normalized. If the pixel is an edge point, then the side length of the pixel is included in the perimeter. If the pixel is an interior point, the perimeter remains unchanged, where the perimeter represents the number of edge points; The perimeter includes: in, Indicates the perimeter. Indicates an edge point.

6. The method according to claim 5, characterized in that, Each of the refined clustering results corresponds to an image area and a perimeter, and the method further includes: The area of ​​each pixel is normalized, and the number of pixels in each fine clustering result is counted as the image area corresponding to each fine clustering result. The circularity of each fine clustering result is calculated based on the perimeter and the image area. If the roundness of the refined clustering result satisfies the first condition, then the refined clustering result is output; The formula for calculating the roundness includes: in, Indicates roundness, Represents the area of ​​the image. Indicates the perimeter.

7. The method according to claim 6, characterized in that, The calculation of the roundness of each refined clustering result based on the perimeter and the image area includes: The reciprocal of the square of the perimeter is cached in memory according to a preset method; Using the square of the actual perimeter as the read address, read the data in the memory to obtain the reciprocal of the square of the actual perimeter; Substituting the reciprocal of the square of the actual perimeter into the formula for calculating the roundness, the roundness of the refined clustering result is calculated.

8. An electronic device, characterized in that, include: At least one processor, and The memory communicatively connected to the at least one processor, wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the image clustering method according to any one of claims 1-7.

Images