Interactive chip image segmentation method, device, computing equipment and program product

Abstract

The application discloses an interactive chip image segmentation method and device, a computing device and a program product. The method comprises the following steps: in response to an interactive operation performed on a chip image, determining a plurality of specified segmentation regions in the chip image according to the interactive operation, and assigning a corresponding label to each specified segmentation region; using an SLIC superpixel segmentation algorithm, segmenting a plurality of pixels in the chip image based on a plurality of superpixel initial clustering values to obtain superpixel features of a plurality of scale layers; determining a label probability vector of the superpixel in the scale layer as a first label probability vector of each pixel contained in the superpixel in the scale layer; averaging and fusing the first label probability vectors of the pixels in each scale layer to obtain a target label probability vector of the pixel; and determining a corresponding label of each pixel based on the target label probability vector of each pixel, so as to segment the chip image into a plurality of target segmentation regions. The application can improve the image segmentation efficiency and segmentation precision.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to an interactive chip image segmentation method, an interactive chip image segmentation device, a computing device, and a program product. Background Technology

[0002] With the rapid development of the LED industry, manual inspection of LED chips is inefficient and lacks accuracy, failing to meet the requirements of LED product manufacturing. Currently, automated LED chip inspection technology is commonly used in LED product manufacturing. LED chip image segmentation is a crucial step in automated LED chip inspection.

[0003] Image segmentation, a fundamental technique in computer vision and image processing, aims to divide an image into several non-overlapping regions to facilitate further analysis and processing. Interactive image segmentation is a key technology in computer vision, allowing users to participate in the image segmentation process interactively, thereby improving the accuracy and efficiency of segmentation.

[0004] Existing LED chip image segmentation algorithms have the following defects and limitations: the user interaction process is cumbersome and time-consuming, especially for complex images or scenarios requiring fine operation; the application scenarios are relatively limited, the segmentation accuracy for complex chip images is low, and the robustness is poor.

[0005] Therefore, an interactive chip image segmentation method is needed to solve the problems existing in the above technical solutions. Summary of the Invention

[0006] Therefore, the present invention provides an interactive chip image segmentation method and apparatus to solve or at least alleviate the problems mentioned above.

[0007] According to one aspect of the present invention, an interactive chip image segmentation method is provided, executed in a computing device, comprising: acquiring a chip image to be segmented, the chip image including a plurality of pixels; responding to an interactive operation performed on the chip image, determining a plurality of specified segmentation regions in the chip image according to the interactive operation, assigning a corresponding label to each specified segmentation region, the interactive operation including a swiping drawing operation and / or a region selection operation; and using the SLIC superpixel segmentation algorithm to segment the plurality of pixels in the chip image based on a plurality of initial superpixel clustering values ​​to obtain superpixel features of a plurality of scale layers corresponding to the plurality of initial superpixel clustering values, the superpixel features of the scale layers including the superpixel size in the scale layer and the scale. Each superpixel in the scale layer contains the RGB value of each pixel; for each scale layer, a label probability vector for each superpixel in the scale layer is determined, and the label probability vector of the superpixel is used as the first label probability vector of each pixel contained in the superpixel in the scale layer, wherein the label probability vector of the superpixel is used to represent the probability that the superpixel belongs to each label; for each pixel in the chip image, the first label probability vectors of the pixel in each scale layer are averaged and fused to obtain the target label probability vector of the pixel; based on the target label probability vector of each pixel, the label corresponding to each pixel is determined so as to segment the chip image into multiple target segmentation regions corresponding to multiple labels.

[0008] Optionally, in the interactive chip image segmentation method according to the present invention, for each scale layer, determining the label probability vector of each superpixel in the scale layer includes: for each scale layer, extracting multiple superpixels contained in each specified segmentation region from the scale layer; clustering the multiple superpixels contained in each specified segmentation region using the Fuzzy C-means algorithm to obtain a total of C cluster centers of the scale layer; for each superpixel in the scale layer, calculating the similarity between the superpixel and each cluster center of the scale layer to obtain the label prior value of the superpixel; and determining the label probability vector of each superpixel in the scale layer based on the label prior value of each superpixel in the scale layer.

[0009] Optionally, in the interactive chip image segmentation method according to the present invention, clustering multiple superpixels contained in each specified segmentation region using the Fuzzy C-means algorithm includes: for each superpixel contained in each specified segmentation region, calculating the average RGB value of each pixel contained in the superpixel as the first feature value of the superpixel; and clustering the first feature values ​​of multiple superpixels contained in each specified segmentation region using the Fuzzy C-means algorithm.

[0010] Optionally, in the interactive chip image segmentation method according to the present invention, determining the label probability vector of each superpixel in the scale layer based on the label prior value of each superpixel in the scale layer includes: determining the similarity of the scale layer based on the second feature value of any superpixel in the scale layer and the second feature value of the neighboring superpixels of any superpixel, wherein the second feature value is used to represent the RGB value of the superpixel center; constructing an energy function based on the label prior value of each superpixel in the scale layer and the similarity of the scale layer, wherein the energy function is used to describe the relationship between superpixels and labels in the scale layer; obtaining a closed-form solution by minimizing the energy function, and determining the label probability vector of each superpixel in the scale layer based on the closed-form solution and the label prior value of each superpixel in the scale layer.

[0011] Optionally, in the interactive chip image segmentation method according to the present invention, for each superpixel in the scale layer, the similarity between the superpixel and each cluster center is calculated to obtain the label prior value of the superpixel, including: for each superpixel in the scale layer, the normalized Euclidean geometric distance between the feature vector of the superpixel and each cluster center is calculated to obtain the label prior value of the superpixel.

[0012] Optionally, in the interactive chip image segmentation method according to the present invention, the initial clustering value of the superpixels includes the maximum color distance between each superpixel and the maximum spatial distance between each superpixel.

[0013] Optionally, in the interactive chip image segmentation method according to the present invention, for each scale layer, determining the label probability vector of each superpixel in the scale layer includes: for each scale layer, calculating the average variance of all superpixels in the scale layer, and determining whether the scale layer is effective based on the average variance and a predetermined variance threshold, so as to filter out multiple effective scale layers from the multiple scale layers; for each effective scale layer, determining the label probability vector of each superpixel in the effective scale layer.

[0014] Optionally, in the interactive chip image segmentation method according to the present invention, the chip image is an LED chip image; the plurality of target segmentation regions include an electrode lead region, a lead epitaxial region, a chip substrate region, and a chip epitaxial region.

[0015] According to one aspect of the present invention, an interactive chip image segmentation apparatus is provided, deployed in a computing device. The apparatus includes: an acquisition module adapted to acquire a chip image to be segmented, the chip image including a plurality of pixels; a response module adapted to respond to an interactive operation performed on the chip image, determining a plurality of specified segmentation regions in the chip image based on the interactive operation, and assigning a corresponding label to each specified segmentation region, the interactive operation including a swiping drawing operation and / or a region selection operation; and a superpixel segmentation module adapted to segment the plurality of pixels in the chip image based on a plurality of initial superpixel clustering values ​​using the SLIC superpixel segmentation algorithm, to obtain superpixel features at multiple scale layers corresponding to the plurality of initial superpixel clustering values, the superpixel features at the scale layers including the superpixel size in the scale layer and the... The image image is divided into several modules: a scale layer, a scale layer, and a scale layer, each superpixel in the scale layer, comprising the RGB values ​​of each pixel; a scale layer, a scale layer, which determines the label probability vector of each superpixel in the scale layer and uses the label probability vector of the superpixel as the first label probability vector of each pixel in the scale layer, wherein the label probability vector of the superpixel is used to represent the probability that the superpixel belongs to each label; an average fusion module, which is adapted to average and fuse the first label probability vector of each pixel in each scale layer for each pixel in the chip image to obtain the target label probability vector of the pixel; and an image segmentation module, which is adapted to determine the label corresponding to each pixel based on the target label probability vector of each pixel, so as to segment the chip image into multiple target segmentation regions corresponding to multiple labels.

[0016] According to one aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the interactive chip image segmentation method as described above.

[0017] According to one aspect of the present invention, a computer program product is provided, comprising a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the method as described above.

[0018] According to one aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the interactive chip image segmentation method as described above.

[0019] According to the technical solution of the present invention, an interactive chip image segmentation method is provided. Users can perform sliding doodle operations and / or bounding box operations on the chip image to label multiple specified segmentation regions, each corresponding to a label. The computing device utilizes the SLIC superpixel segmentation algorithm to segment multiple pixels in the chip image based on multiple initial superpixel clustering values, obtaining superpixel features at multiple scale layers corresponding to the initial superpixel clustering values. For each scale layer, the label probability vector of the superpixel in the scale layer is used as the first label probability vector of each pixel contained in that superpixel in the scale layer. By averaging and fusing the first label probability vectors of the pixels in each scale layer, the target label probability vector of the pixel is obtained. Then, based on the target label probability vector of each pixel in the chip image, the label corresponding to each pixel is determined, so as to segment the chip image into multiple target segmentation regions corresponding to multiple labels. According to the technical solution of the present invention, users only need to perform a swipe drawing operation and / or select a region to specify the segmentation region and assign labels. Finally, the image is segmented into multiple target segmentation regions based on multiple labels. This not only simplifies the interactive operation, but also makes the segmentation results more accurate and the boundary precision higher. In addition, using superpixels instead of pixels for calculation can reduce the algorithm complexity and computation time while ensuring segmentation accuracy, thereby improving image segmentation efficiency. By adopting a multi-scale averaging fusion strategy, the robustness and accuracy of image segmentation can be improved, ensuring the consistency and stability of the segmentation results.

[0020] Furthermore, by setting an appropriate predetermined variance threshold to filter out the effective scale layer, and by processing the effective scale layer, the present invention can further improve the segmentation accuracy and ensure the boundary recall rate.

[0021] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0022] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of the invention will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout the invention, the same reference numerals generally refer to the same parts or elements.

[0023] Figure 1 A schematic diagram of a computing device 100 provided according to an embodiment of the present invention is shown;

[0024] Figure 2 A flowchart illustrating an interactive chip image segmentation method 200 provided according to an embodiment of the present invention is shown;

[0025] Figure 3 This illustration shows the effect of a user marking multiple specified segmentation regions on a chip image according to an embodiment of the present invention;

[0026] Figure 4 This diagram illustrates the effect of segmenting a chip image into multiple target segmentation regions according to an embodiment of the present invention.

[0027] Figure 5 A schematic diagram of an interactive chip image segmentation apparatus 500 provided according to an embodiment of the present invention is shown. Detailed Implementation

[0028] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0029] To address the problems of cumbersome user interaction processes, low accuracy, and poor robustness in segmenting complex chip images in existing LED chip image segmentation algorithms, this invention proposes an interactive chip image segmentation method. The interactive operation is simple, and the image segmentation efficiency can be improved by using superpixels instead of pixels for calculation. Furthermore, the robustness and accuracy of image segmentation can be improved by adopting a multi-scale averaging fusion strategy.

[0030] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0031] Figure 1 A schematic diagram of a computing device 100 according to an embodiment of the present invention is shown. Figure 1 As shown, in a basic configuration, computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 may be implemented as a processor. System memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 104 includes an operating system 105.

[0032] According to one aspect, operating system 105 is, for example, suitable for controlling the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1 The basic configuration is illustrated by the components within the dashed lines. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1 The middle part is shown by removable storage device 109 and non-removable storage device 110.

[0033] As stated above, according to one aspect, program module 103 is stored in system memory 104. According to one aspect, program module 103 may include one or more applications. The present invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc.

[0034] According to one aspect, program module 103 may include a plurality of program instructions adapted to execute the interactive chip image segmentation method 200 of the present invention, such that computing device 100 is configured to execute the interactive chip image segmentation method 200 of the present invention.

[0035] According to one aspect, program module 103 may include an interactive chip image segmentation apparatus 500, which may be configured to perform the interactive chip image segmentation method 200 of the present invention.

[0036] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0037] According to one aspect, computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0038] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program module 103). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0039] According to one aspect, the communication medium is implemented by computer-readable instructions, data structures, program modules 103, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, the communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0040] In an embodiment of the present invention, a computing device 100 is configured to execute the interactive chip image segmentation method 200 of the present invention. The computing device 100 includes one or more processors and one or more readable storage media storing program instructions that, when configured to be executed by the one or more processors, cause the computing device to execute the interactive chip image segmentation method 200 of the present invention.

[0041] Figure 2 A schematic flowchart of an interactive chip image segmentation method 200 according to an embodiment of the present invention is shown. The interactive chip image segmentation method 200 can be executed in a computing device (e.g., the aforementioned computing device 100).

[0042] In embodiments of the present invention, the computing device 100 for executing the interactive chip image segmentation method 200 of the present invention may be a terminal or a server.

[0043] like Figure 2 As shown, the interactive chip image segmentation method 200 includes the following steps 210 to 260.

[0044] Step 210: The computing device 100 acquires the chip image to be segmented, the chip image including multiple pixels.

[0045] In some embodiments, the chip image may specifically be an LED chip image.

[0046] In some embodiments, after the computing device 100 acquires the chip image to be segmented, it can preprocess the chip image, such as by denoising and enhancing the contrast. Then, subsequent steps can be performed on the preprocessed chip image.

[0047] Step 220: The computing device 100 can respond to the user's interactive operation on the chip image, determine multiple specified segmentation regions in the chip image based on the interactive operation, and assign a corresponding label to each specified segmentation region.

[0048] In embodiments of the present invention, the interactive operation may include a swiping drawing operation and / or a region selection operation. Specifically, in step 220, the computing device 100 may, in response to the user's swiping drawing operation and / or region selection operation on the chip image, determine multiple specified segmented regions in the chip image based on the swiping drawing operation and / or region selection operation, and assign a corresponding label to each specified segmented region.

[0049] Figure 3 This illustration shows the effect of a user marking multiple specified segmented regions on a chip image according to an embodiment of the present invention. In this embodiment, after the computing device 100 loads the chip image, it can display the chip image to the user. The user can perform interactive operations (slide drawing and / or area selection) on the chip image displayed by the computing device 100 to mark multiple specified segmented regions on the chip image. In step 220, the computing device 100 can respond to the user's interactive operation on the chip image, determine the multiple specified segmented regions marked by the user in the chip image based on the interactive operation, and assign a corresponding label to each specified segmented region.

[0050] Step 230: The computing device 100 can use the SLIC superpixel segmentation algorithm to segment multiple pixels in the chip image based on multiple initial superpixel clustering values, so as to obtain superpixel features of multiple scale layers (multiple different scale layers) corresponding to multiple initial superpixel clustering values.

[0051] It should be noted that the superpixel features of the scale layer include the superpixel size in the scale layer and the RGB value of each pixel contained in each superpixel in the scale layer.

[0052] In embodiments of the present invention, the computing device 100 can pre-set multiple different initial superpixel clustering values. For each initial superpixel clustering value, the computing device 100 can utilize the SLIC superpixel segmentation algorithm to segment multiple pixels in the chip image based on the initial superpixel clustering value, thereby obtaining the superpixel features of the scale layer corresponding to the initial superpixel clustering value. This allows for the acquisition of superpixel features of multiple scale layers corresponding to the multiple initial superpixel clustering values.

[0053] Step 240: The computing device 100 can determine the label probability vector of each superpixel in each scale layer, and use the label probability vector of the superpixel as the first label probability vector of each pixel contained in the superpixel in the scale layer.

[0054] It should be noted that the superpixel's label probability vector represents the probability that the superpixel belongs to each label. The first label probability vector represents the probability that a pixel belongs to each label in the scale layer.

[0055] Step 250: The computing device 100 can perform average fusion (sum and average) on the first label probability vector of each pixel in the chip image at each scale layer to obtain the target label probability vector of the pixel.

[0056] Step 260: The computing device 100 can determine the label corresponding to each pixel based on the target label probability vector of each pixel, and then, based on the label corresponding to each pixel, it can segment the chip image into multiple target segmentation regions corresponding to multiple labels.

[0057] It is understood that, according to the embodiments of the present invention, the number of target segmented regions formed in the final segmentation is consistent with the number of tags.

[0058] In some embodiments, before performing step 240, the computing device 100 may, for each scale layer, calculate the variance of each superpixel in that scale layer, and then calculate the average variance of all superpixels in that scale layer. Based on the average variance of all superpixels in that scale layer and a predetermined variance threshold, it is determined whether the scale layer is valid, thereby filtering out multiple valid scale layers from multiple scale layers. Specifically, it can be determined whether the average variance of all superpixels in that scale layer exceeds the predetermined variance threshold. If it does not exceed the predetermined variance threshold, the scale layer is determined to be valid; if it exceeds the predetermined variance threshold, the scale layer is determined to be invalid. Accordingly, the number of superpixels in each scale layer can be controlled to filter out valid scale layers.

[0059] Furthermore, in step 240, a label probability vector for each superpixel in each effective scale layer can be determined. Subsequent steps are also performed on a per-effective scale layer basis.

[0060] Figure 4 This diagram illustrates the effect of segmenting a chip image into multiple target segmentation regions according to an embodiment of the present invention. Figure 4 As shown, in some embodiments, the chip image can be an LED chip image. Segmenting the chip image (LED chip image) into multiple target segmentation regions based on multiple tags can specifically include an electrode lead region, a lead epitaxial region, a chip substrate region, and a chip epitaxial region.

[0061] In some embodiments, the initial clustering value of superpixels in step 230 includes the maximum color distance and the maximum spatial distance between superpixels. The color distance can be the lab color distance, and the spatial distance can be the xy distance.

[0062] In some embodiments, the distance formula of the SLIC superpixel segmentation algorithm is as follows:

[0063]

[0064] In the formula, d c d represents the color distance between the j-th pixel and the i-th superpixel. s This represents the spatial distance between the j-th pixel and the i-th superpixel. N s N represents the maximum spatial distance between superpixels (i.e., the maximum spatial distance between superpixels formed by segmenting multiple pixels in the chip image). c W represents the maximum color distance between all superpixels (i.e., the maximum color distance between superpixels formed by segmenting multiple pixels in the chip image), and W is the final distance metric. l, a, and b are the three color channels (lab) of the chip image.

[0065] Assuming a chip image contains N pixels, and it is divided into K superpixels, then the size of each superpixel is N / K. We can define N... s =sqrt(N / K) is the maximum spatial distance between superpixels.

[0066] In the SLIC superpixel segmentation algorithm, the distance between each pixel and the superpixel (involving color distance and spatial distance) can be calculated based on the above distance formula to determine the superpixel to which each pixel belongs, so as to segment multiple pixels in the chip image into multiple superpixels of the corresponding scale layer, and thus obtain the superpixel features of the scale layer corresponding to the above superpixel initial clustering value.

[0067] In some embodiments, in step 240, the computing device 100 can determine the label probability vector for each superpixel in the scale layer in the following manner:

[0068] First, the computing device 100 can extract multiple superpixels contained in each specified segmentation region from each scale layer.

[0069] Subsequently, the computing device 100 can use the Fuzzy C-means algorithm (i.e., fuzzy C-means clustering algorithm) to cluster the multiple superpixels contained in each specified segmentation region, obtaining a total of C cluster centers at the scale layer. Each specified segmentation region may contain one or more cluster centers. Specifically, for each superpixel contained in each specified segmentation region, the computing device 100 can calculate the average RGB value of each pixel contained in the superpixel, using the average RGB value of each pixel contained in the superpixel as the first feature value of that superpixel. Then, the Fuzzy C-means algorithm can be used to cluster the first feature values ​​of the multiple superpixels contained in each specified segmentation region, obtaining a total of C cluster centers at the scale layer.

[0070] Next, the computing device 100 can calculate the similarity between each superpixel in the scale layer and each cluster center in the scale layer to obtain the label prior value of the superpixel.

[0071] In some embodiments, for each superpixel in the scale layer, the normalized Euclidean geometric distance between the superpixel (superpixel feature vector) and each cluster center can be calculated to obtain the label prior value of the superpixel.

[0072] The C cluster centers at the h-th scale layer can be represented as:

[0073] The prior label value of the superpixel in the h-th scale layer can be calculated using the following formula:

[0074]

[0075] In the formula, It is the prior label value of the superpixel in the h-th scale layer. It is the feature vector of a superpixel. With cluster center The normalized Euclidean geometric distance between them. N represents the number of superpixels in the h-th scale layer.

[0076] Finally, the computing device 100 can determine the label probability vector for each superpixel in the scale layer based on the label prior value of each superpixel in the scale layer.

[0077] In some embodiments, the computing device 100 determines the label probability vector of each superpixel in the scale layer based on the label prior value of each superpixel in the scale layer in the following specific manner:

[0078] First, the similarity of the scale layer is determined based on the second feature value of any superpixel in the scale layer, and the second feature value of any neighboring superpixel (any neighboring superpixel in the neighborhood set). The second feature value represents the RGB value of the superpixel center.

[0079] Subsequently, an energy function can be constructed based on the prior label value of each superpixel in the scale layer and the similarity of the scale layer. The energy function is used to describe the relationship between superpixels and labels in the scale layer.

[0080] Furthermore, by minimizing the energy function, a closed-form solution is obtained, and based on the closed-form solution, the label probability vector of each superpixel in this scale layer can be determined.

[0081] Specifically, for the h-th scale layer, it can be based on any superpixel p in that scale layer. i The second eigenvalue (superpixel p)i The RGB value of the center and its neighboring superpixels p j The second eigenvalue (neighborhood superpixel p) j The similarity of the h-th scale layer is calculated using the RGB values ​​of the center. Calculate the similarity of the h-th scale layer The formula is as follows:

[0082]

[0083] In the formula, Ψ i For superpixel p i The neighborhood set of δ, where δ is the control constant.

[0084] The energy function F(x) is constructed based on the prior label value of each superpixel in the scale layer and the similarity of that scale layer. l For each scale layer (e.g., the h-th scale layer), the label probability vector of each superpixel in the scale layer can be calculated using the following formula:

[0085]

[0086] In this formula, the second row is a matrix representation of the first row. The first term restricts the boundary relationships, thus mitigating the problem of weak boundaries. The subsequent terms are region terms, which ensure strong internal consistency in the segmentation results.

[0087] It is the probability vector of each superpixel in the scale layer belonging to label l, x il Represents superpixel x i The probability of belonging to label l. λ is a balancing factor used to adjust the importance of the two terms. L = DR is the Laplacian matrix of the scaling layer, where D = diag([d 11 ,...,d NN ])and N is the number of superpixels. R is the similarity matrix of the scale layer, where This represents the prior label value of each superpixel in the scale layer.

[0088] By minimizing the energy function in the above equation, the closed-form solution can be obtained as follows:

[0089]

[0090] Where I is the identity matrix, L r =D -1 R, α = 1 / (λ+1).

[0091] Furthermore, the label probability vector for each superpixel in the scale layer can be determined based on the closed-form solution. Specifically, the label probability vector for each superpixel in the scale layer can be determined based on the closed-form solution and the prior label value of each superpixel in the scale layer.

[0092] Figure 5 A schematic diagram of an interactive chip image segmentation apparatus 500 according to an embodiment of the present invention is shown. The interactive chip image segmentation apparatus 500 can be deployed in a computing device 100, and the interactive chip image segmentation apparatus 500 is configured to perform the interactive chip image segmentation method 200 of the present invention.

[0093] like Figure 5 As shown, in an embodiment of the present invention, the interactive chip image segmentation device 500 includes an acquisition module 510, a response module 520, a superpixel segmentation module 530, a determination module 540, an average fusion module 550, and an image segmentation module 560 that are sequentially connected in communication.

[0094] The acquisition module 510 can acquire the chip image to be segmented, and the chip image includes multiple pixels.

[0095] The response module 520 can respond to user interaction on the chip image, determine multiple specified segmentation regions in the chip image based on the interaction, and assign a corresponding label to each specified segmentation region.

[0096] The superpixel segmentation module 530 can use the SLIC superpixel segmentation algorithm to segment multiple pixels in the chip image based on multiple initial superpixel clustering values ​​to obtain superpixel features of multiple scale layers corresponding to the multiple initial superpixel clustering values. The superpixel features of the scale layer include the superpixel size in the scale layer and the RGB value of each pixel contained in each superpixel in the scale layer.

[0097] The determination module 540 can determine the label probability vector of each superpixel in each scale layer, and use the label probability vector of the superpixel as the first label probability vector of each pixel contained in the superpixel in the scale layer, wherein the label probability vector of the superpixel is used to represent the probability of the superpixel belonging to each label.

[0098] The averaging fusion module 550 can average and fuse the first label probability vector of each pixel in the chip image at each scale layer to obtain the target label probability vector of that pixel.

[0099] The image segmentation module 560 can determine the label corresponding to each pixel based on the target label probability vector of each pixel, so as to segment the chip image into multiple target segmentation regions corresponding to multiple labels.

[0100] It should be noted that the acquisition module 510, response module 520, superpixel segmentation module 530, determination module 540, average fusion module 550, and image segmentation module 560 are respectively used to perform the aforementioned steps 210 to 260. Here, the specific execution logic of each module can be found in the description of steps 210 to 260 in the previous method 200, and will not be repeated here.

[0101] According to the interactive chip image segmentation method 200 of this embodiment, a user can perform swiping and drawing operations and / or bounding box operations on a chip image to label multiple specified segmentation regions, each of which corresponds to a label. The computing device uses the SLIC superpixel segmentation algorithm to segment multiple pixels in the chip image based on multiple initial superpixel clustering values, thereby obtaining superpixel features at multiple scale layers corresponding to the initial superpixel clustering values. For each scale layer, the label probability vector of the superpixel in the scale layer is used as the first label probability vector of each pixel contained in that superpixel in the scale layer. By averaging and fusing the first label probability vectors of the pixels in each scale layer, the target label probability vector of the pixel is obtained. Then, based on the target label probability vector of each pixel in the chip image, the label corresponding to each pixel is determined, so as to segment the chip image into multiple target segmentation regions corresponding to multiple labels. According to the technical solution of the present invention, users only need to perform a swipe drawing operation and / or select a region to specify the segmentation region and assign labels. Finally, the image is segmented into multiple target segmentation regions based on multiple labels. This not only simplifies the interactive operation, but also makes the segmentation results more accurate and the boundary precision higher. In addition, using superpixels instead of pixels for calculation can reduce the algorithm complexity and computation time while ensuring segmentation accuracy, thereby improving image segmentation efficiency. By adopting a multi-scale averaging fusion strategy, the robustness and accuracy of image segmentation can be improved, ensuring the consistency and stability of the segmentation results.

[0102] Furthermore, by setting an appropriate predetermined variance threshold to filter out the effective scale layer, and processing the effective scale layer, the present invention can further improve the segmentation accuracy and ensure the boundary recall rate.

[0103] Furthermore, embodiments of the present invention also disclose: A8, the method as described in any one of A1-A7, wherein the chip image is an LED chip image; the plurality of target segmentation regions include an electrode lead region, a chip substrate region, and a chip epitaxial region. B12, a readable storage medium storing program instructions, which, when read and processed by a computing device, cause the computing device to process the method as described in any one of A1-A8.

[0104] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0105] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the interactive chip image segmentation method of the present invention according to instructions in the program code stored in the memory.

[0106] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0107] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0108] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0109] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the description of exemplary embodiments of the invention above, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof.

[0110] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0111] Unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

Claims

1. An interactive chip image segmentation method, executed in a computing device, comprising: Obtain a chip image to be segmented, the chip image comprising multiple pixels; In response to an interactive operation performed on a chip image, a plurality of specified segmented regions in the chip image are determined based on the interactive operation, and a corresponding label is assigned to each specified segmented region. The interactive operation includes a swiping drawing operation and / or a region selection operation. Using the SLIC superpixel segmentation algorithm, multiple pixels in the chip image are segmented based on multiple initial superpixel clustering values ​​to obtain superpixel features of multiple scale layers corresponding to the multiple initial superpixel clustering values. The superpixel features of the scale layers include the superpixel size in the scale layer and the RGB value of each pixel contained in each superpixel in the scale layer. For each scale layer, the average variance of all superpixels in the scale layer is calculated. Based on the average variance and a predetermined variance threshold, it is determined whether the scale layer is effective, so as to select multiple effective scale layers from the multiple scale layers. For each effective scale layer, multiple superpixels contained in each specified segmentation region are extracted from the effective scale layer, and the multiple superpixels contained in each specified segmentation region are clustered using the Fuzzy C-means algorithm to obtain a total of C cluster centers of the effective scale layer; for each superpixel in the effective scale layer, the similarity between the superpixel and each cluster center of the effective scale layer is calculated to obtain the label prior value of the superpixel; Based on the second feature value of any superpixel in the effective scale layer and the second feature values ​​of its neighboring superpixels, the similarity of the effective scale layer is determined, wherein the second feature value represents the RGB value of the superpixel center; based on the label prior value of each superpixel in the effective scale layer and the similarity of the effective scale layer, an energy function is constructed, wherein the energy function describes the relationship between superpixels and labels in the effective scale layer; by minimizing the energy function, a closed-form solution is obtained, and based on the closed-form solution and the label prior value of each superpixel in the effective scale layer, the label probability vector of each superpixel in the effective scale layer is determined, and the label probability vector of the superpixel is used as the first label probability vector of each pixel contained in the superpixel in the effective scale layer, wherein the label probability vector of the superpixel represents the probability that the superpixel belongs to each label; For each pixel in the chip image, the first label probability vector of the pixel in each of the effective scale layers is averaged and fused to obtain the target label probability vector of the pixel. Based on the target label probability vector of each pixel, the label corresponding to each pixel is determined so as to segment the chip image into multiple target segmentation regions corresponding to multiple labels.

2. The method as described in claim 1, wherein, The Fuzzy C-means algorithm is used to cluster multiple superpixels contained in each specified segmentation region, including: For each superpixel contained in each specified segmentation region, the average RGB value of each pixel contained in the superpixel is calculated as the first feature value of the superpixel; The Fuzzy C-means algorithm is used to cluster the first feature values ​​of multiple superpixels contained in each specified segmentation region.

3. The method as described in claim 1, wherein, For each superpixel in the effective scale layer, the similarity between the superpixel and each cluster center of the effective scale layer is calculated to obtain the prior label value of the superpixel, including: For each superpixel in the effective scale layer, the normalized Euclidean geometric distance between the feature vector of the superpixel and each cluster center of the effective scale layer is calculated to obtain the label prior value of the superpixel.

4. The method according to any one of claims 1-3, wherein, The initial clustering value of the superpixels includes the maximum color distance between each superpixel and the maximum spatial distance between each superpixel.

5. The method according to any one of claims 1-3, wherein, The chip image is an LED chip image; The multiple target segmentation regions include electrode lead regions, chip substrate regions, and chip epitaxial regions.

6. An interactive chip image segmentation apparatus, deployed in a computing device, adapted to perform the method as described in any one of claims 1-5, the apparatus comprising: An acquisition module is adapted to acquire a chip image to be segmented, the chip image comprising multiple pixels; A response module is adapted to respond to interactive operations performed on a chip image, determine multiple specified segmentation regions in the chip image based on the interactive operations, and assign a corresponding label to each specified segmentation region. The interactive operations include swiping and drawing operations and / or selecting regions by drawing boxes. The superpixel segmentation module is suitable for using the SLIC superpixel segmentation algorithm to segment multiple pixels in the chip image based on multiple initial superpixel clustering values, so as to obtain superpixel features of multiple scale layers corresponding to the multiple initial superpixel clustering values. The superpixel features of the scale layers include the superpixel size in the scale layer and the RGB value of each pixel contained in each superpixel in the scale layer. The determining module is adapted to determine, for each scale layer, a label probability vector for each superpixel in the scale layer, and use the label probability vector of the superpixel as the first label probability vector of each pixel contained in the superpixel in the scale layer, wherein the label probability vector of the superpixel is used to represent the probability that the superpixel belongs to each label; The average fusion module is adapted to average and fuse the first label probability vector of each pixel in the chip image at each of the scale layers to obtain the target label probability vector of the pixel. The image segmentation module is adapted to determine the label corresponding to each pixel based on the target label probability vector of each pixel, so as to segment the chip image into multiple target segmentation regions corresponding to multiple labels.

7. A computing device, comprising: At least one processor; and A memory storing program instructions, wherein the program instructions are configured to be processed by the at least one processor, the program instructions including instructions for processing the method as described in any one of claims 1-5.

8. A computer program product comprising a computer program / instructions, wherein, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1-5.

9. A readable storage medium storing program instructions that, when read and processed by a computing device, cause the computing device to perform the method as described in any one of claims 1-5.

Images