Optoelectronic chip image segmentation model training method and apparatus, and computing device

Abstract

The application discloses a photoelectric chip image segmentation model training method, comprising: performing superpixel segmentation on a photoelectric chip sample image and performing superpixel merging to generate a pseudo-labeled image; training a segmentation network based on the photoelectric chip sample image and the pseudo-labeled image to obtain an initial training segmentation model; generating a second sample segmentation probability map based on the photoelectric chip sample image by using the initial training segmentation model to determine the pixel category of each pixel in the photoelectric chip sample image, calculate the second loss value of reliable pixels, and calculate the dense energy loss value of uncertain pixels; and training the initial training segmentation model based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain a photoelectric chip image segmentation model. The application can generate high-quality pseudo-labeled images, effectively suppress the interference of noise labeling on the model training process, and improve the segmentation accuracy of the model on photoelectric chip images.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a method for training an image segmentation model for an optoelectronic chip, an image segmentation device for an optoelectronic chip, and a computing device. Background Technology

[0002] As the core functional unit for achieving efficient electro-optical conversion, optoelectronic chips possess advantages such as high luminous efficiency, long lifespan, low power consumption, and solid-state integration, making them a key fundamental component in modern lighting, display, and optical communication fields. Intelligent development is driving their deep integration with artificial intelligence, giving rise to adaptive intelligent lighting systems and ambient light sensing modules, and propelling functional lighting towards service-oriented intelligent terminals. However, during the manufacturing process, optoelectronic chips are prone to various defects such as cracking, surface flaws, scratches, and missing light-emitting areas. These defects are often distributed across various functional areas, including leads, electrodes, light-emitting areas, and outer edges, and sometimes multiple areas exhibit defects simultaneously. Therefore, accurate detection of defects in various areas of the optoelectronic chip is crucial for ensuring its quality. However, current detection methods are insufficient to efficiently address the complexity of these defects.

[0003] Existing inspection technologies have significant limitations in optoelectronic chip defect detection. For example, traditional non-destructive testing techniques such as X-ray inspection and ultrasonic testing are either difficult to integrate into production lines due to stringent sensor installation requirements, or inefficient due to the inability to generate visual records and reliance on manual judgment. Magnetic particle inspection is only applicable to ferromagnetic materials and cannot be applied to optoelectronic chips. Furthermore, in machine vision-based defect detection solutions, mainstream image segmentation models rely on large amounts of full-density labeled data. Acquiring such full-density labeled data is costly and time-consuming, especially challenging in labeling small functional areas. Existing pseudo-labeling techniques generate labels that are prone to noise, which can mislead model learning. At the same time, existing models have poor adaptability to the features of multi-configuration optoelectronic chips, resulting in low segmentation accuracy.

[0004] Therefore, a training method for an image segmentation model of an optoelectronic chip is needed to solve the problems existing in the above-mentioned technical solutions. Summary of the Invention

[0005] Therefore, the present invention provides a method and apparatus for training an image segmentation model for an optoelectronic chip, in order to solve or at least alleviate the problems mentioned above.

[0006] According to one aspect of the present invention, a method for training an image segmentation model for an optoelectronic chip is provided, executed in a computing device, comprising: acquiring multiple optoelectronic chip sample images covering various configurations of optoelectronic chips; for each optoelectronic chip sample image, performing superpixel segmentation on the optoelectronic chip sample image to obtain an optoelectronic chip superpixel segmentation image containing multiple superpixels; performing superpixel merging on the optoelectronic chip superpixel segmentation image to generate a pseudo-annotated image, the pseudo-annotated image containing key structural edges of multiple semantic regions for indicating noise annotations of each pixel in the optoelectronic chip sample image, the multiple semantic regions including electrode and lead regions, light-emitting regions, outer edge regions, and background regions; training a segmentation network based on the optoelectronic chip sample images and the pseudo-annotated image to obtain an initial trained segmentation model; and using the initial trained segmentation model to generate a second sample segmentation based on the optoelectronic chip sample images. The probability map includes a second sample segmentation probability map containing second probability prediction values ​​for each pixel in the optoelectronic chip sample image belonging to each semantic region. Based on the second sample segmentation probability map, the pixel category of each pixel in the optoelectronic chip sample image is determined, and the pixel category includes reliable pixels and uncertain pixels. For each reliable pixel, a second loss value is calculated between the second probability prediction value of the reliable pixel and the corresponding noise label in the pseudo-labeled image. For each uncertain pixel, a dense energy loss value is calculated based on the color information and spatial location information of the uncertain pixel. Based on the second loss values ​​of each reliable pixel and the dense energy loss values ​​of each uncertain pixel, the initial training segmentation model is trained to obtain an optoelectronic chip image segmentation model, which is used to generate a target segmentation probability map based on the target image of the optoelectronic chip to be detected.

[0007] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, training the initial training segmentation model based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain the photoelectric chip image segmentation model includes: using a joint loss function to perform weighted fusion of the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain a fused loss value; and training the initial training segmentation model based on the fused loss value to obtain the photoelectric chip image segmentation model.

[0008] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, determining the pixel category of each pixel in the photoelectric chip sample image based on the second sample segmentation probability map includes: determining each pixel in the second sample segmentation probability map whose second probability prediction value is greater than the reliability threshold as a reliable pixel, and determining each pixel in the second sample segmentation probability map whose second probability prediction value is less than or equal to the reliability threshold as an uncertain pixel.

[0009] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, the segmentation network is trained based on the photoelectric chip sample image and the pseudo-labeled image to obtain an initial training segmentation model, including: using the segmentation network to generate a first sample segmentation probability map based on the photoelectric chip sample image, the first sample segmentation probability map containing a first probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region; calculating a first loss value between the first probability prediction value of each pixel belonging to each semantic region and the noise label corresponding to the pseudo-labeled image; and training the segmentation network based on the first loss value of each pixel to obtain an initial training segmentation model.

[0010] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, the segmentation network adopts an improved Trans-Unet architecture, including an input layer, an encoder, a bottleneck layer, a decoder, and an output layer coupled sequentially, and a multi-scale attention module is added to the skip connections between each layer in the encoder and the corresponding layer in the decoder; using the segmentation network, a first sample segmentation probability map is generated based on the photoelectric chip sample image, including: performing preprocessing operations on the photoelectric chip sample image through the input layer to obtain a tensor format image, wherein the preprocessing operations include normalization operations, size alignment operations, and data augmentation operations, and the data augmentation operations include horizontal and vertical flipping and random rotation operations; and performing step-by-step processing on the tensor format image through the encoder. The encoder performs step-by-step downsampling to extract low-level features, and captures semantic information based on these low-level features to generate high-level semantic features. The low-level features include edge features and texture features. Each multi-scale attention module performs attention processing on the output features of each layer in the encoder to obtain attention features for each layer, and then passes these attention features to the corresponding layer in the decoder. The high-level semantic features are fused through the bottleneck layer to obtain fused semantic features. The decoder then progressively upsamples the fused semantic features to obtain upsampled features, and concatenates these upsampled features with the attention features of each layer to obtain a first Logits value. Finally, the output layer uses an activation function to convert the first Logits value into a first sample segmentation probability map.

[0011] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, superpixel merging is performed on the superpixel segmented image of the photoelectric chip to generate a pseudo-labeled image, including: extracting the core features of each superpixel in the superpixel segmented image of the photoelectric chip, the core features including color features and spatial features; determining the similarity between the core features of each superpixel, and selecting multiple superpixel pairs with similarity higher than a similarity threshold to form multiple superpixel candidate groups; determining the semantic region to which each superpixel belongs based on the core features of each superpixel; and determining each superpixel candidate based on the semantic region to which each superpixel belongs. The coverage of each semantic region is determined by grouping the candidate superpixels. The coverage is used to represent the ratio of the number of superpixels belonging to the semantic region in the candidate superpixel group to the total number of superpixels in the semantic region. For each semantic region, multiple candidate superpixel groups with the largest coverage of the semantic region and no overlap are selected from the multiple candidate superpixel groups and merged to obtain multiple new superpixels. The above steps are repeated for each new superpixel until no new superpixel pairs with similarity higher than the similarity threshold are selected, no new candidate superpixel groups are formed, or the largest coverage set that completely covers all semantic regions and has no overlap is obtained. At this point, a pseudo-annotated image is generated.

[0012] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, determining the similarity between the core features of each superpixel includes: constructing feature vectors of each superpixel based on the core features of each superpixel; and calculating the similarity between the feature vectors of each superpixel as the similarity between the core features of each superpixel.

[0013] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, superpixel segmentation is performed on the photoelectric chip sample image to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels, including: using the SLIC superpixel segmentation algorithm, combined with the f_max path cost function and the ROOT arc cost function, to perform superpixel segmentation on each pixel in the photoelectric chip sample image to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels.

[0014] Optionally, in the photoelectric chip image segmentation model training method according to the present invention, calculating a first probability prediction value between each pixel belonging to each semantic region and a corresponding noise label in the pseudo-labeled image includes: using a cross-entropy loss function to calculate a first cross-entropy loss value between the first probability prediction value between each pixel belonging to each semantic region and a corresponding noise label in the pseudo-labeled image, as the first loss value; calculating a second probability prediction value between a reliable pixel and a corresponding noise label in the pseudo-labeled image includes: using a cross-entropy loss function to calculate a second cross-entropy loss value between the second probability prediction value between the reliable pixel and a corresponding noise label in the pseudo-labeled image, as the second loss value.

[0015] According to one aspect of the present invention, a photoelectric chip image segmentation apparatus is provided, deployed in a computing device, the apparatus comprising: The acquisition unit is suitable for acquiring multiple optoelectronic chip sample images covering various configurations of optoelectronic chips; The superpixel segmentation unit is adapted to perform superpixel segmentation on each of the photoelectric chip sample images to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels. The superpixel merging unit is adapted to merge superpixel segments of the optoelectronic chip to generate a pseudo-annotated image. The pseudo-annotated image contains key structural edges of multiple semantic regions to indicate noise annotations of each pixel in the optoelectronic chip sample image. The multiple semantic regions include electrode and lead regions, light-emitting regions, outer edge regions, and background regions. The first training unit is adapted to train the segmentation network based on the photoelectric chip sample image and the pseudo-labeled image to obtain an initial training segmentation model. The second training unit is adapted to use the initial training segmentation model to generate a second sample segmentation probability map based on the photoelectric chip sample image. The second sample segmentation probability map includes a second probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region. Based on the second sample segmentation probability map, the pixel category of each pixel in the photoelectric chip sample image is determined, and the pixel category includes reliable pixels and uncertain pixels. For each reliable pixel, a second loss value is calculated between the second probability prediction value of the reliable pixel and the noise label corresponding to the pseudo-labeled image. For each uncertain pixel, a dense energy loss value of the uncertain pixel is calculated based on the color information and spatial location information of the uncertain pixel. Based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel, the initial training segmentation model is trained to obtain a photoelectric chip image segmentation model. The photoelectric chip image segmentation model is used to generate a target segmentation probability map based on the photoelectric chip target image to be detected.

[0016] According to one aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the photoelectric chip image segmentation model training method as described above.

[0017] According to one aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method as described above.

[0018] According to one aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the photoelectric chip image segmentation model training method as described above.

[0019] According to the technical solution of the present invention, a method for training a photoelectric chip image segmentation model is provided. By performing superpixel segmentation on sample images of photoelectric chips with various configurations and combining it with a superpixel merging method, high-quality pseudo-annotated images with low noise can be generated, thereby reducing the cost of acquiring annotation data. Based on the initial training segmentation model obtained from the photoelectric chip sample images and pseudo-annotated images, the reliability of pixels is further judged through the initial training segmentation model and noisy supervised training is performed. This effectively suppresses the interference of noise annotations in pseudo-annotated images on the model training process, enhances the robustness and segmentation accuracy of the model in the photoelectric chip segmentation task, and enables the finally trained photoelectric chip image segmentation model to accurately segment each region of the target image of the photoelectric chip to be detected, thereby achieving accurate and efficient detection of defects in each region of the photoelectric chip. Moreover, the photoelectric chip image segmentation model trained according to the present invention is applicable to the segmentation of photoelectric chip images with various configurations, thus adapting to the detection requirements of photoelectric chips with different configurations.

[0020] Furthermore, by adding a multi-scale attention module to the skip connection between the encoder and decoder, the segmentation accuracy of the optoelectronic chip image can be further improved.

[0021] Furthermore, during the initial training and noisy supervised training processes, data augmentation operations are performed on the photoelectric chip sample images through the input layer. This can enhance the diversity of sample data, avoid overfitting of the trained model, and improve the generalization ability of the final trained photoelectric chip image segmentation model.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0023] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of the invention will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout the invention, the same reference numerals generally refer to the same parts or elements.

[0024] Figure 1 A schematic diagram of a computing device 100 provided according to an embodiment of the present invention is shown; Figure 2 A flowchart illustrating a photoelectric chip image segmentation model training method 200 provided according to an embodiment of the present invention is shown. Figure 3 A schematic diagram showing sample images of optoelectronic chips with various configurations according to some embodiments of the present invention is provided. Figure 4 A schematic diagram of superpixel segmentation images of an optoelectronic chip with various configurations according to some embodiments of the present invention is shown; Figure 5 A schematic diagram of a pseudo-annotated image generated according to some embodiments of the present invention is shown; Figure 6 A schematic diagram of the structure of a segmentation network 600 according to some embodiments of the present invention is shown; Figure 7 A schematic diagram of pixel-level segmentation probability maps corresponding to optoelectronic chip images of different configurations according to some embodiments of the present invention is shown; Figure 8 A schematic diagram of an optoelectronic chip image segmentation apparatus 800 provided according to an embodiment of the present invention is shown. Detailed Implementation

[0025] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0026] To address the problems of poor generalization ability, low segmentation accuracy of optoelectronic chip images, high cost of acquiring full-density labeled data that the models rely on, and noise interference in labels generated by pseudo-labeling techniques, this invention proposes a training method for optoelectronic chip image segmentation models. By combining superpixel segmentation with superpixel merging, high-quality pseudo-labeled images with less noise can be generated, thereby reducing the cost of acquiring labeled data. Based on the initial training of the segmentation model, the reliability of pixels is judged and noisy supervised training is performed, which can effectively suppress the interference of noise labels in pseudo-labeled images on the model training process. This enables the finally trained model to accurately segment each region of the optoelectronic chip image, thereby achieving accurate and efficient detection of defects in each region of the optoelectronic chip.

[0027] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] Figure 1 A schematic diagram of a computing device 100 according to an embodiment of the present invention is shown. Figure 1As shown, in a basic configuration, computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 may be implemented as a processor. System memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 104 includes an operating system 105.

[0029] According to one aspect, operating system 105 is, for example, suitable for controlling the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1 The basic configuration is illustrated by the components within the dashed lines. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1 The middle part is shown by removable storage device 109 and non-removable storage device 110.

[0030] As stated above, according to one aspect, program module 103 is stored in system memory 104. According to one aspect, program module 103 may include one or more applications. The present invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc.

[0031] According to one aspect, program module 103 may include a plurality of program instructions adapted to execute the photoelectric chip image segmentation model training method 200 of the present invention, such that computing device 100 is configured to execute the photoelectric chip image segmentation model training method 200 of the present invention.

[0032] According to one aspect, program module 103 may include an optoelectronic chip image segmentation device 800, which may be configured to perform the optoelectronic chip image segmentation model training method 200 of the present invention.

[0033] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0034] According to one aspect, computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0035] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program module 103). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0036] According to one aspect, the communication medium is implemented by computer-readable instructions, data structures, program modules 103, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, the communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0037] In an embodiment of the present invention, a computing device 100 is configured to execute the photoelectric chip image segmentation model training method 200 of the present invention. The computing device 100 includes one or more processors and one or more readable storage media storing program instructions, which, when configured to be executed by the one or more processors, cause the computing device to execute the photoelectric chip image segmentation model training method 200 of the present invention.

[0038] Figure 2 A schematic flowchart of a photoelectric chip image segmentation model training method 200 according to an embodiment of the present invention is shown. The photoelectric chip image segmentation model training method 200 can be executed in a computing device (such as the aforementioned computing device 100).

[0039] In an embodiment of the present invention, the computing device 100 for executing the photoelectric chip image segmentation model training method 200 of the present invention may be a terminal or a server.

[0040] like Figure 2 As shown, the photoelectric chip image segmentation model training method 200 includes the following steps 210-280.

[0041] Step 210: Obtain multiple optoelectronic chip sample images covering various configurations of optoelectronic chips.

[0042] Figure 3 Schematic diagrams of sample images of optoelectronic chips with various configurations according to some embodiments of the present invention are shown. For example... Figure 3 As shown, each type of optoelectronic chip (optoelectronic chip sample image) can contain multiple functional regions, which can include electrode and lead regions, light-emitting regions, and outer edge regions.

[0043] Step 220: For each photoelectric chip sample image, superpixel segmentation can be performed on the photoelectric chip sample image to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels.

[0044] Specifically, the SLIC algorithm can be used to perform superpixel segmentation on the photoelectric chip sample image to obtain a superpixel segmented image of the photoelectric chip containing multiple superpixels. Figure 4 A schematic diagram of superpixel segmentation images of an optoelectronic chip with various configurations according to some embodiments of the present invention is shown.

[0045] Step 230: Perform superpixel merging on the superpixel segmented image of the optoelectronic chip to generate a pseudo-annotated image (to replace the traditional manually annotated label image).

[0046] According to the embodiments of the present invention, the pseudo-annotated image generated based on the superpixel segmentation combined with the superpixel merging method has a low noise ratio, which can ensure the consistency of the annotation of various configuration optoelectronic chip images, and can significantly reduce the time cost of manual annotation of various configuration optoelectronic chip images.

[0047] Figure 5 A schematic diagram of a pseudo-annotated image generated according to some embodiments of the present invention is shown. See also Figure 5 The pseudo-annotated image retains the key structural edges of multiple semantic regions in the optoelectronic chip sample image with a low noise ratio. These key structural edges can be used to indicate noise annotations for each pixel in the optoelectronic chip sample image. Thus, this invention generates pseudo-annotated images based on a superpixel segmentation combined with superpixel merging method. This significantly reduces the time cost of manual annotation for optoelectronic chip images of various configurations while ensuring consistency in annotation across different configurations. It should be understood that errors are inevitable when generating pseudo-annotated images; therefore, the annotations for each pixel obtained from the pseudo-annotated image are noisy annotations (i.e., noisy semantic region annotations).

[0048] It should be noted that the multiple semantic regions in the pseudo-annotated image cover multiple functional regions of the optoelectronic chip. Specifically, the multiple semantic regions include the electrode and lead region, the light-emitting region, the outer edge region, and the background region.

[0049] Step 240: Train the pre-built segmentation network based on the photoelectric chip sample image and the pseudo-annotated image (the pseudo-annotated image is used as supervision information) to obtain the initial trained segmentation model.

[0050] In step 240, the photoelectric chip sample image and the pseudo-annotated image can be input into a pre-constructed segmentation network. The segmentation network can generate a first sample segmentation probability map based on the photoelectric chip sample image. Here, the first sample segmentation probability map contains the first probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region. Furthermore, a first loss value can be calculated between the first probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region and the noise annotation corresponding to each pixel in the pseudo-annotated image. The segmentation network is trained based on the first loss value of each pixel to obtain an initial trained segmentation model.

[0051] In some embodiments, the cross-entropy loss function can be used to calculate the first cross-entropy loss value between the first probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region and the noise label corresponding to each pixel in the pseudo-labeled image, as the first loss value.

[0052] It is worth noting that, in order to reduce the interference and negative impact of noisy annotations in pseudo-annotated images on the model training process, in this embodiment of the invention, after obtaining the initial training segmentation model, noisy supervised training can be performed on the initial training segmentation model based on a joint loss function training strategy. For example, the computing device 100 can obtain the optimal weights of the initial training segmentation model, initialize the segmentation network based on the optimal weights to obtain the initial training segmentation model, and then perform noisy supervised training on the initial training segmentation model.

[0053] The noisy supervised training process specifically includes the following steps 250~280.

[0054] Step 250: Using the initially trained segmentation model, generate a second sample segmentation probability map based on the photoelectric chip sample image. The second sample segmentation probability map contains the second probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region.

[0055] Step 260: Based on the second sample segmentation probability map, determine the pixel category of each pixel in the photoelectric chip sample image. The pixel category includes reliable pixels and uncertain pixels.

[0056] In some embodiments, in step 260, pixels in the second sample segmentation probability map whose second probability prediction value is greater than the reliability threshold can be determined as reliable pixels, and pixels in the second sample segmentation probability map whose second probability prediction value is less than or equal to the reliability threshold can be determined as uncertain pixels. For example, according to the above method of determining pixel categories, pixels at the edge of the luminescent region in the photoelectric chip sample image and pixels in the transition region between any two semantic regions can be determined as uncertain pixels.

[0057] In one specific embodiment, the reliability threshold may be set to 0.7, but the present invention is not limited to the specific value of the reliability threshold.

[0058] Step 270: For each reliable pixel in the photoelectric chip sample image, a second loss value can be calculated between the second probability prediction value of the reliable pixel and the corresponding noise label (corresponding noise probability true value) in the pseudo-labeled image. Also, for each uncertain pixel in the photoelectric chip sample image, the color information (RGB value) and spatial location information of the uncertain pixel can be obtained, and the dense energy loss value of the uncertain pixel can be calculated based on the color information (RGB value) and spatial location information of the uncertain pixel.

[0059] Step 280: Train the initial training segmentation model (noisy supervised training) based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain the optoelectronic chip image segmentation model.

[0060] The optoelectronic chip image segmentation model trained according to embodiments of the present invention can accurately segment the target image of the optoelectronic chip to be detected. Specifically, it can generate a target segmentation probability map based on the target image of the optoelectronic chip to be detected, and then generate a target segmentation mask image based on the target segmentation probability map, thereby laying the foundation for efficient detection of defects in various regions of the optoelectronic chip and achieving accurate and efficient detection of defects in various regions of the optoelectronic chip. Moreover, the optoelectronic chip image segmentation model trained according to embodiments of the present invention is applicable to the segmentation of optoelectronic chip images of various configurations, thereby adapting to the detection requirements of optoelectronic chips with different configurations.

[0061] In some embodiments, in step 270, the cross-entropy loss function can be used to calculate the second cross-entropy loss value between the second probability prediction value of the reliable pixel and the corresponding noise label (the corresponding noise probability true value) in the pseudo-labeled image, as the second loss value.

[0062] In some embodiments, in step 280, when training the initial training segmentation model based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel, the joint loss function can be used to perform weighted fusion of the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain a fused loss value. Then, the initial training segmentation model is trained based on the fused loss value to obtain the optoelectronic chip image segmentation model.

[0063] In some embodiments, the Adam optimizer (with an initial learning rate of 1e-4) can be used to iteratively optimize the model during both the initial training of the segmentation network (initial training) and the noisy supervised training of the initially trained segmentation model.

[0064] In some embodiments, the segmentation network employs an improved Trans-Unet architecture. Figure 6 A schematic diagram of the structure of a segmentation network 600 according to some embodiments of the present invention is shown.

[0065] like Figure 6 As shown, the segmentation network 600 includes an input layer, an encoder, a bottleneck layer, a decoder, and an output layer, all coupled sequentially. The encoder and decoder each contain three convolutional layers and one Transformer layer. Furthermore, a multi-scale attention module is added to the skip connections between each layer in the encoder and the corresponding layer in the decoder. By adding the multi-scale attention module, the output features of each layer in the encoder can be processed with attention and then passed to the corresponding layer in the decoder. This not only alleviates the vanishing gradient problem in deep networks to stabilize training but also supplements spatial details to improve segmentation and localization accuracy, achieving the fusion of semantic and localization information.

[0066] During the training of the segmentation network 600, the specific process by which the segmentation network 600 generates the first sample segmentation probability map based on the photoelectric chip sample image is as follows: First, the photoelectric chip sample images are preprocessed through the input layer to obtain tensor format images. The preprocessing operations include normalization, size alignment, and data augmentation. Data augmentation operations can include, for example, horizontal and vertical flipping and random rotation. That is, the photoelectric chip sample images can be horizontally and vertically flipped and randomly rotated to obtain multiple photoelectric chip sample images in different orientations based on each sample image, thereby enhancing the diversity of sample data, avoiding overfitting of the trained model, and improving the generalization ability of the trained model. Furthermore, data augmentation operations may also include brightness adjustment.

[0067] Subsequently, the tensor-format image is progressively downsampled by an encoder to extract low-level features (including edge and texture features), and semantic information is captured based on these low-level features to generate high-level semantic features. Specifically, the encoder can extract low-level features through various convolutional layers, and then capture semantic information based on these low-level features through Transformer layers to generate high-level semantic features, while also expanding the channel dimension to enhance feature representation.

[0068] Simultaneously, during the progressive downsampling process of the encoder, the output features of each layer in the encoder can be processed by the corresponding multi-scale attention module to obtain the attention features of each layer, and then the attention features of each layer are passed to the corresponding layer in the decoder. In other words, attention features of each layer can be obtained by processing the output features of each layer in the encoder separately through each multi-scale attention module, and then the attention features of each layer are passed to the corresponding layer in the decoder.

[0069] The high-level semantic features output by the encoder can be input into the bottleneck layer. The bottleneck layer can fuse the high-level semantic features and condense the global context information to obtain fused semantic features.

[0070] Then, the decoder progressively upsamples the fused semantic features (restoring spatial resolution and reducing channel dimension) to obtain upsampled features, and concatenates the upsampled features with the attention features of each layer to supplement spatial detail features and improve segmentation and localization accuracy, so as to obtain the first Logits value.

[0071] Finally, the first Logits value can be converted into a first sample segmentation probability map by using an activation function in the output layer.

[0072] It should be understood that during the noisy supervised training of the initial training segmentation model, the process of generating the second sample segmentation probability map based on the photoelectric chip sample image is similar to the above process, and will not be repeated here.

[0073] It should also be noted that the first sample segmentation probability map, the second sample segmentation probability map, and the target segmentation probability map generated by the model in the embodiments of the present invention are all pixel-level segmentation probability maps. Figure 7 A schematic diagram of pixel-level segmentation probability maps corresponding to optoelectronic chip images of different configurations (first configuration optoelectronic chip image, second configuration optoelectronic chip image, and third configuration optoelectronic chip image) is shown in some embodiments of the present invention.

[0074] In some embodiments, a dataset containing multiple high-quality photoelectric chip sample images covering various configurations can be pre-established. For example, the dataset may contain 1650 photoelectric chip sample images, including 550 sample images of a first configuration, 550 sample images of a second configuration, and 550 sample images of a third configuration. Subsequently, the dataset can be divided into a training set containing 1200 photoelectric chip sample images, a validation set containing 150 photoelectric chip sample images, and a test set containing 300 photoelectric chip sample images. In step 210, multiple photoelectric chip sample images covering various configurations can be obtained from the training set so that the segmentation network 600 can be trained in step 240 based on the multiple photoelectric chip sample images in the training set and the corresponding generated pseudo-annotated images.

[0075] In some embodiments, in step 220, the SLIC superpixel segmentation algorithm, combined with the f_max path cost function and the ROOT arc cost function, can be used to perform superpixel segmentation on each pixel in the optoelectronic chip sample image to obtain a superpixel segmented image of the optoelectronic chip containing multiple superpixels. Specifically, an initial seed point of a preset number of superpixels can be evenly distributed within the optoelectronic chip sample image based on a preset number of superpixels (i.e., the number of semantic regions), and an oversampling operation can be performed on the preset number of initial seed points to obtain multiple oversampled seed points. Multiple superpixels are then generated based on these multiple oversampled seed points. During the generation of multiple superpixels, the f_max path cost function and the ROOT arc cost function can be used to optimize the path cost in order to optimize the superpixel boundaries, so that the generated superpixel boundaries can fit the physical boundaries of the optoelectronic chip (i.e., the boundaries of each semantic region in the optoelectronic chip sample image). Specifically, an 8-neighborhood directed graph (vertices are pixels and edges are adjacent pixels) can be constructed based on the sample image of the optoelectronic chip. Based on the 8-neighborhood directed graph, the f_max path cost function can be used to retain the maximum arc cost of the path to capture the boundary features between each semantic region. Furthermore, the ROOT arc cost function can be used to calculate the feature Euclidean distance between each pixel and its oversampled seed point so as to use seed stable features to ensure uniform features within the final generated superpixel. Finally, a seed-restricted version of the IFT algorithm can be executed to generate multiple superpixels that fit the physical boundary of the optoelectronic chip.

[0076] In some embodiments, step 230 includes the following steps C1 to C6: C1. Extract the core features of each superpixel in the superpixel segmentation image of the optoelectronic chip. The core features include color features (color histogram) and spatial features (edge ​​gradient).

[0077] C2. Determine the similarity between the core features of each superpixel, and select multiple superpixel pairs with similarity higher than a similarity threshold to form multiple superpixel candidate groups. It should be understood that the similarity between the core features of each superpixel reflects the color differences and spatial adjacency relationships between them. This allows for the retention of superpixel pairs with small color differences and spatial adjacency. Furthermore, superpixel pairs within the same semantic region are more likely to be selected due to the consistency of their color and spatial features, while superpixel pairs across semantic regions are excluded due to large color and spatial differences. In some embodiments, feature vectors for each superpixel can be constructed based on their core features to form a feature vector library. Subsequently, the similarity between the feature vectors of each superpixel can be calculated as the similarity between the core features of each superpixel.

[0078] Then, multiple superpixel candidate groups can be merged based on the maximum coverage criterion (the maximum coverage set must completely cover all semantic regions without overlap). The details are as follows.

[0079] C3. Based on the core features of each superpixel, determine the semantic region to which each superpixel belongs.

[0080] C4. Based on the semantic region to which each superpixel belongs, determine the coverage of each superpixel candidate group to each semantic region. Coverage represents the ratio of the number of superpixels belonging to the semantic region in the superpixel candidate group to the total number of superpixels in the semantic region.

[0081] C5. For each semantic region, select multiple superpixel candidate groups from multiple superpixel candidate groups that have the largest coverage of the semantic region and are non-overlapping, and merge them to obtain multiple new superpixels. Specifically, for each semantic region, select multiple superpixel candidate groups that have the largest coverage of the semantic region (i.e., cover the semantic region to the greatest extent possible—including as many superpixels of the semantic region as possible and excluding superpixels of other semantic regions) as multiple high-coverage superpixel candidate groups for that semantic region, so as to prioritize the selection of superpixel candidate groups that have the most complete coverage of the semantic region. Furthermore, for each semantic region, select multiple high-coverage superpixel candidate groups that are non-overlapping (without spatial intersection) and merge them to obtain multiple new superpixels.

[0082] C6. Repeat steps C1 to C5 for each new superpixel until no new superpixel pairs with similarity higher than the similarity threshold are selected, no new superpixel candidate groups are formed, or the largest coverage set that completely covers all semantic regions without overlap is obtained (the final multiple new superpixels can completely cover all semantic regions without overlap). Then, generate a pseudo-labeled image based on the final merging result (i.e., the final merged multiple new superpixels).

[0083] Figure 8 A schematic diagram of an optoelectronic chip image segmentation apparatus 800 according to an embodiment of the present invention is shown. The optoelectronic chip image segmentation apparatus 800 can be deployed in a computing device 100, and the optoelectronic chip image segmentation apparatus 800 is configured to execute the optoelectronic chip image segmentation model training method 200 of the present invention.

[0084] like Figure 8 As shown, in an embodiment of the present invention, the photoelectric chip image segmentation device 800 includes an acquisition unit 810, a superpixel segmentation unit 820, a superpixel merging unit 830, a first training unit 840, and a second training unit 850 that are connected in sequence.

[0085] The acquisition unit 810 is used to acquire multiple optoelectronic chip sample images covering various configurations of optoelectronic chips.

[0086] The superpixel segmentation unit 820 is used to perform superpixel segmentation on each of the photoelectric chip sample images to obtain a photoelectric chip superpixel segmented image containing multiple superpixels.

[0087] The superpixel merging unit 830 is used to merge superpixel segments of the optoelectronic chip to generate a pseudo-annotated image. The pseudo-annotated image contains key structural edges of multiple semantic regions to indicate noise annotations of each pixel in the optoelectronic chip sample image. The multiple semantic regions include electrode and lead regions, light-emitting regions, outer edge regions, and background regions.

[0088] The first training unit 840 is used to train the segmentation network based on photoelectric chip sample images and pseudo-labeled images to obtain an initial training segmentation model.

[0089] The second training unit 850 is used to generate a second sample segmentation probability map based on the optoelectronic chip sample image using the initial trained segmentation model. The second sample segmentation probability map contains the second probability prediction value of each pixel in the optoelectronic chip sample image belonging to each semantic region. Based on the second sample segmentation probability map, the pixel category of each pixel in the optoelectronic chip sample image is determined, and the pixel category includes reliable pixels and uncertain pixels. For each reliable pixel, a second loss value is calculated between the second probability prediction value of the reliable pixel and the corresponding noise label in the pseudo-labeled image. For each uncertain pixel, a dense energy loss value of the uncertain pixel is calculated based on the color information and spatial location information of the uncertain pixel. The initial trained segmentation model is trained based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain an optoelectronic chip image segmentation model. The optoelectronic chip image segmentation model is used to generate a target segmentation probability map based on the target image of the optoelectronic chip to be detected.

[0090] It should be noted that the acquisition unit 810, the superpixel segmentation unit 820, the superpixel merging unit 830, and the first training unit 840 are used to execute the aforementioned steps 210-240, respectively, and the second training unit 850 is used to execute the aforementioned steps 250-280 (i.e., the noisy supervised training process). The specific execution logic of each unit can be found in the description of method 200 above, and will not be repeated here.

[0091] The photoelectric chip image segmentation model training method 200 according to embodiments of the present invention generates low-noise, high-quality pseudo-annotated images by performing superpixel segmentation on sample images of photoelectric chips with various configurations and combining it with a superpixel merging method, thereby reducing the cost of acquiring annotation data. Based on the initial training segmentation model obtained from the photoelectric chip sample images and pseudo-annotated images, the method further judges pixel reliability and performs noisy supervised training through the initial training segmentation model. This effectively suppresses the interference of noise annotations in the pseudo-annotated images on the model training process, enhancing the robustness and segmentation accuracy of the model in photoelectric chip segmentation tasks. This allows the finally trained photoelectric chip image segmentation model to accurately segment each region of the target image of the photoelectric chip to be detected, thereby achieving accurate and efficient detection of defects in each region of the photoelectric chip. Furthermore, the photoelectric chip image segmentation model trained according to the present invention is applicable to the segmentation of photoelectric chip images with various configurations, thus adapting to the detection requirements of photoelectric chips with different configurations.

[0092] Furthermore, by adding a multi-scale attention module to the skip connection between the encoder and decoder, the segmentation accuracy of the optoelectronic chip image can be further improved.

[0093] Furthermore, during the initial training and noisy supervised training processes, data augmentation operations are performed on the photoelectric chip sample images through the input layer. This can enhance the diversity of sample data, avoid overfitting of the trained model, and improve the generalization ability of the final trained photoelectric chip image segmentation model.

[0094] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0095] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the photoelectric chip image segmentation model training method of the present invention according to instructions in the program code stored in the memory.

[0096] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0097] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0098] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0099] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof.

[0100] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0101] Unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

Claims

1. A method for training an image segmentation model for an optoelectronic chip, executed in a computing device, comprising: Acquire multiple optoelectronic chip sample images covering various configurations; For each of the photoelectric chip sample images, superpixel segmentation is performed on the photoelectric chip sample image to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels; Superpixel merging is performed on the superpixel segmentation image of the optoelectronic chip to generate a pseudo-annotated image. The pseudo-annotated image contains key structural edges of multiple semantic regions used to indicate noise annotations for each pixel in the optoelectronic chip sample image. These multiple semantic regions include electrode and lead regions, light-emitting regions, outer edge regions, and background regions. The superpixel merging process for generating the pseudo-annotated image includes the following steps: extracting core features of each superpixel in the optoelectronic chip superpixel segmentation image, where the core features include color features and spatial features; determining the similarity between the core features of each superpixel and selecting multiple superpixel pairs with similarity higher than a similarity threshold to form multiple superpixel candidate groups; and based on the core features of each superpixel... Features are used to determine the semantic region to which each superpixel belongs; based on the semantic region to which each superpixel belongs, the coverage of each superpixel candidate group to each semantic region is determined, where the coverage is used to represent the ratio of the number of superpixels belonging to the semantic region in the superpixel candidate group to the total number of superpixels in the semantic region; for each semantic region, multiple superpixel candidate groups with the largest coverage of the semantic region and no overlap are selected from the multiple superpixel candidate groups and merged to obtain multiple new superpixels; the above steps are repeated for each new superpixel until no new superpixel pairs with similarity higher than the similarity threshold are selected, no new superpixel candidate groups are formed, or the largest coverage set that completely covers all semantic regions and has no overlap is obtained, at which point a pseudo-annotated image is generated; The segmentation network is trained based on the photoelectric chip sample image and the pseudo-labeled image to obtain an initial trained segmentation model. Using the initial training segmentation model, a second sample segmentation probability map is generated based on the photoelectric chip sample image. The second sample segmentation probability map contains a second probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region. Based on the second sample segmentation probability map, the pixel category of each pixel in the photoelectric chip sample image is determined, and the pixel category includes reliable pixels and uncertain pixels; For each reliable pixel, a second probability prediction value of the reliable pixel is calculated and a second loss value is calculated between the second probability prediction value of the reliable pixel and the noise label corresponding to the pseudo-label image; and for each uncertain pixel, a dense energy loss value of the uncertain pixel is calculated based on the color information and spatial location information of the uncertain pixel. The initial training segmentation model is trained based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain an optoelectronic chip image segmentation model. The optoelectronic chip image segmentation model is used to generate a target segmentation probability map based on the target image of the optoelectronic chip to be detected.

2. The method as described in claim 1, wherein, The initial training segmentation model is trained based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel to obtain an optoelectronic chip image segmentation model, including: Using a joint loss function, the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel are weighted and fused to obtain a fused loss value; The initial training segmentation model is trained based on the fusion loss value to obtain the optoelectronic chip image segmentation model.

3. The method as described in claim 1, wherein, Based on the second sample segmentation probability map, the pixel category of each pixel in the photoelectric chip sample image is determined, including: Each pixel in the second sample segmentation probability map whose second probability prediction value is greater than the reliability threshold is determined as a reliable pixel, and each pixel in the second sample segmentation probability map whose second probability prediction value is less than or equal to the reliability threshold is determined as an uncertain pixel.

4. The method according to any one of claims 1-3, wherein, The segmentation network is trained based on the photoelectric chip sample image and the pseudo-annotated image to obtain an initial trained segmentation model, including: Using the segmentation network, a first sample segmentation probability map is generated based on the photoelectric chip sample image. The first sample segmentation probability map contains a first probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region. Calculate the first probability prediction value of each pixel belonging to each semantic region and the first loss value between the noise label corresponding to the pseudo-annotated image; The segmentation network is trained based on the first loss value of each pixel to obtain an initial trained segmentation model.

5. The method of claim 4, wherein, The segmentation network adopts an improved Trans-Unet architecture, which includes an input layer, an encoder, a bottleneck layer, a decoder, and an output layer that are coupled in sequence. In addition, a multi-scale attention module is added to the skip connection between each layer in the encoder and the corresponding layer in the decoder. Using the segmentation network, a first sample segmentation probability map is generated based on the photoelectric chip sample image, including: The input layer performs preprocessing operations on the photoelectric chip sample image to obtain a tensor format image. The preprocessing operations include normalization, size alignment, and data augmentation. The data augmentation operations include horizontal and vertical flipping and random rotation. The encoder progressively downsamples the tensor format image to extract low-level features, and captures semantic information based on the low-level features to generate high-level semantic features. The low-level features include edge features and texture features. Each multi-scale attention module performs attention processing on the output features of each layer in the encoder to obtain the attention features of each layer, and then passes the attention features of each layer to the corresponding layer in the decoder. By fusing the high-level semantic features through the bottleneck layer, fused semantic features are obtained; The decoder progressively upsamples the fused semantic features to obtain upsampled features, and then concatenates the upsampled features with the attention features of each layer to obtain the first Logits value. The first Logits value is converted into a first sample segmentation probability map using an activation function through the output layer.

6. The method according to any one of claims 1-3, wherein, Determine the similarity between the core features of each superpixel, including: Based on the core features of each superpixel, construct the feature vector of each superpixel; Calculate the similarity between the feature vectors of each superpixel, and use this similarity as the core feature similarity between each superpixel.

7. The method according to any one of claims 1-3, wherein, The photoelectric chip sample image is subjected to superpixel segmentation to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels, including: The SLIC superpixel segmentation algorithm, combined with the f_max path cost function and the ROOT arc cost function, is used to perform superpixel segmentation on each pixel in the photoelectric chip sample image to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels.

8. The method of claim 4, wherein, Calculating a first loss value between the first probability prediction value of each pixel belonging to each semantic region and the corresponding noise label in the pseudo-annotated image includes: Using the cross-entropy loss function, the first probability prediction value of each pixel belonging to each semantic region is calculated and the first cross-entropy loss value between the noise label corresponding to the pseudo-labeled image is used as the first loss value. Calculating the second probability prediction value of the reliable pixel and the second loss value corresponding to the noise label in the pseudo-labeled image includes: Using the cross-entropy loss function, the second cross-entropy loss value between the second probability prediction value of the reliable pixel and the corresponding noise label in the pseudo-labeled image is calculated as the second loss value.

9. A photoelectric chip image segmentation apparatus, deployed in a computing device, adapted to perform the method as described in any one of claims 1-8, the apparatus comprising: The acquisition unit is suitable for acquiring multiple optoelectronic chip sample images covering various configurations of optoelectronic chips; The superpixel segmentation unit is adapted to perform superpixel segmentation on each of the photoelectric chip sample images to obtain a photoelectric chip superpixel segmentation image containing multiple superpixels. The superpixel merging unit is adapted to merge superpixel segments of the optoelectronic chip to generate a pseudo-annotated image. The pseudo-annotated image contains key structural edges of multiple semantic regions to indicate noise annotations of each pixel in the optoelectronic chip sample image. The multiple semantic regions include electrode and lead regions, light-emitting regions, outer edge regions, and background regions. The first training unit is adapted to train the segmentation network based on the photoelectric chip sample image and the pseudo-labeled image to obtain an initial training segmentation model. The second training unit is adapted to use the initial training segmentation model to generate a second sample segmentation probability map based on the photoelectric chip sample image. The second sample segmentation probability map includes a second probability prediction value of each pixel in the photoelectric chip sample image belonging to each semantic region. Based on the second sample segmentation probability map, the pixel category of each pixel in the photoelectric chip sample image is determined, and the pixel category includes reliable pixels and uncertain pixels. For each reliable pixel, a second loss value is calculated between the second probability prediction value of the reliable pixel and the noise label corresponding to the pseudo-labeled image. For each uncertain pixel, a dense energy loss value of the uncertain pixel is calculated based on the color information and spatial location information of the uncertain pixel. Based on the second loss value of each reliable pixel and the dense energy loss value of each uncertain pixel, the initial training segmentation model is trained to obtain a photoelectric chip image segmentation model. The photoelectric chip image segmentation model is used to generate a target segmentation probability map based on the photoelectric chip target image to be detected.

10. A computing device, comprising: At least one processor; and A memory storing program instructions, wherein the program instructions are configured to be processed by the at least one processor, the program instructions including instructions for processing the method as claimed in any one of claims 1-8.

11. A computer program product comprising computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method as described in any one of claims 1-8.

12. A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-8.

Images