LED chip positioning method and LED chip positioning device based on knowledge distillation

Abstract

The application discloses a LED chip positioning method and device based on knowledge distillation. The LED chip positioning method based on knowledge distillation comprises the following steps: segmenting a to-be-detected LED chip image into multiple image regions with overlapping areas, and magnifying each image region to obtain a slice image; inputting the to-be-detected LED chip image and each slice image into a first target detection model for processing to obtain an LED chip detection result in each image, wherein the detection result comprises a chip detection region and a corresponding chip type, and the first target detection model is obtained by training with a detection result of a second target detection model as supervision; and merging the LED chip detection regions in each image to determine an LED chip region and a chip type of the to-be-detected LED chip image. According to the application, the efficiency of LED chip detection and positioning can be improved, and the hardware performance requirement during deployment can be reduced while ensuring detection accuracy.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an LED chip positioning method and device based on knowledge distillation. Background Technology

[0002] LED (Light-Emitting Diode) chips possess advantages such as high energy conversion efficiency, high brightness, long lifespan, small size, environmental friendliness, and diverse colors, making them widely used in communications, agriculture, healthcare, biology, transportation, and medicine. With the continuous development of manufacturing technology and materials science, LED chip manufacturing processes are becoming increasingly complex, leading to a significant increase in chip demand. As chip integration levels continue to rise, the challenges in manufacturing and testing are also growing. Traditional methods heavily rely on semi-automatic equipment and manual production lines, are significantly affected by human subjective judgment and environmental factors, making quality control difficult, resulting in low overall efficiency, high labor costs, and high error costs, placing enormous economic pressure on enterprises. Therefore, current testing methods are insufficient to meet the current demands for high-efficiency and high-precision LED chip manufacturing.

[0003] Therefore, a non-destructive, safe, accurate, and efficient solution is needed to locate and screen LED chips. Summary of the Invention

[0004] This application provides an LED chip positioning method and an LED chip positioning device based on knowledge distillation, in an attempt to solve or at least alleviate at least one of the problems mentioned above.

[0005] According to one aspect of this application, a knowledge distillation-based LED chip localization method is provided, comprising: segmenting an LED chip image to be detected into multiple image regions with overlapping areas, and enlarging each image region to obtain corresponding slice images; inputting the LED chip image to be detected and each slice image into a first target detection model for processing to obtain LED chip detection results in each image, the detection results including chip detection regions and their corresponding chip types, wherein the first target detection model is obtained by supervised training using the detection results of a second target detection model; and merging the LED chip detection regions in each image to determine the LED chip regions and chip types of the LED chip image to be detected.

[0006] Optionally, in the method according to this application, the first object detection model includes a first feature extraction network component and a prediction component, the second object detection model includes a second feature extraction network component and a prediction component, and the computational cost and parameter count of the first feature extraction network component are less than those of the second feature extraction network component.

[0007] Optionally, in the method according to this application, the first object detection model is trained as follows: the training samples are labeled to obtain labeled data, the labeled data including labeled chip regions and labeled types; the training samples are input into the second object detection model, and after processing, the second chip region and the second type are output; the training samples are input into the initial first object detection model, and after processing, the first chip region and the first type are output; at least the labeled data, the first chip region and the first type, and the second chip region and the second type are combined to calculate the total loss function, and the parameters of the first object detection model are adjusted based on the total loss function until the training is completed, thus obtaining the first object detection model.

[0008] Optionally, in the method according to this application, at least in combination with the labeled data, the first chip region and the first type, the second chip region and the second type, the total loss function is calculated, including: calculating a target loss function based on the second chip region, the first chip region, and the labeled chip region; calculating a classification loss function based on the second type and its corresponding second chip region confidence, the first type, and the labeled type; calculating a regression loss function based on the second chip region and its corresponding second chip region confidence, the first chip region, and the labeled chip region; and determining the total loss function by combining the target loss function, the classification loss function, and the regression loss function.

[0009] Optionally, in the method according to this application, calculating the target loss function based on the second chip region, the first chip region, and the labeled chip region includes: calculating a second target loss based on the second chip region and the labeled chip region; calculating a first target loss based on the first chip region and the labeled chip region; and determining the target loss function by using the product of the second target loss and the balance coefficient, and the sum of the product and the first target loss.

[0010] Optionally, in the method according to this application, calculating a classification loss function based on the second type and its corresponding second chip region confidence, the first type, and the labeling type includes: calculating a second classification loss based on the second type and the labeling type; calculating a first classification loss based on the first type and the labeling type; and determining the classification loss function by using the product of the second classification loss and the balance coefficient, the second chip region confidence, and the sum of the product and the first classification loss.

[0011] Optionally, in the method according to this application, calculating a regression loss function based on the second chip region and its corresponding second chip region confidence level, the first chip region, and the labeled chip region includes: calculating a second regression loss based on the second chip region and the labeled chip region; calculating a first regression loss based on the first chip region and the labeled chip region; and determining the regression loss function by using the product of the second regression loss and the balance coefficient, the second chip region confidence level, and the first regression loss.

[0012] Optionally, the method according to this application further includes: training a second target detection model using the enhanced chip dataset, wherein the enhanced chip dataset is generated by: acquiring LED chip images containing different types of chips as original chip images; slicing the original chip images to generate enhanced chip images; and using the original chip images and the enhanced chip images as the enhanced chip dataset.

[0013] Optionally, in the method according to this application, slicing the original chip image to generate an enhanced chip image includes: dividing the original chip image into multiple image regions with overlapping areas according to preset parameters; and enlarging each image region to obtain a corresponding enhanced chip image.

[0014] Optionally, in the method according to this application, merging the LED chip detection regions in each image to determine the LED chip region and chip type of the LED chip image to be detected includes: using non-maximum suppression to convert the LED chip detection regions in each image onto the LED chip image to be detected for merging, thereby obtaining the LED chip region and its corresponding chip type.

[0015] Optionally, the method according to this application further includes: filtering out chips of a predetermined type from the LED chip image to be detected based on the LED chip region and chip type.

[0016] According to another aspect of this application, an LED chip positioning device is provided, comprising: an image processing unit adapted to segment an LED chip image to be detected into multiple image regions with overlapping areas, and to magnify each image region to obtain corresponding slice images; a detection unit adapted to input the LED chip image to be detected and each slice image into a first target detection model for processing, and to obtain LED chip detection results in each image, the detection results including chip detection regions and their corresponding chip types; and a chip positioning unit adapted to merge the LED chip detection regions in each image to determine the LED chip regions and chip types of the LED chip image to be detected.

[0017] Optionally, the apparatus according to this application further includes a model training unit, adapted to train the first object detection model using the detection result of the second object detection model as supervision, wherein the first object detection model includes a first feature extraction network component and a prediction component, the second object detection model includes a second feature extraction network component and a prediction component, and the computational complexity and parameter count of the first feature extraction network component are less than those of the second feature extraction network component.

[0018] According to another aspect of this application, a computing device is provided, comprising: one or more processor memories; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0019] According to another aspect of this application, a computer-readable storage medium is provided for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0020] In summary, according to the scheme of this application, based on the existing detection accuracy of the second target detection network, the training of the first target detection model is guided by the second target detection model to better learn complex features and relationships, thereby improving the detection accuracy of the lightweight network. This enables the LED chip positioning scheme of this application to improve the efficiency of LED chip detection and positioning while ensuring detection accuracy, and to reduce the hardware performance requirements during deployment.

[0021] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0022] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles claimed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this application will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this application, the same reference numerals generally refer to the same parts or elements.

[0023] Figure 1 The present application illustrates an LED chip wafer disk and an LED chip to be tested according to some embodiments thereof;

[0024] Figure 2A schematic diagram of a visual inspection system 200 according to some embodiments of this application is shown;

[0025] Figure 3 A schematic diagram of an LED chip positioning device 300 according to some embodiments of this application is shown;

[0026] Figure 4 A schematic diagram of a computing device 400 according to some embodiments of this application is shown;

[0027] Figure 5 A flowchart illustrating a knowledge distillation-based LED chip positioning method 500 according to some embodiments of this application is shown.

[0028] Figure 6 A schematic diagram of the network structure of a first target detection model according to some embodiments of this application is shown;

[0029] Figure 7 A schematic diagram of a knowledge distillation model according to some embodiments of this application is shown;

[0030] Figure 8 A schematic diagram of training samples according to some embodiments of this application is shown;

[0031] Figure 9 A schematic diagram of an enhanced chip dataset according to some embodiments of this application is shown. Detailed Implementation

[0032] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0033] The emergence of machine vision technology has brought about a tremendous transformation in the chip manufacturing and inspection industry. LED chip inspection based on machine vision refers to the use of imaging equipment such as industrial cameras, lenses, and light sources to acquire images of the chips to be inspected. Then, through computers and corresponding image processing algorithms, high-speed, high-precision inspection and analysis of the chips in automated production lines is performed, enabling real-time monitoring and control of the automated production lines. This can significantly improve the efficiency and accuracy of chip manufacturing inspection, ensuring product quality and production safety. Using machine vision to replace manual visual inspection has become an important trend in LED chip development, helping to push chip automation production to new heights of intelligence, flexibility, and unmanned operation.

[0034] This application researches and verifies some key technologies involved in the LED chip manufacturing and testing process, such as LED chip positioning and screening methods. By integrating, improving, and innovating upon existing computer vision algorithms, the research content of this application can meet the practical industrial application needs of LED chip testing and has certain engineering value. Furthermore, the research results of this application can be extended to testing tasks in other similar fields, providing valuable reference for related tasks in other areas.

[0035] The manufacturing process of LED chips is divided into front-end and back-end processes, specifically consisting of four main parts: wafer processing, wafer probing, chip assembly, and chip testing. First, in the wafer processing stage, multiple independent square areas are etched into the wafer to create the LED chip structure. Next, in the wafer probing stage, probing equipment is used to test the electrical characteristics of each area, and a laser is used to diced the wafer into individual chips. The testing object in this application is the wafer disk that has undergone the wafer processing and wafer probing stages and is about to be sent to the assembly stage. The laser-diced wafer is placed on a blue film on the wafer disk, and a wafer extender is used to stretch the blue film to form densely packed LED chips. For example... Figure 1 The image shows an LED chip wafer disk (left) to be tested according to this application and the densely packed LED chips on it (as enlarged on the right).

[0036] Due to the wafer fabrication process, chip defects (hereinafter referred to as blank chips) are generated at the edge areas. During storage and transportation, contamination is inevitably introduced, resulting in contaminated chips. The LED chip positioning and screening task involves using target detection algorithms to identify various LED chip types, screen out blank and contaminated defective chips, and crop the original image based on the positioning results to extract an image containing only a single chip.

[0037] To achieve efficient and accurate LED chip positioning and screening, high-quality chip images and a suitable detection system are required. Figure 2 A schematic diagram of an LED chip vision inspection system 200 according to some embodiments of this application is shown. This system 200 can improve the efficiency of LED chip vision inspection systems, achieving highly efficient automated imaging and inspection.

[0038] like Figure 2 As shown, an efficient visual inspection system 200 for LED chips is constructed by alternating and cyclically detecting three core stations: the loading / unloading station (as shown in ①), the angle correction station (as shown in ②), and the positioning and screening station (as shown in ③). The overall efficiency of the system 200 is improved by parallel processing of angle correction and chip detection. The specific inspection process is as follows:

[0039] (1) The robotic arm clamps the LED chip wafer disk 1 from the loading station and sends it into the angle correction station. The LED chip image is collected once in the center area of ​​the wafer. The chip correction angle is obtained by the LED chip angle correction algorithm. The LED chip angle is corrected by rotating the wafer disk through the rotating platform.

[0040] (2) During the rotation and correction process of LED chip wafer disk 1, the robotic arm clamps the LED chip wafer disk 2, which has completed the chip positioning and screening process, from the detection station and puts it back to the unloading station.

[0041] (3) The LED chip wafer disk 1, which has completed angle correction, is clamped by the robotic arm from the angle correction station and sent to the positioning and screening station. Full coverage imaging and LED chip detection are performed. While positioning the chip, dirt and blank chips are screened out.

[0042] (4) During the inspection of LED chip wafer 1, the robotic arm clamps the new LED chip wafer 3 from the loading station and repeats the process (1)-(4).

[0043] According to this application, image acquisition units for acquiring chip images are arranged at both the angle correction station and the positioning and screening station. These image acquisition units can be composed of devices such as cameras, lenses, point light sources, and backlights, to provide high-quality images for subsequent LED chip angle correction and positioning and screening tasks. This application does not impose excessive restrictions on the specific models of the aforementioned devices.

[0044] This application primarily addresses the positioning and screening process of LED chips in system 200. According to some implementation methods, the positioning and screening process of LED chips can be achieved using an LED chip positioning device.

[0045] The positioning and screening of LED chips needs to simultaneously meet the detection requirements for chip classification and positioning. If the chip type is incorrectly identified, it is considered a detection error; otherwise, further judgment is made regarding chip positioning. The positioning requirement is that the positioning frame can contain a single chip, facilitating subsequent processes to perform further detection based on the image of that single chip.

[0046] Figure 3 A schematic diagram of an LED chip positioning device 300 according to some embodiments of this application is shown. The device 300 is capable of performing a knowledge distillation-based LED chip positioning method to achieve LED chip positioning and category detection. In some embodiments, the LED chip positioning device 300 may be partially or entirely arranged at the positioning and screening station of system 200.

[0047] like Figure 3As shown, the LED chip positioning device 300 includes at least: an image processing unit 310, a detection unit 320, a chip positioning unit 330, and a model training unit 340. As mentioned above, the device 300 also includes an image acquisition unit (not shown) to acquire images of the angle-corrected LED chip wafer disk to obtain an image of the LED chip to be detected.

[0048] The image of the LED chip to be inspected contains multiple LED chips arranged in a dense array. As mentioned earlier, these chips may include defective chips such as blank chips and dirty chips. Target detection is needed to identify the chip type, filter out defective chips, and crop the image to be inspected based on the localization results to extract an image containing only a single chip.

[0049] The image processing unit 310 divides the image of the LED chip to be detected into multiple image regions with overlapping areas, and magnifies each image region to obtain a slice image.

[0050] refer to Figure 1 LED chips are densely arranged on LED chip wafers, and each individual LED chip is small. Considering that small targets occupy a small proportion in an image, and that pooling layers in target detection networks further reduce the amount of target information—for example, a 24×24 pixel target might only have one pixel after four pooling layers, resulting in excessively low dimensionality—this embodiment first uses the image processing unit 310 to perform slice enhancement processing on the LED chip image to be detected. The image is first sliced ​​into multiple image regions, i.e., each region is a small-sized image. Then, while maintaining the aspect ratio of the small-sized images, their size is increased (i.e., magnified), resulting in sliced ​​images. In this way, by slice enhancement of the LED chip image to be detected, the coverage area of ​​the chip is increased while expanding the feature region of the small target chip, reducing interference caused by detection box (chip positioning area) perturbation and mitigating feature loss due to excessive convolution. Simultaneously, slice enhancement also alleviates target aggregation, avoiding the problem of feature aggregation of multiple targets in deep features.

[0051] The detection unit 320 inputs the image of the LED chip to be detected and each slice image into the first target detection model for processing, and obtains the LED chip detection results in each image. According to this application, the detection results include the chip detection area and its corresponding chip type. In some embodiments, the chip detection area is the smallest bounding box containing a single LED chip, which is usually a rectangle. The chip types include: first type chips, second type chips, third type chips, and defective chips (e.g., blank chips, dirty chips, etc.), wherein the first type chips, second type chips, and third type chips are chips of different sizes, namely large, medium, and small LED chips, respectively.

[0052] The first object detection model is generated by training model training unit 340. Typically, to enhance feature extraction capabilities, object detection models employ modules with deep network structures (e.g., ResNet50-vd) as feature extraction network components. While improving detection accuracy, this also results in a large number of parameters and computational overhead, leading to slower inference speeds and higher requirements for deployment memory and computing power. Therefore, in some embodiments, lightweight networks with fewer floating point operations (FLOPs) and fewer parameters than traditional modules are selected to replace the feature extraction network components in the object detection model, achieving a lightweight improvement. However, with a significant decrease in the depth of the model's network structure, its detection accuracy inevitably decreases to some extent. Therefore, in embodiments according to this application, a knowledge distillation model is constructed, utilizing a complex, high-precision second object detection model to guide the training of the lightweight first object detection model, thereby improving the detection accuracy of the first object detection model. In other words, the model training unit 340 trains the first object detection model using the detection results of the second object detection model as supervision. The first object detection model includes a first feature extraction network component and a prediction component, and the second object detection model includes a second feature extraction component and a prediction component. The computational cost and parameter count of the first feature extraction network component are less than those of the second feature extraction network component.

[0053] In a preferred embodiment, the second object detection model employs the PP-YOLO model. The model training unit 340 can also be used to train the second object detection model. The second feature extraction network component of the second object detection model consists of a ResNet50-vd backbone feature extraction network and an FPN feature pyramid network for feature fusion. The ResNet50-vd is replaced with the lighter MobileNetV3. In other words, MobileNetV3 is used as the backbone feature extraction network, and the effective feature layers downsampled by 8x, 16x, and 32x are input into the FPN while retaining the PP-YOLO detection head (i.e., the prediction component), thus forming the network structure of the first object detection model.

[0054] MobileNetV3 inherits the advantages of MobileNetV1 and MobileNetV2 and incorporates a series of innovative improvements, achieving excellent performance on ImageNet classification and COCO detection tasks. While maintaining the model's lightweight and efficient nature, MobileNetV3 improves accuracy and generalization performance through a series of methods, including replacing traditional convolutions with depthwise separable convolutions, introducing SE attention mechanisms, using inverse residual structures with linear bottlenecks, and employing the H-Swish activation function. Further details are omitted here.

[0055] The specific process of model training using model training unit 340 will be explained in detail below.

[0056] The chip positioning unit 330 merges the LED chip detection areas in each image to finally determine the LED chip area and chip type of the LED chip image to be detected.

[0057] According to the embodiments of this application, defective chips in the image to be inspected are screened out according to the chip type, and the image to be inspected is cropped according to the LED chip area to extract each image containing only a single chip.

[0058] For specific details regarding the execution process of device 300, please refer to the detailed description of chip positioning method 500 below.

[0059] According to the LED chip positioning device 300 of this application, based on the existing detection accuracy of the second target detection network, the second target detection model guides the training of the first target detection model to better learn complex features and relationships, thereby improving the detection accuracy of the lightweight network. This enables the LED chip positioning scheme of this application to improve the efficiency of LED chip detection and positioning while ensuring algorithm accuracy, and reduce the hardware performance requirements during deployment.

[0060] Furthermore, during the detection process, the image of the LED chip to be detected was sliced ​​and enhanced to generate sliced ​​images. Then, the detection results from the original LED chip image and the sliced ​​images were combined to obtain the final chip localization result. This further improves the detection accuracy for densely packed small target chips.

[0061] The LED chip positioning scheme according to this application can be implemented by one or more computing devices. Figure 4 A structural block diagram of a computing device 400 according to some embodiments of this application is shown. It should be noted that... Figure 4 The computing device 400 shown is merely an example. In practice, the computing device used to implement this application can be any type of device, and its hardware configuration can be similar to... Figure 4 The computing device 400 shown is the same as, and can also be with Figure 4 The computing device 400 shown is different. In practice, the computing device used to implement this application can... Figure 4 The hardware components of the computing device 400 shown may be added or removed. This application does not limit the specific hardware configuration of the computing device.

[0062] like Figure 4As shown, in the basic configuration 402, the computing device 400 typically includes a system memory 406 and one or more processors 404. A memory bus 408 can be used for communication between the processors 404 and the system memory 406.

[0063] Depending on the desired configuration, processor 404 can be any type of processor, including but not limited to: microprocessors (μP), microcontrollers (μC), digital information processors (DSPs), or any combination thereof. Processor 404 may include one or more levels of cache such as L1 cache 410 and L2 cache 412, processor core 414, and registers 416. Example processor core 414 may include an arithmetic logic unit (ALU), a floating-point unit (FPU), a digital signal processing (DSP) core, or any combination thereof. Example memory controller 418 may be used with processor 404, or in some implementations, memory controller 418 may be an internal part of processor 404.

[0064] Depending on the desired configuration, system memory 406 can be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. Physical memory in a computing device typically refers to volatile RAM, and data on a disk needs to be loaded into physical memory before it can be read by processor 404. System memory 406 may include operating system 420, one or more applications 422, and program data 424. In some embodiments, applications 422 may be arranged to execute instructions on the operating system using program data 424 by one or more processors 404. Operating system 420 may be, for example, Linux, Windows, etc., and includes program instructions for handling basic system services and performing hardware-dependent tasks. Applications 422 include program instructions for implementing various user-desired functions, and applications 422 may be, for example, browsers, instant messaging software, software development tools (such as integrated development environments (IDEs), compilers, etc.), but are not limited to these.

[0065] When the computing device 400 starts up, the processor 404 reads the program instructions of the operating system 420 from the memory 406 and executes them. Application 422 runs on top of the operating system 420, utilizing the interfaces provided by the operating system 420 and the underlying hardware to implement various user-expected functions. When the user starts application 422, application 422 is loaded into the memory 406, and the processor 404 reads and executes the program instructions of application 422 from the memory 406.

[0066] The computing device 400 also includes a storage device 432, which includes a removable storage device 436 (e.g., CD, DVD, USB flash drive, portable hard drive, etc.) and a non-removable storage device 438 (e.g., hard disk drive, HDD, etc.). Both the removable storage device 436 and the non-removable storage device 438 are connected to the storage interface bus 434.

[0067] The computing device 400 may also include a storage interface bus 434. The storage interface bus 434 enables communication from storage devices 432 (e.g., removable memory 436 and non-removable memory 438) to the basic configuration 402 via a bus / interface controller 430. At least a portion of the operating system 420, application 422, and program data 424 may be stored on the removable memory 436 and / or the non-removable memory 438, and loaded into system memory 406 via the storage interface bus 434 when the computing device 400 is powered on or when application 422 is to be executed, and executed by one or more processors 404.

[0068] The computing device 400 may also include an interface bus 440 that facilitates communication from various interface devices (e.g., output devices 442, peripheral interfaces 444, and communication devices 446) to the basic configuration 402 via a bus / interface controller 430. Example output devices 442 include a graphics processing unit 448 and an audio processing unit 450. They may be configured to facilitate communication with various external devices such as displays or speakers via one or more A / V ports 452. Example peripheral interfaces 444 may include a serial interface controller 454 and a parallel interface controller 456, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 458. Example communication devices 446 may include a network controller 460, which may be arranged to facilitate communication with one or more other computing devices 462 via a network communication link through one or more communication ports 464.

[0069] A network communication link can be an example of a communication medium. A communication medium can typically be embodied in a modulated data signal, such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A “modulated data signal” can be a signal whose data set, or its modifications, can be encoded as information within the signal. As a non-limiting example, a communication medium can include wired media such as wired networks or leased lines, and various wireless media including sound, radio frequency (RF), microwave, infrared (IR), or other wireless media. The term “computer-readable medium” as used herein can include both storage media and communication media.

[0070] The computing device 400 can be implemented as a personal computer, including desktop and laptop computer configurations. Of course, the computing device 400 can also be implemented as part of a small-sized portable (or mobile) electronic device, such as a cellular phone, digital camera, personal digital assistant (PDA), personal media player device, wireless network browsing device, personal head-mounted device, application-specific device, or a hybrid device that may include any of the above functions. It can even be implemented as a server, such as a file server, database server, application server, and web server. The embodiments of this application do not impose any limitations on this.

[0071] In an embodiment according to this application, a computing device 400 is configured to execute an LED chip positioning method 500 based on knowledge distillation according to this application. An application 422 arranged on an operating system includes multiple program instructions for executing one or more of the above methods, which can instruct a processor 404 to execute the method 500 of this application.

[0072] Figure 5 A flowchart illustrating a knowledge distillation-based LED chip positioning method 500 according to some embodiments of this application is shown. It should be noted that method 500 can be implemented using the aforementioned LED chip positioning device 300. Method 500 and device 300 are complementary; related aspects will not be elaborated further.

[0073] With the continuous development of deep learning, deep learning-based object detection models have been widely used due to their superior performance. An object detection model includes a feature extraction network component for extracting features from the input image and a prediction component for predicting the location and type of objects in the image. To enhance feature extraction capabilities, object detection models often employ modules with deep network structures as feature extraction network components. This results in a large number of parameters and computational load, leading to slow inference speed and high requirements for deployment memory and computing power. However, lightweight feature extraction network components inevitably experience a certain degree of decrease in detection accuracy. Therefore, in the embodiments of this application, a knowledge distillation model is constructed using a second object detection model with a deep network structure and a lightweight first object detection model to improve the detection accuracy of the first object detection model.

[0074] Therefore, in some embodiments, lightweight networks with lower computational cost (floating point operations, FLOPs) and parameter count than traditional modules are selected to replace the feature extraction network components in the object detection model, thereby achieving a lightweight improvement in the object detection model. However, with the significant decrease in the depth of the model network structure, [further improvements are needed]. Therefore, in the embodiments according to this application, a knowledge distillation model is constructed, utilizing a complex and high-precision second object detection model to guide the training of the lightweight first object detection model, thereby improving the detection accuracy of the first object detection model. The second object detection model can be an existing object detection model such as Cascade R-CNN, FCOS, PP-YOLO, etc., and this application does not impose many restrictions on it. Due to space limitations, this section uses the PP-YOLO model structure as an example for the second object detection model for illustration.

[0075] In some embodiments, the second object detection model includes a second feature extraction network component (e.g., ResNet50-vd+FPN) and a prediction component (e.g., PP-YOLO detection head). Figure 6A schematic diagram of the network structure of a first object detection model 600 according to some embodiments of this application is shown. The first object detection model 600 includes a first feature extraction network component 610 and a prediction component 620, wherein the prediction component 620 is consistent with the prediction component in the second object detection model, and the first feature extraction network component 610 includes a MobileNetV3 lightweight network and an FPN. Specifically, the ResNet50-vd in the second object detection model is replaced by a MobileNetV3 lightweight network, removing the last four convolutional layers and pooling layers used for classification in the ResNet50-vd, and inputting the effective feature layers downsampled by 8x, 16x, and 32x from the MobileNetV3 backbone network into the FPN. The bneck module structure contains multiple separable convolutional blocks of various sizes and depths, batch normalization (BN) layers, H-Switch activation functions, etc., which can be adjusted according to the task and network requirements. It can be seen that the computational cost and parameter count of the first feature extraction network component are smaller than those of the second feature extraction network component.

[0076] According to an embodiment of this disclosure, method 500 further includes a process of training a first object detection model. As described above, the training process is based on a knowledge distillation model. Figure 7 A schematic diagram of a knowledge distillation model according to some embodiments of this application is shown. For example... Figure 7 The relatively complex and high-performance second object detection model is considered the teacher network, while the lightweight first object detection model is considered the student network. Since the teacher network's predictions contain more hidden information than the true labels, this information is passed to the student network as additional knowledge. The teacher network's predictions are used as supervision to continuously optimize and adjust the network model parameters during the student network's training process. The following combines... Figure 7 The training process will be explained.

[0077] The first step is to label the training samples to obtain labeled data, which includes labeled chip regions and label types.

[0078] The image acquisition unit built in the visual inspection system 200 acquires LED chip images as training samples. In some embodiments, 1010 images of three types of LED chips are acquired as a dataset of training samples. Figure 8 A schematic diagram of the training samples is shown, where (a), (b), and (c) represent three chip types (hereinafter referred to as chip 1, chip 2, and chip 3, respectively). It should be understood that the detection and identification of chip types depends on the types of chips involved when the training samples are collected. In this embodiment, images were collected for three types of chips, but this application is not limited to this.

[0079] As mentioned earlier, the LED chip region can be represented by the coordinates of the smallest bounding box that includes a single LED chip. The chip type, in addition to... Figure 8 In addition to the three types shown, there are also defective chips, which can be denoted as: Chip 1, Chip 2, Chip 3, and Defective Chip. For example... Figure 8 The image shown is the original chip image after annotation. Individual LED chips are marked with rectangles, and the chip type can be distinguished by the color (or grayscale) of the rectangle.

[0080] The second step, as Figure 7 As shown, training samples are input into the knowledge distillation model. Specifically, the training samples are input into the second object detection model, and after feature extraction and prediction processing, the model outputs a second chip region and a second type. Training samples are also input into the initial first object detection model, and after feature extraction and prediction processing, the model outputs a first chip region and a first type. The chip region output by the object detection model typically includes the coordinates of the bounding box of the chip (object) and the confidence score of that bounding box. This confidence score reflects the probability (objectness) that the bounding box contains the object and the accuracy of the bounding box.

[0081] The third step involves combining at least the labeled data, the first chip region and the first type, and the second chip region and the second type to calculate the total loss function. Based on the total loss function, the parameters of the first object detection model are adjusted until training is completed, thus obtaining the first object detection model.

[0082] like Figure 7 As shown, based on the true label loss (Detection Loss), the total loss function is constructed by introducing additional objectness loss, classification loss, and regression loss (shown by dashed lines) by measuring the difference between the teacher network prediction information and the student network prediction information. The following details the calculation method of the total loss function according to some embodiments of this application.

[0083] (1) Calculate the target loss function based on the second chip region, the first chip region, and the labeled chip region. The target loss function is a loss function that characterizes whether the target is included, such as... Figure 7 The objectness loss function is measured by the confidence level of objectness.

[0084] In some embodiments, a second target loss is calculated based on the confidence level contained in the second chip region and the labeled chip region; a first target loss is calculated based on the first chip region and the labeled chip region; more specifically, the second target loss is calculated based on the confidence level contained in the second chip region and the confidence level of the labeled chip region, and the first target loss is calculated based on the confidence level contained in the first chip region and the confidence level of the labeled chip region. Then, the target loss function is determined by using the product of the second target loss and the balance coefficient, and the sum of the product and the first target loss. Optionally, the target loss function... It can be expressed by the following formula:

[0085]

[0086] In the formula, It is the confidence level of the first chip region. It is the confidence level of the labeled chip area. λ represents the confidence level of the second chip region. D As a balancing coefficient, it is used to balance the two parts of the loss. Objective loss function. Loss of true labels without knowledge distillation (i.e., the first target loss) and distillation loss The loss consists of two parts: (i.e., the product of the second objective loss and the balance coefficient). The difference between distillation loss and true label loss lies in the true label of one of the inputs. Replace with the confidence level of the second chip region and using λ D Balance the losses from both parts.

[0087] (2) Calculate the classification loss function based on the second type and its corresponding second chip region confidence, the first type, and the labeling type.

[0088] like Figure 7 The classification loss function is measured by chip type classification.

[0089] In some embodiments, a second classification loss is calculated based on the second type and the labeling type; a first classification loss is calculated based on the first type and the labeling type; and a classification loss function is determined by summing the product of the second classification loss and the balance coefficient, the confidence level of the second chip region, and the first classification loss. Specifically, the classification loss function... This can be expressed by the following formula:

[0090]

[0091] In the formula, λ represents the confidence level of the second chip region.D As a balance coefficient It is a label type. It is the first type. It is the second type. The classification loss function is also composed of the loss from the true label. (i.e., first category loss) and distillation loss (That is, the product of the second classification loss, the balance coefficient, and the confidence score of the second chip region) is composed of the distillation loss function, and the biggest difference from the target loss function is that the distillation loss part includes the confidence score output by the second target detection model. This represents the probability that the target is contained within the bounding box, thus ensuring that the first target detection model will not mislearn background information when the bounding box selects the background.

[0092] (3) Calculate the regression loss function based on the second chip region and its corresponding second chip region confidence, the first chip region, and the labeled chip region.

[0093] like Figure 7 The regression loss function is measured by the bounding box coordinates (regression) contained in the chip region.

[0094] In some embodiments, a second regression loss is calculated based on the second chip region and the labeled chip region; a first regression loss is calculated based on the first chip region and the labeled chip region. In other words, the second regression loss is calculated based on the bounding box coordinates of the second chip region and the labeled chip region; the first regression loss is calculated based on the bounding box coordinates of the first chip region and the labeled chip region. Then, the regression loss function is determined by summing the product of the second regression loss and the balance coefficient, the confidence level of the second chip region, and the first regression loss. Optionally, it can be expressed by the following formula:

[0095]

[0096] In the formula, λ represents the confidence level of the second chip region. D As a balance coefficient These are the coordinates of the bounding box of the labeled chip area. These are the bounding box coordinates of the first chip region. These are the bounding box coordinates of the second chip region. As can be seen from the above formula, the regression loss function is constructed in the same way as the classification loss function. The regression loss function also consists of the true label loss. (i.e., first regression loss) and distillation loss (That is, it is the product of the second regression loss, the balance coefficient, and the confidence score of the second chip region.) Similarly, the confidence score of the second chip region output by the second target detection model is used. The distillation loss function is restricted only when... Only when the value is relatively high will it contribute to the final loss function of the first object detection model.

[0097] Finally, the total loss function is determined by combining the target loss function, classification loss function, and regression loss function. In some embodiments, the total loss function is obtained by adding the three together. Optionally, the total loss function L... final This can be expressed by the following formula:

[0098]

[0099] According to this application, a first target detection model is trained based on backpropagation of the total loss function, so that the output distribution of the first target detection model is as similar as possible to the output distribution of the second target detection model, thereby improving the detection accuracy of the first target detection model.

[0100] According to some embodiments of this application, the second object detection model is the PP-YOLO model. The training process of the PP-YOLO model will not be elaborated upon here. Considering that the training samples used in this application are mostly dense small-object chips, in some embodiments, the training samples are optimized to generate an enhanced chip dataset when training the second object detection model. The enhanced chip dataset is then used to train the second object detection model.

[0101] Specifically, since the LED chip images are acquired by the image acquisition unit built in the vision inspection system 200, the number of samples in the dataset is relatively small due to limitations in imaging conditions and the number of chip samples. This necessitates using a pre-trained weight model from a publicly available dataset to fine-tune the second object detection model with only a few thousand images and limited training time, achieving fast convergence while maintaining good generalization performance. This is in contrast to building massive datasets from scratch at great cost and spending a significant amount of time on training.

[0102] The data in public datasets mainly involve low-resolution images (640x480) with considerable target and pixel coverage (average coverage of 60% of the image height). However, in the LED chip positioning of this application, small target type chips occupy only 5% or less of the image size and are clustered (e.g., Figure 8 (As shown). Due to the significant scale difference between the small target chip data and the pre-training data, the detection accuracy of small targets is significantly reduced. Furthermore, the small size of the chip after downsampling the input image also results in less usable information, making it difficult to extract features that effectively distinguish between qualified and defective chips.

[0103] Therefore, in this application, some optimization is needed for the training samples of the second objective function. According to the implementation method of this application, the enhanced chip dataset is generated in the following manner.

[0104] First, obtain LED chip images containing different types of chips as the original chip images. The original chip images can be the training samples mentioned above. For an explanation of the original chip images, please refer to the explanation of the training samples above, which will not be repeated here.

[0105] Secondly, the original chip image is sliced ​​to generate an enhanced chip image.

[0106] In some embodiments, the original chip image is segmented into multiple image regions with overlapping areas according to preset parameters; each image region is then enlarged to obtain corresponding enhanced chip images. The preset parameters include slice size and overlap rate. Each image region is considered a slice, the slice size refers to the size of the image region, and the overlap rate refers to the proportion of the overlapping area between two adjacent image regions. Furthermore, in some embodiments, the preset parameters, such as the slice size, can be determined based on the image sizes in a pre-trained public dataset, aiming to minimize the size difference between the enhanced chip image and the pre-trained image. Additionally, when enlarging each image region, adjustments are made while preserving the original aspect ratio of the image region to maintain the original size proportion of the chip in the image.

[0107] Then, the original chip image and the enhanced chip image are used as the enhanced chip dataset. Figure 9 A schematic diagram of an enhanced chip dataset according to some embodiments of this application is shown.

[0108] like Figure 9 As shown, the pre-trained images I1, I2, ... serve as the pre-trained dataset, and the original chip images... As a chip dataset, the original chip image is sliced ​​into image regions with some overlap based on slice size and overlap rate: While preserving the image aspect ratio, the size of the image region is adjusted to produce an enhanced chip image with a larger chip size relative to the original chip image: I1',I'2,...,I' k The enhanced chip images I1', I'2, ..., I' will be displayed. k Compared with the original chip image Together, they serve as the enhanced chip dataset. According to this embodiment, by mixing the original scale data with the large-scale sub-image data, the scale difference with the public dataset data (i.e., the pre-training dataset) is mitigated, thereby improving the detection performance of the second object detection model for small object chips.

[0109] According to this application, the acquired original chip images are sliced ​​to obtain enhanced chip images. These enhanced chip images are then added to the LED chip dataset to form an enhanced chip dataset, which is used for training the second target detection model. This reduces the size difference between the enhanced and pre-trained images, thereby improving the detection accuracy of densely packed small target chips.

[0110] Subsequently, based on the second target detection model, the first target detection model is trained using the knowledge distillation model. This allows the first target detection model to reduce computation and parameters while maintaining detection accuracy, thereby improving the model's inference speed and reducing memory usage.

[0111] The following describes the LED chip positioning process.

[0112] like Figure 5 As shown, method 500 begins at 510. In 510, the image of the LED chip to be detected is segmented into multiple image regions with overlapping areas, and each image region is magnified to obtain a corresponding slice image.

[0113] As mentioned earlier, an image of the LED chip wafer disk after angle correction is acquired to obtain the image of the LED chip to be inspected. The process of slicing the image of the LED chip to be inspected can refer to the process of generating an enhanced chip image from the original chip image mentioned earlier, and will not be repeated here. The preset parameters (slice size and overlap rate) can be set as hyperparameters.

[0114] In step 520, the image of the LED chip to be detected and each slice image are input into the first target detection model for processing, resulting in the LED chip detection results for each image. The detection results include the chip detection area and its corresponding chip type.

[0115] Regarding the first target detection model, please refer to the relevant descriptions above; they will not be repeated here. Regarding the chip detection area and chip type, please refer to the labeled data above; they will not be repeated here.

[0116] According to embodiments of this application, for each LED chip image to be detected, multiple sub-images, i.e., sliced ​​images, are derived after slicing. During target detection, in addition to inputting the LED chip image to be detected into the first target detection model to predict the chip detection area and its corresponding chip type, each sliced ​​image is also input into the first target detection model to output the chip detection area and its corresponding chip type. By enhancing the sliced ​​image to be detected, the coverage area of ​​the chip is increased while expanding the feature region of small target chips, thereby alleviating the feature loss due to excessive convolution. At the same time, the sliced ​​images are magnified images of each image region, which alleviates the target aggregation phenomenon and avoids the problem of feature aggregation of multiple targets in deep features.

[0117] In step 530, the LED chip detection areas in each image are merged to determine the LED chip area and chip type of the image to be detected.

[0118] In some embodiments, non-maximum suppression is employed to transform the LED chip detection regions in each image onto the LED chip image to be detected and merge them to obtain the LED chip region and its corresponding chip type. It should be understood that "each image" here includes both the LED chip image to be detected and each slice image.

[0119] According to this application, after obtaining the LED chip region and chip type, based on the need for positioning and screening, chips of a predetermined type are further screened out from the LED chip image to be inspected based on the LED chip region and chip type. In some embodiments, the predetermined type is a defective chip, such as a blank chip or a dirty chip, and such defective or unqualified chips are screened out from the LED chip image to be inspected. In this way, only qualified chips remain in the LED chip image to be inspected. Subsequently, each qualified chip can be cropped from the LED chip image to be inspected based on its chip detection area for subsequent processing.

[0120] To illustrate the effectiveness of method 500 according to this application in chip localization and screening tasks, comparative experiments were conducted based on some embodiments of this application. The experimental results are shown in Table 1. In addition to the first and second object detection models, a third object detection model and the original PP-YOLO model were also used in the experiments. The third object detection model directly replaced ResNet50-vd in the second object detection model with MobileNetV3, without using a knowledge distillation model for training. The introduced metrics include: mAP (mean Average Precision, the average AP across all categories, AP is evaluated by calculating the area under the PR curve), which measures detection accuracy; FPS (GPU) (inference speed on high-performance computing devices (V100)) and FPS (CPU) (inference speed on low-floating-point computing devices (SD855)), which measures detection efficiency; and model size, which evaluates the memory usage of the trained model.

[0121] Table 1 Performance Comparison of Different Target Detection Models

[0122]

[0123] As shown in Table 1, the second object detection model trained using the enhanced chip dataset compensates for the shortcomings in accuracy for detecting dense small objects, and its detection accuracy is significantly improved compared to the original PP-YOLO model. Based on this, the detection accuracy of the first to third object detection models is also improved. Specifically, the first object detection model shows a 0.73% increase in mAP compared to the third object detection model, which did not use knowledge distillation for accuracy improvement, verifying the effectiveness of the knowledge distillation strategy in mitigating the decrease in detection accuracy after lightweight improvements. Meanwhile, the knowledge distillation strategy does not improve the model's network structure; therefore, its inference speed on GPUs, inference speed on low-floating-point CPUs, and model size are not significantly different from the third object detection model, and its performance far exceeds that of the original second object detection model. In principle, the first object detection model effectively utilizes the knowledge transferred from the second object detection model, better learning the complex features and relationships of LED chips based on the original training, thus achieving a slight improvement in accuracy compared to before distillation, approaching the detection accuracy of the second object detection model.

[0124] In summary, the first target detection model effectively mitigates the decrease in detection accuracy after the original PP-YOLO network is implemented with lightweight improvements. Furthermore, compared to the second target detection model with higher detection accuracy, its network inference speed is significantly improved, and its memory usage is greatly reduced. Therefore, the LED chip localization scheme based on this application significantly reduces model inference time and algorithm hardware requirements while still meeting the accuracy requirements for LED chip detection, making it well-suited for practical application in LED chip localization and screening tasks.

[0125] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of this application, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing this application.

[0126] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the knowledge distillation-based LED chip positioning scheme of this application according to instructions in the program code stored in the memory.

[0127] This application also discloses:

[0128] A7. The method as described in any one of A4-6, wherein calculating the regression loss function based on the second chip region and its corresponding second chip region confidence, the first chip region, and the labeled chip region includes: calculating a second regression loss based on the second chip region and the labeled chip region; calculating a first regression loss based on the first chip region and the labeled chip region; and determining the regression loss function by summing the product of the second regression loss and the balance coefficient, the second chip region confidence, and the first regression loss. A8. The method as described in any one of A1-7 further includes: training a second target detection model using the enhanced chip dataset, wherein the enhanced chip dataset is generated by: acquiring LED chip images containing different types of chips as original chip images; slicing the original chip images to generate enhanced chip images; and using the original chip images and the enhanced chip images as the enhanced chip dataset. A9. The method as described in A8, wherein slicing the original chip images to generate enhanced chip images includes: dividing the original chip images into multiple image regions with overlapping areas according to preset parameters; and enlarging each image region to obtain corresponding enhanced chip images. A10. The method of any one of A1-9, wherein merging the LED chip detection regions in each image to determine the LED chip region and chip type of the LED chip image to be detected includes: using non-maximum suppression to convert the LED chip detection regions in each image onto the LED chip image to be detected for merging, thereby obtaining the LED chip region and its corresponding chip type. A11. The method of any one of A1-10 further includes: based on the LED chip region and chip type, filtering out chips of a predetermined type from the LED chip image to be detected.

[0129] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0130] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this application. Based on the above description, the required structure for constructing such a system is obvious. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of this application.

[0131] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0132] Similarly, it should be understood that, for the purpose of simplification and aiding understanding of one or more aspects of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the application.

[0133] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0134] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0135] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0136] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the method or method elements forms means for implementing the method or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by the elements for the purposes of this disclosure.

[0137] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, sequence, or any other manner. Furthermore, the quantifier “multiple” means “two” and / or “more than two.”

[0138] Although this application has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of this application described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for interpreting or limiting the subject matter of this application. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of this application is illustrative and not restrictive, and the scope of this application is defined by the appended claims.

Claims

1. A knowledge distillation-based LED chip positioning method, including: The image of the LED chip to be detected is divided into multiple image regions with overlapping areas, and each image region is magnified to obtain a slice image. The LED chip image to be detected and each slice image are respectively input into the first target detection model for processing to obtain the LED chip detection results in each image. The detection results include the chip detection area and its corresponding chip type. The first target detection model is obtained by supervised training with the detection results of the second target detection model. The first target detection model includes a first feature extraction network component and a prediction component. The second target detection model includes a second feature extraction network component and a prediction component. The computational amount and number of parameters of the first feature extraction network component are less than those of the second feature extraction network component. as well as The LED chip detection areas in each image are merged to determine the LED chip area and chip type of the LED chip image to be detected; The second target detection model is trained and generated based on the enhanced chip dataset, which is generated in the following way: Obtain LED chip images containing different types of chips as the original chip images; The original chip image is sliced ​​to generate an enhanced chip image; The original chip image and the enhanced chip image are used as the enhanced chip dataset.

2. The method of claim 1, wherein, The first object detection model was trained in the following way: The training samples are labeled to obtain labeled data, which includes labeled chip regions and label types. The training samples are input into the second target detection model, and after processing, the second chip region and the second type are output. The training samples are input into the initial first target detection model, and after processing, the first chip region and the first type are output. The total loss function is calculated by combining at least the labeled data, the first chip region and the first type, the second chip region and the second type, and the parameters of the first target detection model are adjusted based on the total loss function until the training is completed, thus obtaining the first target detection model.

3. The method of claim 2, wherein, At least in conjunction with the labeled data, the first chip region and first type, the second chip region and second type, calculate the total loss function, including: Calculate the target loss function based on the second chip region, the first chip region, and the labeled chip region; Based on the second type and its corresponding second chip region confidence, the first type, and the labeling type, calculate the classification loss function; Based on the second chip region and its corresponding second chip region confidence, the first chip region, and the labeled chip region, a regression loss function is calculated. The total loss function is determined by combining the target loss function, the classification loss function, and the regression loss function.

4. The method of claim 3, wherein, The step of calculating the target loss function based on the second chip region, the first chip region, and the labeled chip region includes: The second target loss is calculated based on the second chip region and the labeled chip region; Calculate the first target loss based on the first chip region and the labeled chip region; The product of the second target loss and the balance coefficient is used to determine a target loss function.

5. The method of claim 3 or 4, wherein, The classification loss function is calculated based on the second type, the corresponding second chip region confidence, the first type, and the labeled type. The second classification loss is calculated based on the second type and the labeled type. The first classification loss is calculated based on the first type and the labeled type. The product of the second classification loss and the balance coefficient and the second chip region confidence is used to determine a classification loss function.

6. The method of claim 5, wherein, The regression loss function is calculated based on the second chip region, the corresponding second chip region confidence, the first chip region, and the labeled chip region. The second regression loss is calculated based on the second chip region and the labeled chip region. The first regression loss is calculated based on the first chip region and the labeled chip region. The product of the second regression loss and the balance coefficient and the second chip region confidence is used to determine a regression loss function.

7. The method of claim 1, wherein, The original chip image is sliced to generate an enhanced chip image, including: The original chip image is segmented into multiple image regions with overlapping areas according to a preset parameter. Each image region is enlarged to obtain a corresponding enhanced chip image.

8. The method of claim 1, wherein, The LED chip detection regions in each image are merged to determine the LED chip region and the chip type of the LED chip image to be detected, including: Non-maximum suppression is used to convert the LED chip detection regions in each image to the LED chip image to be detected for merging to obtain the LED chip region and the corresponding chip type.

9. The method of claim 1, further comprising: Based on the LED chip region and the chip type, the chips of a predetermined type are filtered out from the LED chip image to be detected.

10. An LED chip positioning device, comprising: An image processing unit adapted to segment the LED chip image to be detected into multiple image regions with overlapping areas, and to enlarge each image region to obtain a corresponding slice image; A detection unit adapted to input the LED chip image to be detected and each slice image into a first target detection model for processing to obtain an LED chip detection result in each image, the detection result including a chip detection region and a corresponding chip type; A chip positioning unit adapted to merge the LED chip detection regions in each image to determine the LED chip region and the chip type of the LED chip image to be detected; A model training unit adapted to train the first target detection model using the detection result of a second target detection model as supervision, wherein the first target detection model includes a first feature extraction network component and a prediction component, the second target detection model includes a second feature extraction network component and a prediction component, and the calculation amount and parameter amount of the first feature extraction network component are less than those of the second feature extraction network component. The second target detection model is trained and generated based on the enhanced chip dataset, which is generated in the following way: Obtain LED chip images containing different types of chips as the original chip images; The original chip image is sliced ​​to generate an enhanced chip image; The original chip image and the enhanced chip image are used as the enhanced chip dataset.

11. A computing device, comprising: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method as described in any one of claims 1-9.

12. A computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method as claimed in any one of claims 1-9.

Images