Computing resource scheduling method, optical detection system and semiconductor equipment

Abstract

The invention provides a computing resource scheduling method, an optical detection system and semiconductor equipment, which can be applied to the technical field of optical detection. The computing resource scheduling method is applied to an optical detection system, the optical detection system comprises an image acquisition unit, a pre-processing unit and a computing service unit, and the computing resource scheduling method comprises the following steps: monitoring a first task load of the pre-processing unit; when the first task load is greater than a first threshold value, migrating a part of image preprocessing tasks of the preprocessing unit to the computing service unit; when the first task load is smaller than a second threshold value, stopping migrating the image preprocessing task to the computing service unit; wherein the first threshold value is greater than the second threshold value. According to the method, load balancing and resource maximization utilization of the optical detection system are realized based on multi-stage dynamic load awareness and cooperative task allocation, and the utilization rate of computing resources of the optical detection system is improved.

Description

Technical Field

[0001] This application relates to the field of optical inspection technology, and in particular to a computing resource scheduling method, an optical inspection system, and a semiconductor device. Background Technology

[0002] Semiconductor optical inspection is a key step in the semiconductor manufacturing process to ensure product quality and yield. It uses high-precision optical imaging technology and image analysis algorithms to perform detailed inspections on semiconductor wafers and chips at each stage of manufacturing to accurately identify surface defects, dimensional deviations, structural anomalies and other problems.

[0003] As semiconductor processes continue to evolve towards smaller processes and higher integration, and as AI algorithms and other image analysis technologies are widely applied in the field of semiconductor optical inspection, the amount of data generated by optical inspection systems during the inspection process has exploded in order to cope with increasingly complex optical inspection tasks. This massive amount of data places extremely high demands on the data processing capabilities of optical inspection systems.

[0004] However, the current unreasonable scheduling of computing resources in semiconductor optical inspection systems prevents the various units in the optical inspection system from working together efficiently, resulting in the system's performance not being fully utilized and failing to meet the urgent needs of semiconductor optical inspection systems for efficient and accurate detection. Summary of the Invention

[0005] This application provides a computing resource scheduling method, an optical inspection system, and a semiconductor device to solve the technical problem that current semiconductor optical inspection systems cannot fully utilize their system performance due to unreasonable computing resource scheduling methods, making it difficult to meet the urgent needs of semiconductor optical inspection systems for efficient and accurate detection.

[0006] According to a first aspect disclosed in this application, this application provides a computational resource scheduling method applied to an optical inspection system. The optical inspection system includes an image acquisition unit, a preprocessing unit, and a computational service unit. The image acquisition unit is used to acquire image data, the preprocessing unit is used to perform image preprocessing operations on the image data, and the computational service unit is used to perform image analysis operations on the image data after the image preprocessing operations. The method includes:

[0007] Monitor the first task load of the preprocessing unit;

[0008] When the first task load exceeds the first threshold, part of the image preprocessing task of the preprocessing unit is migrated to the computing service unit;

[0009] When the first task load is less than the second threshold, the migration of the image preprocessing task to the computing service unit is stopped; wherein the first threshold is greater than the second threshold.

[0010] In this scheme, when the load of the first task exceeds a first threshold, some image preprocessing tasks of the preprocessing unit are migrated to the computing service unit. This proactively offloads computational pressure to the computing service unit when the preprocessing unit faces overload risk, thus improving the overall processing efficiency of the system during peak periods and under high loads, and preventing task backlog, system lag, or even data loss due to insufficient preprocessing capacity. Therefore, through the coordinated scheduling of computing resources between the preprocessing unit and the computing service unit, efficient utilization of system computing resources is achieved, fully leveraging system performance.

[0011] Meanwhile, when the load of the first task is less than the second threshold, the migration of image preprocessing tasks to the computing service unit is stopped, and the first threshold is set to be greater than the second threshold. This ensures that the load point at which migration stops is lower than the load point at which migration starts. This avoids frequent and repeated switching of the task migration strategy when the system load fluctuates slightly around a single threshold, reducing the additional overhead caused by frequent changes in the task migration strategy and enhancing the stability of system operation.

[0012] In one feasible implementation, the method further includes:

[0013] When the first task load is less than the third threshold, at least a portion of the image preprocessing tasks that have been migrated to the computing service unit are migrated back to the preprocessing unit; wherein the third threshold is less than the second threshold.

[0014] In this embodiment, when the load of the first task is less than the third threshold, at least a portion of the image preprocessing tasks that have been migrated to the computing service unit are migrated back to the preprocessing unit. This efficiently allocates idle computing resources in the preprocessing unit, achieving efficient utilization of system computing resources. Furthermore, setting the third threshold to be less than the second threshold ensures that the preprocessing unit is sufficiently idle and capable of taking over tasks, preventing the task migration from being triggered again immediately after the migration due to small load fluctuations.

[0015] In one feasible implementation, the computing service unit includes multiple computing servers, and the method further includes:

[0016] When the preprocessing unit distributes the task to be processed to the computing service unit, it obtains the second task load of each computing server; wherein the task to be processed is an image preprocessing task or an image analysis task.

[0017] If a computing server has a second task load less than the fourth threshold, then the task to be processed is assigned to the corresponding computing server.

[0018] In this embodiment, load balancing between computing servers is achieved through load awareness and task allocation across computing servers, solving the problem of inefficient scheduling of computing tasks between computing servers, and ensuring that all computing servers are in a high-efficiency operating state, so as to achieve efficient utilization of computing resources.

[0019] In one feasible implementation, the preprocessing unit further includes a storage unit, and the method further includes:

[0020] If there is no computing server with a second task load less than the fourth threshold, then the task to be processed is stored in the storage unit.

[0021] In this embodiment, overload protection and task buffering are implemented through the storage unit, thereby effectively avoiding task blocking and data loss.

[0022] In one feasible implementation, the method further includes:

[0023] Monitor the second task load of each of the aforementioned computing servers;

[0024] When a computing server has a second task load less than the fourth threshold, the tasks to be processed in the storage unit are loaded into the preprocessing unit for distribution.

[0025] In this embodiment, by loading the pending tasks in the storage unit and distributing them for processing when the computing server is under low load, efficient utilization of the computing server's computing resources is achieved.

[0026] In one feasible implementation, obtaining the second task load of each of the computing servers includes:

[0027] Obtain the first queue length of the tasks to be processed in the task queue of each computing server, and the first task processing speed of each computing server in the previous time window;

[0028] Based on the first queue length and first task processing speed of each computing server, a first estimated processing time for each computing server to process the task to be processed is obtained; wherein, the first estimated processing time is used to characterize the second task load.

[0029] In this embodiment, the load of a computing server is characterized by its estimated task processing time. This not only intuitively reflects the current amount of tasks pending on the computing server but also, combined with the server's historical processing efficiency, dynamically and accurately reflects the actual task load status, facilitating more rational resource allocation and scheduling. Furthermore, even with identical data such as GPU utilization and memory usage, different computing servers exhibit varying processing speeds and queue backlogs, making it impossible to evaluate their load status on a single dimension. Therefore, estimating processing time places the load data of each computing server on the same dimension, more accurately reflecting their load status.

[0030] In one feasible implementation, the computing server includes multiple sub-computing units, and the method further includes:

[0031] When the computing server receives the task to be processed, it obtains the third task load of each of the sub-computing units;

[0032] If a sub-computing unit has a third task load that is less than the fifth threshold, then the task to be processed is assigned to the corresponding sub-computing unit.

[0033] In this embodiment, by load perception and task allocation of sub-computing units within the computing server, and based on the load assessment results, the computing tasks allocated to the computing server are redistributed, thereby achieving load balancing among sub-computing units. This solves the problem of inefficient scheduling of computing tasks among sub-computing units, ensuring that all sub-computing units are in a high-efficiency operating state, thus achieving efficient utilization of computing resources.

[0034] In one feasible implementation, obtaining the task load of each of the sub-computing units includes:

[0035] Obtain the second queue length of the tasks to be processed in the subtask queue of each of the sub-computing units, and the second task processing speed of each of the sub-computing units in the previous time window;

[0036] Based on the second queue length and second task processing speed of each of the sub-computing units, a second estimated processing time for each of the sub-computing units to process the task to be processed is obtained; wherein, the second estimated processing time is used to characterize the third task load.

[0037] In this embodiment, the load of a sub-computing unit is characterized by calculating its estimated processing time. This not only intuitively reflects the current number of tasks to be processed by the sub-computing unit, but also, combined with the historical processing efficiency of the sub-computing unit, dynamically and accurately reflects its actual task load status, which helps to allocate and schedule resources for the sub-computing unit more rationally. Furthermore, even if GPU utilization and memory usage are the same, the processing speed and queue backlog of different sub-computing units will vary, making it impossible to evaluate the load status of sub-computing units on the same dimension. Therefore, estimating processing time places the load data of each sub-computing unit on the same dimension, more accurately reflecting the load status of the sub-computing unit.

[0038] According to the second aspect disclosed in this application, this application provides an optical detection system, including an image acquisition unit, a preprocessing unit, and a computing service unit;

[0039] The image acquisition unit is used to acquire image data;

[0040] The preprocessing unit is used to perform image preprocessing operations on the image data;

[0041] The computing service unit is used to perform image analysis operations on the image data after the image preprocessing operation;

[0042] The preprocessing unit is used for:

[0043] Monitor the first task load of the preprocessing unit;

[0044] When the first task load exceeds the first threshold, part of the image preprocessing task of the preprocessing unit is migrated to the computing service unit;

[0045] When the first task load is less than the second threshold, the migration of the image preprocessing task to the computing service unit is stopped; wherein the first threshold is greater than the second threshold.

[0046] In this scheme, when the load of the first task exceeds a first threshold, some image preprocessing tasks of the preprocessing unit are migrated to the computing service unit. This proactively offloads computational pressure to the computing service unit when the preprocessing unit faces overload risk, thus improving the overall processing efficiency of the system during peak periods and under high loads, and preventing task backlog, system lag, or even data loss due to insufficient preprocessing capacity. Therefore, through the coordinated scheduling of computing resources between the preprocessing unit and the computing service unit, efficient utilization of system computing resources is achieved, fully leveraging system performance.

[0047] Meanwhile, when the load of the first task is less than the second threshold, the migration of image preprocessing tasks to the computing service unit is stopped, and the first threshold is set to be greater than the second threshold. This ensures that the load point at which migration stops is lower than the load point at which migration starts. This avoids frequent and repeated switching of the task migration strategy when the system load fluctuates slightly around a single threshold, reducing the additional overhead caused by frequent changes in the task migration strategy and enhancing the stability of system operation.

[0048] In one feasible implementation, the preprocessing unit includes a preprocessing load sensing unit and a preprocessing load migration unit.

[0049] The preprocessing load sensing unit is used to monitor the first task load of the preprocessing unit;

[0050] The preprocessing load migration unit is used to migrate part of the image preprocessing tasks of the preprocessing unit to the computing service unit when the first task load is greater than a first threshold, and to stop migrating the image preprocessing tasks to the computing service unit when the first task load is less than a second threshold.

[0051] In this embodiment, the preprocessing load sensing unit dynamically senses the load of the preprocessing unit and the computing service unit. Based on the load sensing results, the load migration unit will migrate some image preprocessing tasks to the computing service unit for processing when the preprocessing unit's task load is greater than a first threshold, and stop the task migration when the preprocessing unit's task load is less than a second threshold, so as to achieve system load balancing and maximize the utilization of computing resources.

[0052] In one feasible implementation, the preprocessing load migration unit is used to migrate at least a portion of the image preprocessing tasks that have been migrated to the computing service unit back to the preprocessing unit when the first task load is less than a third threshold; wherein the third threshold is less than the second threshold.

[0053] In this embodiment, when the preprocessing unit's task load is less than a third threshold, the preprocessing load migration unit migrates at least a portion of the image preprocessing tasks that have been migrated to the computing service unit back to the preprocessing unit, thereby achieving efficient utilization of the preprocessing unit's idle computing resources.

[0054] In one feasible implementation, the computing service unit includes a cross-server load awareness unit, a cross-server task allocation unit, and multiple computing servers;

[0055] The cross-server load sensing unit is used to dynamically sense the second task load of each of the computing servers, and the cross-server task allocation unit is used to allocate the pending tasks assigned to the computing service unit to each of the computing servers according to the evaluation result of the second task load; wherein, the pending tasks are image preprocessing tasks or image analysis tasks.

[0056] In this embodiment, cross-server load perception and task allocation are achieved through cross-server load perception unit and cross-server task allocation unit, thus realizing load balancing between computing servers and solving the problem of inefficient scheduling of computing tasks between computing servers. This ensures that all computing servers are in a high-efficiency operating state, thereby achieving efficient utilization of computing resources.

[0057] In one feasible implementation, the cross-server load awareness unit is used to obtain the first queue length of the pending tasks in the task queue of each computing server, and the first task processing speed of each computing server in the previous time window, and to obtain the first estimated processing time of each computing server for processing the pending tasks based on the first queue length and the first task processing speed of each computing server; wherein, the first estimated processing time is used to characterize the second task load.

[0058] In this embodiment, the estimated processing time of computing servers is used by the server load awareness unit to characterize the computing server load. This not only intuitively reflects the current amount of tasks pending on the computing server, but also, combined with the historical processing efficiency of the computing server, dynamically and accurately reflects the actual task load status of the computing server, which helps to allocate and schedule computing server resources more rationally. Furthermore, even if GPU utilization and memory usage are the same, the processing speed and queue backlog of different computing servers will vary, making it impossible to evaluate the computing server load status on the same dimension. Therefore, by estimating the processing time, the load data of each computing server can be placed on the same dimension, more accurately reflecting the computing server load status.

[0059] In one feasible implementation, each of the computing servers includes an in-server load awareness unit, an in-server task allocation unit, and multiple sub-computing units.

[0060] The server load sensing unit is used to dynamically sense the third task load of each of the sub-computing units, and the server task allocation unit is used to allocate the tasks to be processed on the computing server to each of the sub-computing units according to the evaluation result of the third task load.

[0061] In this embodiment, load perception and task allocation of sub-computing units within the computing server are realized through the server-side load perception unit and server-side task allocation unit. Based on the load assessment results, the computing tasks allocated to the computing server are redistributed, thereby achieving load balancing among sub-computing units. This solves the problem of inefficient scheduling of computing tasks among sub-computing units, ensuring that all sub-computing units are in a high-efficiency operating state, thus achieving efficient utilization of computing resources.

[0062] In one feasible implementation, the server load awareness unit is used to obtain the second queue length of the pending tasks in the sub-task queue of each of the sub-computing units, and the second task processing speed of each of the sub-computing units in the previous time window, and to obtain the second estimated processing time for each of the sub-computing units to process the pending tasks based on the second queue length and the second task processing speed of each of the sub-computing units; wherein, the second estimated processing time is used to characterize the third task load.

[0063] In this embodiment, the estimated processing time of sub-computing units is calculated by the load awareness unit within the server to characterize the load of sub-computing units. This not only intuitively reflects the current amount of tasks pending processing in a sub-computing unit, but also, combined with the historical processing efficiency of the sub-computing unit, dynamically and accurately reflects the actual task load status of the sub-computing unit, which helps to allocate and schedule resources for sub-computing units more rationally. Furthermore, even if GPU utilization and memory usage are the same, the processing speed and queue backlog of different sub-computing units will vary, making it impossible to evaluate the load status of sub-computing units on the same dimension. Therefore, estimating processing time places the load data of each sub-computing unit on the same dimension, more accurately reflecting the load status of the sub-computing units.

[0064] In one feasible implementation, the preprocessing unit includes a storage unit and an overload protection unit;

[0065] The overload protection unit is used to temporarily store the task to be processed in the storage unit when there is no computing server with a second task load less than the fourth threshold, and to release the task to be processed in the storage unit to the preprocessing unit for distribution when there is a computing server with a second task load less than the fourth threshold.

[0066] In this embodiment, overload protection and task buffering of the computing service unit are achieved through a storage unit and an overload protection unit, thereby effectively avoiding task blocking and data loss.

[0067] In one feasible implementation, the optical inspection system further includes a host computer, a management service unit, and a management switch;

[0068] The management service unit is used to manage the computing service unit, summarize the calculation results of the computing service unit, and upload the calculation results to the host computer through the management switch.

[0069] In this embodiment, the management service unit enables centralized management of the computing servers in the computing service unit, thereby improving the operational efficiency of the computing service unit; the host computer enables centralized monitoring, centralized management and error handling of the entire system, improving the user interaction experience and work efficiency; the management switch enables precise forwarding and efficient control of data, ensuring the stability and security of data transmission.

[0070] According to a third aspect disclosed in this application, this application provides a semiconductor device comprising an optical detection system as described in any one of the second aspects.

[0071] In this solution, based on multi-level dynamic load perception and collaborative task allocation, load balancing and maximum resource utilization of semiconductor devices are achieved, effectively improving the utilization rate of semiconductor device computing resources. Attached Figure Description

[0072] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0073] Figure 1 This application provides a schematic diagram of the system architecture of an optical detection system.

[0074] Figure 2 This is a schematic diagram of the structure of an optical detection system provided in an embodiment of this application;

[0075] Figure 3 A flowchart illustrating a computing resource scheduling method provided in an embodiment of this application;

[0076] Figure 4 A flowchart illustrating a dynamic task scheduling method among computing servers provided in an embodiment of this application;

[0077] Figure 5 This is a flowchart illustrating a dynamic task scheduling method within a computing server, as provided in an embodiment of this application.

[0078] Explanation of reference numerals in the attached figures:

[0079] 1000-Optical Inspection System;

[0080] 1100 - Image Acquisition Unit;

[0081] 1110 - Camera;

[0082] 1120 - Acquisition Card;

[0083] 1200 - Preprocessing unit;

[0084] 1210 - Storage unit;

[0085] 1220 - Overload Protection Unit;

[0086] 1230 - Pre-processing load sensing unit;

[0087] 1240 - Preprocessing load migration unit;

[0088] 1300 - Computing Service Unit;

[0089] 1310 - Cross-server load awareness unit;

[0090] 1320 - Cross-server task allocation unit;

[0091] 1330 - Computing Server;

[0092] 1331 - Server-side load sensing unit;

[0093] 1332 - Server-wide task allocation unit;

[0094] 1340 - Sub-computing unit;

[0095] 1400-Data Switch;

[0096] 1500-Management Switch;

[0097] 1600 - Management Service Unit;

[0098] 1700 - Host Computer.

[0099] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0100] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0101] Semiconductor optical inspection is a crucial step in the semiconductor manufacturing process to ensure product quality and yield. It utilizes high-precision optical imaging technology and image analysis algorithms to meticulously inspect semiconductor wafers and chips at each manufacturing stage, accurately identifying surface defects, dimensional deviations, structural anomalies, and other issues. For example, in wafer manufacturing, which involves multiple processes such as photolithography, etching, coating, and chemical mechanical polishing (CMP), semiconductor optical inspection technology can capture process defects at each stage in real time, preventing defective wafers from entering subsequent processes and causing the entire batch to be scrapped, thus achieving full-process monitoring of wafer manufacturing.

[0102] Semiconductor optical inspection systems are key systems that perform non-contact optical inspection and measurement of wafers, chips, and other components in the semiconductor manufacturing process based on optical principles and using high-precision optical and image processing equipment.

[0103] Figure 1 A schematic diagram of the system architecture of an optical detection system 1000 provided in this application embodiment is shown below. Figure 1 The system architecture includes an image acquisition unit 1100, a preprocessing unit 1200, and a computing service unit 1300, which are connected sequentially. The image acquisition unit 1100 acquires image data of the semiconductor to be tested and transmits the acquired image data to the preprocessing unit 1200. The preprocessing unit 1200 performs image preprocessing operations on the image data and transmits the processed image data to the computing service unit 1300. The computing service unit 1300 performs image analysis operations on the preprocessed image data and obtains the final optical detection result.

[0104] Specifically, the image acquisition unit 1100 includes a camera 1110, which converts light signals into electrical signals through a photosensitive chip (such as CCD or CMOS) and outputs digital images; the preprocessing unit 1200 includes an embedded processor (FPGA), which can perform image preprocessing directly on the device without relying on external computing resources; the computing service unit 1300 includes a cluster of computing servers 1330, which can efficiently support complex image analysis algorithms and ensure the real-time performance and reliability of the detection system.

[0105] With the emergence of advanced semiconductor manufacturing technologies such as EUV lithography (Extreme Ultraviolet Lithography), semiconductor processes are continuously evolving towards smaller process sizes and higher integration levels. Simultaneously, the widespread application of image analysis technologies such as AI algorithms in the field of semiconductor optical inspection has led to an explosive growth in the amount of data generated by the Optical Inspection System 1000 during inspection processes to cope with increasingly complex optical inspection tasks. This has caused the data volume that the Optical Inspection System 1000 needs to process per second to soar to tens to hundreds of gigabytes. Therefore, in the current context of increasingly stringent requirements for optical inspection accuracy and speed, such a massive amount of data places extremely high demands on the data processing capabilities of the Optical Inspection System 1000.

[0106] However, most current computational resource scheduling algorithms in the semiconductor optical inspection system 1000 still operate independently on each unit, considering only the current unit's load. With the continuous increase in semiconductor optical inspection data and the widespread application of AI algorithms in the inspection field, the demand for computing power has increased significantly. This unreasonable allocation of computational resources prevents the various units in the optical inspection system 1000 from working efficiently and collaboratively, resulting in underperformance. For example, insufficient preprocessing capabilities can lead to task backlog, system lag, and even data loss. This makes it difficult to meet the urgent need for efficient and accurate inspection in the semiconductor optical inspection system 1000.

[0107] To address the aforementioned technical problems, this application proposes a computing resource scheduling method, an optical detection system 1000, and a semiconductor device. This computing resource scheduling method, based on multi-level dynamic load perception and collaborative task allocation, achieves load balancing and maximizes resource utilization of the system, effectively improving the utilization rate of the system's computing resources.

[0108] The technical solutions for the computing resource scheduling method, optical detection system 1000, and semiconductor device provided in this application will be described in detail below through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other; for the same or similar content, descriptions may not be repeated in different embodiments.

[0109] Figure 2 This is a schematic diagram of the structure of an optical detection system 1000 provided in an embodiment of this application. (See attached diagram.) Figure 2The optical detection system 1000 includes an image acquisition unit 1100, a preprocessing unit 1200, and a computing service unit 1300. The image acquisition unit 1100 is used to acquire image data. The preprocessing unit 1200 is used to perform image preprocessing operations on the image data. The computing service unit 1300 is used to perform image analysis operations on the image data after image preprocessing. The preprocessing unit 1200 is used to: monitor the first task load of the preprocessing unit 1200; when the first task load is greater than a first threshold, migrate part of the image preprocessing task of the preprocessing unit 1200 to the computing service unit 1300; when the first task load is less than a second threshold, stop migrating the image preprocessing task to the computing service unit 1300; wherein, the first threshold is greater than the second threshold.

[0110] Specifically, the image acquisition unit 1100 is used to capture images of the semiconductor to be tested and obtain image data. The image acquisition unit 1100 includes a camera 1110 and a data acquisition card 1120. The combined use of the camera 1110 and the data acquisition card 1120 achieves efficient collaboration of the image acquisition system. The high-performance sensor of the camera 1110 captures high-resolution, high-frame-rate raw image data, and the data acquisition card 1120 provides a high-speed, stable data transmission channel, ensuring that a large amount of image data can be imported into the preprocessing unit 1200 in real time without loss.

[0111] Specifically, the preprocessing unit 1200 performs image preprocessing operations on the image data. These operations preprocess the image data to provide a standardized image with a high signal-to-noise ratio and clear structure for subsequent image analysis, significantly improving image analysis accuracy and system robustness. The image preprocessing tasks include at least one of the following: image filtering, image correction, image denoising, contrast enhancement, geometric correction, format conversion, region extraction, histogram equalization, image binarization, and geometric correction.

[0112] The preprocessing unit 1200 can be a coarse-grained reconfigurable architecture, a general-purpose graphics processor (GPGPU), an FPGA (Field Programmable Gate Array), or a SoC FPGA (System-on-a-Chip FPGA). The first task load can be represented by parameters such as CPU utilization, memory usage, the number of image frames waiting to be processed in the processing queue, the number of images that cannot be processed in time per unit time, and the queue length of the image preprocessing task.

[0113] Specifically, the computing service unit 1300 performs image analysis operations on the image data after image preprocessing. Furthermore, the computing service unit 1300 can also execute image preprocessing tasks assigned by the preprocessing unit 1200. These image analysis tasks include defect detection, attribute calculation, false positive filtering, and defect classification. The computing service unit 1300 can utilize a server cluster or a multi-XPU (containing various types of processors) computing platform.

[0114] In this embodiment, when the first task load exceeds a first threshold, the preprocessing unit 1200 sends a task migration request to the computing service unit 1300. This request includes task metadata (e.g., task type, data volume, input / output addresses, dependencies, etc.) and a resource requirement list. Upon receiving the request, the computing service unit 1300 returns a confirmation response and reserves corresponding resources. Subsequently, the preprocessing unit 1200 compresses the task's current runtime context (e.g., register data, intermediate calculation results, process control blocks, etc.) and transmits it to the computing service unit 1300 via a communication link. During transmission, a verification algorithm (e.g., CRC32) can be used to ensure data integrity. After receiving the data, the computing service unit 1300 migrates the image preprocessing task from the preprocessing unit 1200 to the computing service unit 1300. Simultaneously, when the first task load is less than a second threshold, the preprocessing unit 1200 stops migrating the image preprocessing task to the computing service unit 1300.

[0115] The optical inspection system 1000, through load sensing and task migration of the preprocessing unit 1200, proactively offloads computational pressure to the computing service unit 1300 when the preprocessing unit 1200 faces overload risk. This improves the overall processing efficiency of the system during peak periods and under high loads, preventing task backlog, system lag, or even data loss due to insufficient preprocessing capacity. Thus, through the coordinated scheduling of computing resources between the preprocessing unit 1200 and the computing service unit 1300, efficient utilization of system computing resources is achieved, fully leveraging system performance.

[0116] Meanwhile, when the load of the first task is less than the second threshold, the migration of image preprocessing tasks to the computing service unit 1300 is stopped, and the first threshold is set to be greater than the second threshold. This makes the load point at which migration stops lower than the load point at which migration starts. This avoids frequent and repeated switching of task migration strategies when the system load fluctuates slightly around a single threshold, reduces the additional overhead caused by frequent changes in task migration strategies, and enhances the stability of system operation.

[0117] Optionally, the preprocessing unit 1200 is equipped with a task migration flag. When the first task load is greater than the first threshold, part of the image preprocessing tasks of the preprocessing unit 1200 are migrated to the computing service unit 1300, and the migration flag is set to "true". When the first task load is less than the second threshold and the migration flag is "true", the migration of image preprocessing tasks to the computing service unit 1300 is stopped, and the migration flag is set to "false".

[0118] By setting a task migration flag, the preprocessing unit 1200 and other units in the system can clearly determine whether they are in a task migration state, thus avoiding behavioral conflicts. The task migration flag serves as a "switch credential" for the migration process, ensuring the controllability and integrity of startup, execution, and shutdown. The task migration flag also simplifies system logic judgment, eliminating the need for complex status queries. The "true" or "false" status of the task migration flag can quickly coordinate the behavior of each unit.

[0119] Specifically, the first threshold and the second threshold can be flexibly set according to the performance of the preprocessing unit 1200. For example, if the task load is represented by CPU utilization, the first threshold can be set to 80% and the second threshold can be set to 50%.

[0120] See Figure 2 In some embodiments, the preprocessing unit 1200 includes a preprocessing load sensing unit 1230 and a preprocessing load migration unit 1240; the preprocessing load sensing unit 1230 is used to monitor a first task load of the preprocessing unit 1200; the preprocessing load migration unit 1240 is used to migrate part of the image preprocessing task of the preprocessing unit 1200 to the computing service unit 1300 when the first task load is greater than a first threshold, and to stop migrating the image preprocessing task to the computing service unit 1300 when the first task load is less than a second threshold.

[0121] In this embodiment, the preprocessing load sensing unit 1230 periodically calls the system API to obtain the load data of the preprocessing unit 1200 and writes the data into the shared memory area of ​​the preprocessing unit 1200. The preprocessing load migration unit 1240 reads the load data from the shared memory area and performs task migration by calling the network communication interface according to a threshold measurement. For example, if the task load is represented by CPU utilization, the preprocessing load sensing unit 1230 periodically obtains the CPU utilization of the preprocessing unit 1200.

[0122] The preprocessing load sensing unit 1230 dynamically senses the load of the preprocessing unit 1200 and the computing service unit 1300. Based on the load sensing results, the load migration unit will migrate some image preprocessing tasks to the computing service unit 1300 for processing when the task load of the preprocessing unit 1200 is greater than a first threshold, and stop the task migration when the task load of the preprocessing unit 1200 is less than a second threshold, so as to achieve system load balancing and maximize the utilization of computing resources.

[0123] Specifically, the preprocessing load sensing unit 1230 and the preprocessing load migration unit 1240 can be two independent software threads or functional modules in the load scheduling management program running on the preprocessing unit 1200.

[0124] In some embodiments, the preprocessing load migration unit 1240 is used to migrate at least a portion of the image preprocessing tasks that have been migrated to the computing service unit 1300 back to the preprocessing unit 1200 when the first task load is less than a third threshold; wherein the third threshold is less than the second threshold.

[0125] In this embodiment, when the first task load is less than the third threshold, the preprocessing unit 1200 sends a task migration request to the computing service unit 1300. After receiving the request, the computing service unit 1300 suspends the processing of the task. Subsequently, the computing service unit 1300 compresses the current running context of the task (e.g., register data, intermediate calculation results, process control block, etc.) and transmits it to the computing service unit 1300 through the original communication link. During the transmission, a verification algorithm (e.g., CRC32, etc.) can be used to ensure data integrity. After the preprocessing unit 1200 receives the data, the image preprocessing task is migrated from the computing service unit 1300 to the preprocessing unit 1200.

[0126] When the task load of the preprocessing unit 1200 is less than the third threshold, the preprocessing load migration unit 1240 migrates at least a portion of the image preprocessing tasks that have been migrated to the computing service unit 1300 back to the preprocessing unit 1200, thereby achieving efficient utilization of the idle computing resources of the preprocessing unit 1200.

[0127] Specifically, the task rollback request sent by the preprocessing unit 1200 to the computing service unit 1300 carries a task ID, so as to notify the computing service unit 1300 to suspend the processing of the task through the task ID.

[0128] Specifically, the pre-processing unit 1200 can ensure the consistency of task data migration through hash value verification. First, the computing service unit 1300 uses a hash algorithm (e.g., SHA-256) to generate a fixed-length hash value for the task data; then, the task data and the hash value are transmitted back to the pre-processing unit 1200. After receiving the data, the pre-processing unit 1200 recalculates the hash value using the same algorithm and compares it with the received hash value: if they match, it proves that the task data has not been tampered with or damaged during transmission or storage; if they do not match, it indicates that the task data is abnormal.

[0129] Specifically, the third threshold can be flexibly set according to the performance of the preprocessing unit 1200. For example, if the task load is represented by CPU utilization, the third threshold can be set to 20%.

[0130] See Figure 2 In some embodiments, the computing service unit 1300 includes a cross-server load sensing unit 1310, a cross-server task allocation unit 1320, and a plurality of computing servers 1330; the cross-server load sensing unit 1310 is used to dynamically sense the second task load of each computing server 1330, and the cross-server task allocation unit 1320 is used to allocate the tasks to be processed assigned to the computing service unit 1300 to each computing server 1330 according to the evaluation result of the second task load; wherein, the tasks to be processed are image preprocessing tasks or image analysis tasks.

[0131] In this embodiment, the cross-server load sensing unit 1310 establishes a communication connection with multiple computing servers 1330 in the computing service unit 1300. It periodically acquires the load data of each computing server 1330 through polling, timed requests, or other methods, and transmits the data to the cross-server task allocation unit 1320. The cross-server task allocation unit 1320 then calls the network communication interface to perform task allocation based on the load assessment results of the computing servers 1330. For example, if CPU utilization represents task load, the cross-server load sensing unit 1310 periodically acquires the CPU utilization of each computing server 1330.

[0132] The cross-server load awareness unit 1310 and the cross-server task allocation unit 1320 achieve load awareness and task allocation across computing servers 1330, realizing load balancing among computing servers 1330 and solving the problem of inefficient scheduling of computing tasks among computing servers 1330. This ensures that all computing servers 1330 are in a high-efficiency operating state, achieving efficient utilization of computing resources. Furthermore, multiple computing servers 1330 can execute computing tasks in parallel, improving computing efficiency, and the number of computing servers 1330 can be adjusted according to computing needs, demonstrating strong scalability.

[0133] Optionally, the preprocessing unit 1200 is connected to multiple computing servers 1330 via a data switch 1400. The preprocessing unit 1200 will distribute computing tasks to all connected computing servers 1330 according to certain rules.

[0134] Specifically, when the computing service unit 1300 initiates the task distribution process, the cross-server load perception unit 1310 first sends a status query request to each computing server 1330 through a preset communication protocol (such as industrial Ethernet, MQTT, etc.) to obtain core indicators such as the real-time load rate, current task queue length, hardware resource usage (such as CPU, memory, I / O, etc.) and online status of each computing server 1330; then, it follows the preset scheduling strategy (such as load balancing, task priority matching or resource adaptation rules). Once it is determined that each computing server 1330 meets the processing threshold for the task (e.g., load rate < 60%, possessing the computing power / interface support required for the task), the cross-server task allocation unit 1320 then performs structured encapsulation of the task, including key information such as task ID, data payload, processing parameters (e.g., accuracy requirements, timeout threshold), and result feedback format, and transmits it to the computing server 1330 through an encrypted channel. After receiving the data, the computing server 1330 performs verification (data integrity, permission verification). If the verification passes, it adds the task to the local execution queue and returns a "receive confirmation" to the cross-server task allocation unit 1320, and then starts task processing according to preset logic.

[0135] Specifically, the cross-server load awareness unit 1310 and the cross-server task allocation unit 1320 can be deployed as an independent load balancing server, or run as a distributed software on a master node of the computing service unit 1300.

[0136] In some embodiments, the cross-server load sensing unit 1310 is used to obtain the first queue length of the tasks to be processed in the task queue of each computing server 1330, and the first task processing speed of each computing server 1330 in the previous time window, and to obtain the first estimated processing time of each computing server 1330 for processing the tasks to be processed based on the first queue length and the first task processing speed of each computing server 1330; wherein, the first estimated processing time is used to characterize the second task load.

[0137] In this embodiment, the estimated processing time of the task of the computing server 1330 is calculated by the server load perception unit to characterize the load of the computing server 1330. This can not only intuitively reflect the current amount of tasks to be processed by the computing server 1330, but also combine the historical processing efficiency of the computing server 1330 to dynamically and accurately reflect the actual task load status of the computing server 1330, which helps to allocate and schedule resources of the computing server 1330 more reasonably.

[0138] See Figure 2 In some embodiments, each computing server 1330 includes an in-server load sensing unit 1331, an in-server task allocation unit 1332, and a plurality of sub-computing units 1340; the in-server load sensing unit 1331 is used to dynamically sense the third task load of each sub-computing unit 1340, and the in-server task allocation unit 1332 is used to allocate the tasks to be processed assigned to the computing server 1330 to each sub-computing unit 1340 according to the evaluation result of the third task load.

[0139] In this embodiment, the load sensing unit 1331 within the server establishes a communication connection with multiple sub-computing units 1340 in the computing server 1330. It periodically acquires the load data of each sub-computing unit 1340 through polling, timed requests, or other methods, and transmits the data to the task allocation unit 1332 within the server. The task allocation unit 1332 then calls the network communication interface to perform task allocation based on the load assessment results of the sub-computing units 1340. For example, if CPU utilization represents task load, the load sensing unit 1331 periodically acquires the CPU utilization of each sub-computing unit 1340.

[0140] The load perception unit 1331 and the task allocation unit 1332 within the server realize the load perception and task allocation of the sub-computing units 1340 within the computing server 1330. Based on the load assessment results, the computing tasks allocated to the computing server 1330 are redistributed, thereby achieving load balancing among the sub-computing units 1340. This solves the problem of inefficient scheduling of computing tasks among the sub-computing units 1340, ensuring that all sub-computing units 1340 are in a high-efficiency operating state, thus achieving efficient utilization of computing resources.

[0141] Specifically, the server load awareness unit 1331 and the server task allocation unit 1332 can be deployed as an independent load balancer, or run as a distributed software in the computing server 1330.

[0142] Optionally, each computing server 1330 is equipped with multiple XPU processing cards as sub-computing units 1340, thereby forming a heterogeneous computing system by integrating different types of computing units, and providing more powerful computing support based on the advantages of heterogeneous computing systems.

[0143] When the optical inspection system 1000 adopts a heterogeneous computing system architecture, the heterogeneous computing hardware system is based on a distributed architecture design. Its computing server 1330 and sub-computing unit 1340 can be flexibly expanded according to the inspection requirements, which has better flexibility and scalability. This makes it more suitable for high-throughput and low-latency application scenarios, and has the advantages of scalability, high efficiency and centralized management. It solves the problem of insufficient computing efficiency of current defect detection solutions in the semiconductor field and realizes efficient processing of optical images.

[0144] Specifically, XPUs include, but are not limited to, GPUs (Graphics Processing Units), NPUs (Neural Processing Units), DPUs (Deep Learning Processing Units), TPUs (Tensor Processing Units), and APUs (Accelerated Processing Units).

[0145] In some embodiments, the server load sensing unit 1331 is used to obtain the second queue length of the tasks to be processed in the sub-task queue of each sub-computing unit 1340, and the second task processing speed of each sub-computing unit 1340 in the previous time window, and to obtain the second estimated processing time of each sub-computing unit 1340 for processing the tasks to be processed based on the second queue length and the second task processing speed of each sub-computing unit 1340; wherein the second estimated processing time is used to characterize the third task load.

[0146] In this embodiment, the estimated processing time of the sub-computing unit 1340 is calculated by the load perception unit 1331 in the server to characterize the load of the sub-computing unit 1340. This not only reflects the current amount of tasks to be processed by the sub-computing unit 1340, but also combines the historical processing efficiency of the sub-computing unit 1340 to dynamically and accurately reflect the actual task load status of the sub-computing unit 1340, which helps to allocate and schedule resources for the sub-computing unit 1340 more reasonably.

[0147] See Figure 2 In some embodiments, the preprocessing unit 1200 includes a storage unit 1210 and an overload protection unit 1220; the overload protection unit 1220 is used to temporarily store the tasks to be processed in the storage unit 1210 when there is no computing server 1330 with a second task load less than a fourth threshold, and to release the tasks to be processed in the storage unit 1210 to the preprocessing unit 1200 for distribution when there is a computing server 1330 with a second task load less than a fourth threshold.

[0148] In this embodiment, the overload protection unit 1220 establishes a communication connection with the server load sensing unit to periodically obtain the second task load of the computing server 1330 from the server load sensing unit through polling, timed requests, or other methods. When there is no computing server 1330 with a second task load less than the fourth threshold, the overload protection unit 1220 first sends a data write request to the storage unit 1210 according to a preset communication protocol and interface specification. This request includes the target storage location information and the data content to be written. After receiving the request, the storage unit 1210 parses and verifies the request. After confirming that the storage space is available and the permissions are legal, it receives the task data through the data transmission channel and accurately writes the task data to the target location according to the specified storage format. After completion, it returns a write success confirmation signal to the overload protection unit 1220. When the computing server 1330 with a second task load less than the fourth threshold is in operation, the overload protection unit 1220 sends a read instruction containing the target data address to the storage unit 1210. The control module of the storage unit 1210 locates the corresponding storage area, extracts the data, and sends it to the overload protection unit 1220 in the agreed format through the transmission channel. After receiving the data, the overload protection unit 1220 performs parsing processing to obtain task data for distribution by the preprocessing unit 1200.

[0149] The preprocessing unit 1200 is equipped with a storage unit 1210, which can temporarily store computing tasks. When the task load decreases, the preprocessing unit 1200 reloads the computing tasks from the storage unit 1210 and distributes them. By using the storage unit 1210 and the overload protection unit 1220 to temporarily store tasks to be processed in the storage unit 1210 when the computing server 1330 is under high load, and to release and distribute tasks to be processed from the storage unit 1210 when the computing server 1330 is under low load, overload protection and task buffering of the computing server unit 1300 are achieved, thereby effectively avoiding task blocking and data loss.

[0150] In addition, the storage unit 1210 can also be used to temporarily store the acquired image data, which will then be preprocessed in the preprocessing unit 1200.

[0151] Specifically, when the preprocessing unit 1200 initiates the task distribution process, it first sends a status query request to the computing service unit 1300 through a preset communication protocol (such as Industrial Ethernet, MQTT, etc.) to obtain core indicators such as the real-time load rate, current task queue length, hardware resource usage (such as CPU, memory, I / O, etc.), and online status of the computing service unit 1300. Then, based on a preset scheduling strategy (such as load balancing, task priority matching, or resource adaptation rules), if the computing server 1330 unit is determined to meet the processing threshold of the task (e.g., load rate < 60%, possessing the required computing power / interface support), the preprocessing unit 1200 then performs structured encapsulation of the task, including key information such as task ID, data payload, processing parameters (such as accuracy requirements, timeout threshold), and result feedback format, and transmits it to the computing service unit 1300 through an encrypted channel. Upon receiving the data, the computing service unit 1300 performs verification (data integrity, permission verification). If the verification passes, it adds the task to its local execution queue and returns a "receive confirmation" to the preprocessing unit 1200, then initiates task processing according to preset logic.

[0152] Specifically, the overload protection unit 1220 and the storage unit 1210 are part of the preprocessing unit 1200. The overload protection unit 1220 can be a software process responsible for managing the read and write operations of the storage unit 1210 (such as a database or message queue). The storage unit 1210 can be a circular memory buffer on the preprocessing unit 1200 or a file cache on a local solid-state drive.

[0153] Specifically, the fourth threshold can be flexibly set according to the performance of the computing server 1330. For example, if the task load is represented by CPU utilization, the fourth threshold can be set to 80%.

[0154] See Figure 2 In some embodiments, the optical detection system 1000 further includes a host computer 1700, a management service unit 1600, and a management switch 1500; the management service unit 1600 is used to manage the computing service unit 1300, summarize the calculation results of the computing service unit 1300, and upload the calculation results to the host computer 1700 through the management switch 1500.

[0155] In this embodiment, the management service unit 1600 is connected to the computing service unit 1300, and the host computer 1700 is connected to the image acquisition unit 1100, the preprocessing unit 1200 and the management service unit 1600 through the management switch 1500.

[0156] The management service unit 1600 enables centralized management of the computing servers 1330 within the computing service unit 1300, thereby improving the operational efficiency of the computing service unit 1300. Simultaneously, the management service unit 1600 also provides operational services such as algorithm version updates and deployments, and log viewing for all computing servers 1330 within the computing service unit 1300, significantly enhancing work efficiency. Furthermore, the management service unit 1600 aggregates and performs final post-processing of the calculation results from all computing servers 1330, and then uploads the results to the host computer 1700. Specifically, the management service unit 1600 is a separately configured management server.

[0157] The host computer 1700 is connected to the image acquisition unit 1100, preprocessing unit 1200, computing service unit 1300, and management service unit 1600 via the management switch 1500. This allows for unified centralized monitoring, error handling, and user interaction of the image acquisition unit 1100, preprocessing unit 1200, computing service unit 1300, and management service unit 1600, thus achieving unified management of all units in the optical inspection system 1000. The host computer 1700 enables centralized monitoring, management, and error handling of the entire system, improving user experience and work efficiency.

[0158] The management switch 1500 enables precise data forwarding and efficient control, ensuring stable and secure data transmission. It boasts high-speed data forwarding capabilities, rapidly and accurately transmitting commands and data from the image acquisition unit 1100, preprocessing unit 1200, and management service unit 1600 to the host computer 1700. Simultaneously, it promptly relays feedback from the host computer 1700 back to the image acquisition unit 1100, preprocessing unit 1200, and management service unit 1600, ensuring smooth and uninterrupted communication between units. Furthermore, the management switch 1500 supports multiple network management protocols, enabling monitoring and analysis of network traffic. It prioritizes and schedules traffic based on different service requirements, prioritizing the transmission of critical data and improving overall network performance and stability.

[0159] In some embodiments, this application also provides a semiconductor device, which includes the optical detection system 1000 described above.

[0160] In this embodiment, the semiconductor device achieves global load balancing across computing nodes and load balancing and maximum resource utilization within computing nodes based on multi-level dynamic load awareness and collaborative task allocation. This allows its computing server 1330 and all sub-computing units 1340 to operate at full load. Thus, by utilizing dynamic load awareness, two-level (across computing server 1330 and within computing server 1330) intelligent load balancing, overload buffering, and reuse of computing resources, a closed-loop, highly efficient resource management and task scheduling system is formed to dynamically allocate computing tasks to the system, achieving efficient utilization of system computing resources.

[0161] Specifically, through the multi-level dynamic load perception and collaborative task scheduling unit, all computing resources of the computing server 1330 and the sub-computing unit 1340 can be efficiently utilized, thereby improving the overall computing efficiency by more than 20%.

[0162] Figure 3 A flowchart illustrating a computing resource scheduling method provided in this application embodiment is shown below. Figure 3 In some embodiments, this computing resource scheduling method is applied to a management service unit, and its process includes the following steps:

[0163] S301, the first task load of the monitoring preprocessing unit.

[0164] Specifically, by monitoring the first task load of the preprocessing unit, it is determined whether to migrate some image preprocessing tasks to the computing service unit or to stop migrating image preprocessing tasks to the computing service unit based on the load of the preprocessing unit.

[0165] Specifically, the first task load of the preprocessing unit can be evaluated based on factors such as the number and size of tasks to be processed on the unit, processor utilization, task processing time, and memory usage. Specifically, for the preprocessing unit, when the data transfer rate from the acquisition card in the image acquisition unit exceeds the task processing rate of the preprocessing unit, the computational load of the preprocessing unit will increase; conversely, the computational load of the preprocessing unit will decrease.

[0166] S302, when the first task load is greater than the first threshold, some image preprocessing tasks of the preprocessing unit are migrated to the computing service unit.

[0167] When the computational load of the current processing unit exceeds a preset first threshold, it indicates that the computational resources of the preprocessing unit are strained, and the subsequent image preprocessing tasks are migrated to the computing service unit for completion.

[0168] S303, when the first task load is less than the second threshold, stop migrating the image preprocessing task to the computing service unit; wherein, the first threshold is greater than the second threshold.

[0169] Specifically, when the computing load of the computing service unit reaches a preset second threshold, indicating that the computing resources of the surface computing service unit are strained, the migration of image preprocessing tasks to the computing service unit will be stopped.

[0170] S304, when the first task load is less than the third threshold, at least a portion of the image preprocessing tasks that have been migrated to the computing service unit are migrated back to the preprocessing unit; wherein the third threshold is less than the second threshold.

[0171] When the computational load of the current processing unit is less than the preset third threshold, it indicates that the computational resources of the preprocessing unit are idle. At least a portion of the image preprocessing tasks that have been migrated to the computing service unit will be migrated back to the preprocessing computing unit for processing, thereby achieving efficient utilization of the computational resources of the preprocessing unit.

[0172] Optionally, each time the load of the first task is detected to be lower than the third threshold, some image preprocessing tasks are successively migrated back until all migrated image preprocessing tasks are recovered.

[0173] Specifically, the first threshold, the second threshold, and the third threshold can be flexibly set according to the performance of the preprocessing unit. For example, if the task load is represented by CPU utilization, the first threshold can be set to 80%, the second threshold can be set to 50%, and the third threshold can be set to 20%.

[0174] In this embodiment, when the load of the first task exceeds a first threshold, some image preprocessing tasks of the preprocessing unit are migrated to the computing service unit. This proactively offloads computational pressure to the computing service unit when the preprocessing unit faces overload risk, thus improving the overall processing efficiency of the system during peak periods and under high loads, and preventing task backlog, system lag, or even data loss due to insufficient preprocessing capacity. Therefore, through the coordinated scheduling of computing resources between the preprocessing unit and the computing service unit, efficient utilization of system computing resources is achieved, fully leveraging system performance.

[0175] Meanwhile, when the load of the first task is less than the second threshold, the migration of image preprocessing tasks to the computing service unit is stopped, and the first threshold is set to be greater than the second threshold. This ensures that the load point at which migration stops is lower than the load point at which migration starts. This avoids frequent and repeated switching of the task migration strategy when the system load fluctuates slightly around a single threshold, reducing the additional overhead caused by frequent changes in the task migration strategy and enhancing the stability of system operation.

[0176] Furthermore, when the load of the first task is less than the third threshold, at least a portion of the image preprocessing tasks that have been migrated to the computing service unit are migrated back to the preprocessing unit. This efficiently allocates idle computing resources in the preprocessing unit, achieving efficient utilization of system computing resources. Setting the third threshold to be less than the second threshold ensures that the preprocessing unit is sufficiently idle and capable of taking over tasks, preventing task migration from being triggered again immediately after migration due to small load fluctuations.

[0177] exist Figure 3 The computational resource scheduling method shown also requires dynamic task scheduling between computational servers. The following section will discuss this in conjunction with... Figure 4 This paper further introduces the dynamic task scheduling among computing servers in the technical solution of the above computing resource scheduling method.

[0178] Figure 4 A flowchart illustrating a dynamic task scheduling method among computing servers provided in this application embodiment is shown below. Figure 4 In some embodiments, the dynamic task scheduling method among computing servers includes the following steps:

[0179] S401, when the current processing unit distributes the tasks to be processed to the computing service unit, it obtains the second task load of each computing server; wherein, the tasks to be processed are image preprocessing tasks or image analysis tasks.

[0180] When the current processing unit distributes tasks to be processed to the computing service unit, it dynamically senses the task load of all computing servers, and distributes the tasks to be processed to the corresponding computing servers based on the task load of each computing server.

[0181] Specifically, there are generally two scenarios where the preprocessing unit distributes tasks to the computing service unit: one is when the preprocessing unit needs to send the data to the computing service unit for analysis after completing image preprocessing; the other is when, based on the task migration decision mentioned above, the image preprocessing task itself needs to be distributed to the computing service unit. Therefore, tasks to be processed include image preprocessing tasks and image analysis tasks.

[0182] Specifically, when the current processing unit needs to distribute tasks to the computing service unit, the preprocessing unit can send a status query request to the computing service unit to obtain the second task load of each computing server in the computing service unit, and then the preprocessing unit distributes tasks based on the second task load of each computing server.

[0183] Optionally, obtain the second task load of each computing server, including:

[0184] Step 1: Obtain the first queue length of the tasks to be processed in the task queue of each computing server, and the first task processing speed of each computing server in the previous time window.

[0185] When processing tasks, the computing server establishes a task queue and processes tasks sequentially according to the queue. Therefore, to estimate the task processing time of the computing server, it is first necessary to obtain the queue length of tasks to be processed in the task queue of each computing server. And the task processing speed of each computing server in the previous time window. .

[0186] Specifically, the processing speed of the first task satisfies the following formula:

[0187]

[0188] in, This indicates the duration of the previous time window. This indicates the number of tasks processed by the computing server in the previous time window.

[0189] Specifically, the time window can be flexibly set according to the performance of the computing server, such as 1s, 3s, 5s, etc.

[0190] Step 2: Based on the first queue length and the first task processing speed of each computing server, obtain the first estimated processing time for each computing server to process the task to be processed; wherein, the first estimated processing time is used to characterize the second task load.

[0191] Specifically, the estimated processing time for each task to be processed on each computing server is calculated, and the task load of the computing server is characterized based on the estimated processing time. In particular, the longer the estimated processing time, the higher the task load of the computing server.

[0192] Specifically, the first estimated processing time satisfies the following formula:

[0193]

[0194] in, This indicates the length of the first queue of tasks to be processed in the task queue of the computing server.

[0195] Specifically, characterizing compute server load by estimating task processing time provides a direct reflection of the current number of tasks pending and, combined with historical processing efficiency, dynamically and accurately reflects the actual workload of the compute servers. This facilitates more rational resource allocation and scheduling. Furthermore, even with identical GPU utilization and memory usage, different compute servers exhibit varying processing speeds and queue backlogs, making it impossible to assess their load on a single dimension. Therefore, estimating processing time places the load data of each compute server on a unified level, providing a more accurate reflection of their workload.

[0196] S402, if a computing server has a second task load less than the fourth threshold, then the task to be processed is assigned to the corresponding computing server.

[0197] In cases where there is a computing server with a second task load less than the third threshold, the task to be processed is assigned to the corresponding computing server for processing.

[0198] Specifically, the corresponding computing server can be the one with the lowest task load.

[0199] Specifically, the fourth threshold can be flexibly set according to the performance of the computing server. For example, if the task load is represented by CPU utilization, the fourth threshold can be set to 80%.

[0200] S403, if there is no computing server with a second task load less than the fourth threshold, then the task to be processed is stored in the storage unit.

[0201] When the task load of all computing servers exceeds the preset threshold, it indicates that the computing resources of all computing servers are relatively strained. At this time, the tasks to be processed are stored in the storage unit to postpone processing, so as to realize overload protection and task buffering, thereby effectively avoiding task blocking and data loss.

[0202] Specifically, when all computing servers are busy, the tasks to be processed are stored in this storage unit, and task description information and image data are recorded.

[0203] S404 monitors the secondary task load of each computing server.

[0204] Among these methods, the task load of each computing server is monitored to identify the opportune time to reload and process tasks cached in storage units.

[0205] S405, when a computing server with a second task load less than the fourth threshold is used, the tasks to be processed in the storage unit are loaded into the pre-processing unit for distribution.

[0206] Specifically, when the workload of the computing server decreases, the pending tasks temporarily stored in the storage unit will be reloaded and redistributed. By loading and distributing pending tasks from the storage unit when the computing server is under low load, efficient utilization of the computing server's computing resources is achieved.

[0207] Specifically, tasks that can be loaded from the storage unit to the pre-processing unit are distributed to the computing server with the lowest task load.

[0208] In this embodiment, load balancing between computing servers is achieved through load awareness and task allocation across computing servers, solving the problem of inefficient scheduling of computing tasks between computing servers, and ensuring that all computing servers are in a high-efficiency operating state to achieve efficient utilization of computing resources.

[0209] exist Figure 3 The computational resource scheduling method shown requires dynamic task scheduling within the computational server. The following section will combine... Figure 5 This paper further introduces the dynamic task scheduling within the computing server in the technical solution of the above computing resource scheduling method.

[0210] Figure 5 A flowchart illustrating a dynamic task scheduling method within a computing server, as provided in this application embodiment, is shown below. Figure 5 In some embodiments, the dynamic task scheduling method within the computing server includes the following steps:

[0211] S501: When the computing server receives a task to be processed, it obtains the third task load of each sub-computing unit.

[0212] When the computing server receives a task to be processed, it also needs to dynamically sense the task load of all sub-computing units in the computing server, so as to distribute the task to be processed to the corresponding sub-computing unit based on the task load of each sub-computing unit.

[0213] Specifically, each computing server is equipped with multiple XPU processing cards as sub-computing units, thus forming a heterogeneous computing system by integrating different types of computing units. This leverages the advantages of heterogeneous computing systems to provide more powerful computing support. Among them, XPUs include, but are not limited to, GPUs (Graphics Processing Units), NPUs (Neural Processing Units), DPUs (Deep Learning Processing Units), TPUs (Tensor Processing Units), and APUs (Acceleration Processing Units).

[0214] Optionally, obtain the task load of each sub-computing unit, including:

[0215] Step 1: Obtain the second queue length of the tasks to be processed in the subtask queue of each sub-computing unit, and the second task processing speed of each sub-computing unit in the previous time window.

[0216] When processing tasks, each sub-computing unit establishes a sub-task queue and processes the tasks sequentially according to the queue. Therefore, to estimate the task processing time of a sub-computing unit, it is first necessary to obtain the queue length of the tasks to be processed in the sub-task queue of each sub-computing unit. And the task processing speed of each sub-computing unit in the previous time window. .

[0217] Specifically, the processing speed of the second task satisfies the following formula:

[0218]

[0219] in, This indicates the duration of the previous time window. This indicates the number of tasks processed by the sub-computing unit in the previous time window.

[0220] Specifically, the time window can be flexibly set according to the performance of the sub-computing unit, such as 1s, 3s, 5s, etc.

[0221] Step 2: Based on the second queue length and second task processing speed of each sub-computing unit, obtain the second estimated processing time for each sub-computing unit to process the task to be processed; wherein, the second estimated processing time is used to characterize the load of the third task.

[0222] Specifically, the estimated processing time for each task in each sub-computing unit is calculated, and the task load of that sub-computing unit is characterized based on the estimated processing time. In particular, a longer estimated processing time indicates a higher task load for that sub-computing unit.

[0223] Specifically, the second estimated processing time satisfies the following formula:

[0224]

[0225] in, This indicates the length of the first queue of tasks to be processed in the subtask queue of the subcomputation unit.

[0226] Specifically, by calculating the estimated processing time of sub-computing units, the load of each sub-computing unit can be characterized. This not only intuitively reflects the current amount of tasks to be processed by the sub-computing unit, but also, combined with the historical processing efficiency of the sub-computing unit, dynamically and accurately reflects the actual task load status of the sub-computing unit. This helps to allocate and schedule resources for sub-computing units more rationally. Furthermore, even if GPU utilization and memory usage are the same, the processing speed and queue backlog of different sub-computing units can vary, making it impossible to evaluate the load status of sub-computing units on a single dimension. Therefore, estimating processing time places the load data of each sub-computing unit on the same dimension, more accurately reflecting the load status of each sub-computing unit.

[0227] S502, if a sub-computing unit has a third task load less than the fifth threshold, then the task to be processed is assigned to the corresponding sub-computing unit.

[0228] In cases where there is a sub-computing unit with a third task load less than the fifth threshold, the task to be processed is assigned to the corresponding sub-computing unit for processing.

[0229] Specifically, the corresponding sub-computing unit can be the sub-computing unit with the smallest task load.

[0230] Specifically, the fifth threshold can be flexibly set according to the performance of the computing server. For example, if the task load is represented by CPU utilization, the fifth threshold can be set to 60%.

[0231] In this embodiment, by load perception and task allocation of sub-computing units within the computing server, and based on the load assessment results, the computing tasks allocated to the computing server are redistributed, thereby achieving load balancing among sub-computing units. This solves the problem of inefficient scheduling of computing tasks among sub-computing units, ensuring that all sub-computing units are in a high-efficiency operating state, thus achieving efficient utilization of computing resources.

[0232] Specifically, for load awareness and task scheduling within the computing server, in order to maximize the utilization of computing resources, a portion of the computing server's threads are used to schedule sub-computing units, while another portion of the threads are used to execute computing tasks in parallel.

[0233] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0234] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the appended claims.

[0235] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for scheduling computing resources, characterized in that, The method is applied to an optical inspection system, which includes an image acquisition unit, a preprocessing unit, and a computing service unit. The image acquisition unit is used to acquire image data, the preprocessing unit is used to perform image preprocessing operations on the image data, and the computing service unit is used to perform image analysis operations on the image data after the image preprocessing operations. The method includes: Monitor the first task load of the preprocessing unit; When the first task load exceeds the first threshold, part of the image preprocessing task of the preprocessing unit is migrated to the computing service unit; When the first task load is less than the second threshold, the migration of the image preprocessing task to the computing service unit is stopped; wherein the first threshold is greater than the second threshold.

2. The method according to claim 1, characterized in that, The method further includes: When the first task load is less than the third threshold, at least a portion of the image preprocessing tasks that have been migrated to the computing service unit are migrated back to the preprocessing unit; wherein the third threshold is less than the second threshold.

3. The method according to claim 1 or 2, characterized in that, The computing service unit includes multiple computing servers, and the method further includes: When the preprocessing unit distributes the task to be processed to the computing service unit, it obtains the second task load of each computing server; wherein the task to be processed is an image preprocessing task or an image analysis task. If a computing server has a second task load less than the fourth threshold, then the task to be processed is assigned to the corresponding computing server.

4. The method according to claim 3, characterized in that, The preprocessing unit further includes a storage unit, and the method further includes: If there is no computing server with a second task load less than the fourth threshold, then the task to be processed is stored in the storage unit.

5. The method according to claim 4, characterized in that, The method further includes: Monitor the second task load of each of the aforementioned computing servers; When a computing server has a second task load less than the fourth threshold, the tasks to be processed in the storage unit are loaded into the preprocessing unit for distribution.

6. The method according to claim 3, characterized in that, Obtaining the second task load of each of the computing servers includes: Obtain the first queue length of the tasks to be processed in the task queue of each computing server, and the first task processing speed of each computing server in the previous time window; Based on the first queue length and first task processing speed of each computing server, a first estimated processing time for each computing server to process the task to be processed is obtained; wherein, the first estimated processing time is used to characterize the second task load.

7. The method according to claim 3, characterized in that, The computing server includes multiple sub-computing units, and the method further includes: When the computing server receives the task to be processed, it obtains the third task load of each of the sub-computing units; If a sub-computing unit has a third task load that is less than the fifth threshold, then the task to be processed is assigned to the corresponding sub-computing unit.

8. The method according to claim 7, characterized in that, Obtaining the task load of each of the sub-computing units includes: Obtain the second queue length of the tasks to be processed in the subtask queue of each of the sub-computing units, and the second task processing speed of each of the sub-computing units in the previous time window; Based on the second queue length and second task processing speed of each of the sub-computing units, a second estimated processing time for each of the sub-computing units to process the task to be processed is obtained; wherein, the second estimated processing time is used to characterize the third task load.

9. An optical detection system, characterized in that, It includes an image acquisition unit, a preprocessing unit, and a computing service unit; The image acquisition unit is used to acquire image data; The preprocessing unit is used to perform image preprocessing operations on the image data; The computing service unit is used to perform image analysis operations on the image data after the image preprocessing operation; The preprocessing unit is used for: Monitor the first task load of the preprocessing unit; When the first task load exceeds the first threshold, part of the image preprocessing task of the preprocessing unit is migrated to the computing service unit; When the first task load is less than the second threshold, the migration of the image preprocessing task to the computing service unit is stopped; wherein the first threshold is greater than the second threshold.

10. The optical detection system according to claim 9, characterized in that, The preprocessing unit includes a preprocessing load sensing unit and a preprocessing load migration unit. The preprocessing load sensing unit is used to monitor the first task load of the preprocessing unit; The preprocessing load migration unit is used to migrate part of the image preprocessing tasks of the preprocessing unit to the computing service unit when the first task load is greater than a first threshold, and to stop migrating the image preprocessing tasks to the computing service unit when the first task load is less than a second threshold.

11. The optical inspection system according to any one of claims 9 or 10, characterized in that, The computing service unit includes a cross-server load awareness unit, a cross-server task allocation unit, and multiple computing servers. The cross-server load sensing unit is used to dynamically sense the second task load of each of the computing servers, and the cross-server task allocation unit is used to allocate the pending tasks assigned to the computing service unit to each of the computing servers according to the evaluation result of the second task load; wherein, the pending tasks are image preprocessing tasks or image analysis tasks.

12. The optical detection system according to claim 11, characterized in that, Each of the aforementioned computing servers includes a server-side load awareness unit, a server-side task allocation unit, and multiple sub-computing units; The server load sensing unit is used to dynamically sense the third task load of each of the sub-computing units, and the server task allocation unit is used to allocate the tasks to be processed on the computing server to each of the sub-computing units according to the evaluation result of the third task load.

13. The optical inspection system according to claim 11, characterized in that, The preprocessing unit includes a storage unit and an overload protection unit; The overload protection unit is used to temporarily store the task to be processed in the storage unit when there is no computing server with a second task load less than the fourth threshold, and to release the task to be processed in the storage unit to the preprocessing unit for distribution when there is a computing server with a second task load less than the fourth threshold.

14. The optical inspection system according to any one of claims 9 or 10, characterized in that, The optical inspection system also includes a host computer, a management service unit, and a management switch; The management service unit is used to manage the computing service unit, summarize the calculation results of the computing service unit, and upload the calculation results to the host computer through the management switch.

15. A semiconductor device, characterized in that, The semiconductor device includes the optical inspection system as described in any one of claims 9-14.

Images