Safety warning method, device and equipment for semiconductor factory operation scene

Abstract

The application relates to a safety warning method, device and equipment for a semiconductor factory work scene. The method comprises the following steps: acquiring a work scene image of a semiconductor factory; inputting the work scene image into a target detection model and receiving a detection result of the target detection model; the detection result comprises a first detection result corresponding to a first target detection head and a second detection result corresponding to a second target detection head; determining an isolation index according to the first detection result and the second detection result; and determining safety warning information of the work scene according to the isolation index. The method can improve the warning accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a safety early warning method, apparatus and computer equipment for semiconductor factory operation scenarios. Background Technology

[0002] With the continuous development of intelligent industrial safety monitoring technology, computer vision-based risk perception methods for work sites are gradually being applied to various production environments.

[0003] In recent years, with the advancement of deep learning technology, some systems have begun to adopt general target detection models to automatically identify images captured by specific cameras in an attempt to achieve safety monitoring of construction areas.

[0004] Due to the special lighting environment inside semiconductor factories, the holes on the factory floor are tiny, with very low contrast to the floor's color and texture, and lack stable edge and corner features. As a result, the common detection model has a high error rate in detecting holes, leading to low accuracy in safety warnings for operations. Summary of the Invention

[0005] Therefore, it is necessary to provide a safety early warning method, device, and computer equipment for semiconductor factory operation scenarios that can improve the accuracy of early warning, addressing the aforementioned technical problems.

[0006] Firstly, this application provides a safety early warning method for semiconductor factory operation scenarios, the method comprising:

[0007] Acquire images of the operational scene in a semiconductor factory;

[0008] The work scene image is input into the target detection model, and the detection results of the target detection model are received. The target detection model includes an enhanced backbone network and a detection head. The enhanced backbone network is used to extract features from the work scene image to obtain a multi-scale feature map. The detection head is used to obtain a detection result corresponding to the multi-scale feature map. The detection head includes a first-type target detection head and a second-type target detection head, and the first-type target detection head and the second-type target detection head establish spatial association constraints during training. The detection results include a first detection result corresponding to the first-type target detection head and a second detection result corresponding to the second-type target detection head.

[0009] The isolation index is determined based on the first and second test results;

[0010] Based on the isolation index, safety warning information for the work scenario is determined.

[0011] In one embodiment, the detection head includes a third type of target detection head; the detection result also includes a third detection result corresponding to the third type of target detection head; the third detection result includes the location of the workers within the semiconductor factory; determining the safety warning information of the work scenario based on the isolation index includes:

[0012] Based on the isolation index and the location of workers within the semiconductor factory, safety warning information for the work scenario is determined.

[0013] In one embodiment, the first detection result includes the location of warning signs within the semiconductor factory; the second detection result includes the location of holes in the semiconductor factory floor; determining the isolation index based on the first and second detection results includes:

[0014] The location of warning signs within the semiconductor factory is processed using a boundary fitting algorithm to obtain the dynamic boundaries of the warning signs.

[0015] The isolation index is determined based on the dynamic boundary and the location of the holes in the semiconductor factory grounds.

[0016] In one embodiment, determining the isolation index based on the dynamic boundary and the location of holes in the semiconductor factory site includes:

[0017] Determine the distance from the location of the perforation in the semiconductor factory ground surface to the dynamic boundary;

[0018] Determine the compact term of the dynamic boundary; wherein the compact term is determined based on the ratio of the area to the perimeter of the dynamic boundary;

[0019] Determine the coverage metric between the location of the perforations in the semiconductor factory ground surface and the dynamic boundary;

[0020] The isolation index is determined based on the distance, the compactness term, and the coverage metric.

[0021] In one embodiment, the isolation index is calculated using the following formula:

[0022]

[0023] In the formula, IS represents the isolation index; min_dist represents the distance from the location of the hole in the semiconductor factory ground to the dynamic boundary; The compact term represents the dynamic boundary; cover_ratio represents the coverage measure of the location of the holes in the semiconductor factory grounds relative to the dynamic boundary. , , and This is a hyperparameter.

[0024] In one embodiment, the multi-scale feature map includes a first-scale feature map and a second-scale feature map; the target detection model includes a first feature extraction branch and a second feature extraction branch; the first feature extraction branch is used to extract global features of the work scene image to obtain a first-scale feature map; the second feature extraction branch is used to extract local features of the work scene image to obtain a second-scale feature map.

[0025] In one embodiment, the first type of target detection head is connected to the first feature extraction branch; the second type of target detection head is connected to the second feature extraction branch.

[0026] In one embodiment, acquiring the operational scene image of the semiconductor factory includes: acquiring an initial operational scene image of the semiconductor factory; and dividing the initial operational scene image according to a preset area of ​​interest to obtain an operational scene image of the semiconductor factory.

[0027] Secondly, this application provides a safety early warning device for semiconductor factory operation scenarios, the device comprising:

[0028] The acquisition module is used to acquire images of the operational scene in a semiconductor factory.

[0029] A processing module is used to input the work scene image into a target detection model and receive the detection results from the target detection model. The target detection model includes an enhanced backbone network and a detection head. The enhanced backbone network is used to extract features from the work scene image to obtain a multi-scale feature map. The detection head is used to obtain a detection result corresponding to the detection head based on the multi-scale feature map. The detection head includes a first-type target detection head and a second-type target detection head, and the first-type target detection head and the second-type target detection head establish spatial association constraints during training. The detection results include a first detection result corresponding to the first-type target detection head and a second detection result corresponding to the second-type target detection head.

[0030] The first determining module is used to determine the isolation index based on the first detection result and the second detection result;

[0031] The second determining module is used to determine the safety warning information of the work scenario based on the isolation index.

[0032] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0033] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0034] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0035] The aforementioned safety early warning method, device, and computer equipment for semiconductor factory operation scenarios first acquire images of the semiconductor factory operation scenario; then input the operation scenario images into a target detection model and receive the detection results from the target detection model; wherein, the target detection model includes an enhanced backbone network and a detection head; the enhanced backbone network is used to extract features from the operation scenario images to obtain multi-scale feature maps; the detection head is used to obtain the detection results corresponding to the detection head based on the multi-scale feature maps; the detection head includes a first type of target detection head and a second type of target detection head, and the first type of target detection head and the second type of target detection head establish spatial association constraints during training; the detection results include a first detection result corresponding to the first type of target detection head and a second detection result corresponding to the second type of target detection head; the target detection model, trained through spatial association constraints and deploying multiple types of target detection heads, identifies target objects in the operation scenario images; thus improving the accuracy of target object identification. Secondly, based on the first detection result and the second detection result, an isolation index is determined; finally, based on the isolation index, safety early warning information for the operation scenario is determined. The test results are used to determine the isolation index for assessing the safety status of the work scenario based on comprehensive geometric metrics. The safety status is quantified geometrically in multiple dimensions, which is suitable for the delicate work scenarios in semiconductor factories. The isolation index is used to determine the safety warning information for the work scenario in the semiconductor factory, which can improve the accuracy of safety warnings. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating a safety warning method for a semiconductor factory operation scenario in one embodiment;

[0038] Figure 2 This is a schematic diagram of the geometric contour in one embodiment;

[0039] Figure 3A schematic diagram of the geometric contours in another embodiment;

[0040] Figure 4 This is a flowchart illustrating the process of determining the isolation index in one embodiment;

[0041] Figure 5 This is a structural block diagram of a safety early warning device for a semiconductor factory operation scenario in one embodiment;

[0042] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] In one embodiment, such as Figure 1 As shown, a safety early warning method for a semiconductor factory operation scenario is provided. This embodiment illustrates the method applied to a terminal, but it is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps S102 to S108. Wherein:

[0045] Step S102: Obtain images of the semiconductor factory's work scene.

[0046] Optionally, the terminal acquires images of the semiconductor factory's operational scene. These images may include machinery, warning signs within the semiconductor factory such as fences and traffic cones, workers, and holes in the semiconductor factory floor during operations.

[0047] Optionally, an object detection model and a lightweight personnel movement detection model are deployed in the terminal. The terminal first acquires an initial work scene image of the semiconductor factory. Based on a preset Region of Interest (ROI), the initial work scene image is divided, and the ROI is either marked or cropped to retain the ROI, resulting in the work scene image. The terminal uses the object detection model to process the work scene image to obtain high-precision object detection results. The terminal also uses the lightweight personnel movement detection model to process the image outside the ROI in the initial work scene image, enabling the detection of personnel.

[0048] By using static ROI partitioning, the main computing power can be concentrated on high-risk physical areas within the region of interest. While ensuring the detection accuracy of the core area, the system can quickly obtain detection targets outside the region of interest, greatly improving the overall operating efficiency of the system and enabling practical engineering applications.

[0049] Step S104: Input the work scene image into the target detection model and receive the detection results from the target detection model.

[0050] The target detection model includes an enhanced backbone network and a detection head. The enhanced backbone network is used to extract features from the work scene image to obtain a multi-scale feature map. The detection head is used to obtain the detection result corresponding to the detection head based on the multi-scale feature map. The detection head includes a first-type target detection head and a second-type target detection head, and the first-type target detection head and the second-type target detection head establish spatial association constraints during training. The detection result includes a first detection result corresponding to the first-type target detection head and a second detection result corresponding to the second-type target detection head.

[0051] Spatial prior fusion loss is introduced during the training of the detection head in the object detection model. Traditional techniques train the first-class and second-class object detection heads independently, neglecting the inherent spatial correlation between the objects detected by the first-class and second-class object detection heads. Therefore, when training the first-class object detection head, in addition to calculating the conventional prediction box error, another additional constraint is introduced: the spatial prior fusion loss. This loss measures whether the distribution between the detected first and second detection results conforms to the expected statistical law, as shown in formula (1).

[0052] Formula (1)

[0053] In the formula, L is the total loss for training the first type of object detection head; This represents the error in a typical prediction box, such as classification loss or regression loss. Represents the spatial a priori fusion loss; This represents the balance coefficient, used to control the weight of the spatial prior fusion loss.

[0054] in, d represents the distance, which is determined based on the actual position H of the object detected by the second type of target detection head and the predicted position P of the object detected by the first type of target detection head; The prior distribution is a probability density function. For example, suppose that 50% of the objects detected by the first type of target detection head appear within 2 meters of the objects detected by the second type of target detection head. This can significantly reduce the false detection rate of target detection.

[0055] The spatial prior fusion loss function enables the first-type object detection head to utilize the implicit contextual information provided by the second-type object detection head. This implicit contextual information refers to the positional information provided by the second-type object detection head (e.g., a hole detection head) during the training phase (without explicitly inputting hole coordinates). After being constrained by the spatial prior fusion loss function, this statistical distribution pattern is learned and utilized by the first-type object detection head (e.g., a warning sign detection head) in a way that is not directly embedded in the network parameters. In actual testing, this design significantly reduced the false detection rate of the first-type object detection head, for example, demonstrating outstanding performance in distinguishing between "real ice cream cones" and "objects with similar shapes and colors," proving the unexpected synergistic effect achieved through algorithm design across multiple tasks.

[0056] Optionally, the terminal inputs the work scene image into the object detection model. The object detection model extracts features from the work scene image through an enhanced backbone network to obtain multi-scale feature maps. The backbone enhancement network can extract both global semantic features (large-scale feature maps) and local detail features (small-scale feature maps). The object detection model obtains a first detection result based on the multi-scale feature maps using a first type of object detection head; and a second detection result based on the multi-scale feature maps using a second type of object detection head. The terminal receives the detection result output by the object detection model, which includes both the first and second detection results. Because the first and second type of object detection heads establish spatial correlation constraints during training, the accuracy of object detection is greatly improved, filtering out some objects that do not conform to the prior distribution.

[0057] Step S106: Determine the isolation index based on the first and second test results.

[0058] Step S108: Determine the safety warning information for the work scenario based on the isolation index.

[0059] Optionally, the terminal calculates and determines an isolation index characterizing the comprehensive geometric indicators based on the first and second detection results. The terminal then uses this calculated isolation index to issue safety warnings for the semiconductor factory operation scenario, thereby determining whether to issue an alarm or not.

[0060] In the aforementioned safety early warning method for semiconductor factory work scenarios, the method first acquires images of the semiconductor factory work scenarios and inputs these images into a target detection model, receiving the model's detection results. The target detection model includes an enhanced backbone network and detection heads. The enhanced backbone network extracts features from the work scenario images to obtain multi-scale feature maps. The detection heads, based on the multi-scale feature maps, obtain corresponding detection results. The detection heads include a first-type target detection head and a second-type target detection head, which establish spatial correlation constraints during training. The detection results include a first detection result corresponding to the first-type target detection head and a second detection result corresponding to the second-type target detection head. The target detection model, trained using spatial correlation constraints and deploying multiple types of target detection heads, identifies target objects in the work scenario images, improving the accuracy of target object identification. Secondly, an isolation index is determined based on the first and second detection results. Finally, based on the isolation index, safety early warning information for the work scenario is determined. The test results are used to determine the isolation index for assessing the safety status of the work scenario based on comprehensive geometric metrics. The safety status is quantified geometrically in multiple dimensions, which is suitable for the delicate work scenarios in semiconductor factories. The isolation index is used to determine the safety warning information for the work scenario in the semiconductor factory, which can improve the accuracy of safety warnings.

[0061] In an exemplary embodiment, the detection head includes a third type of target detection head; the detection result also includes a third detection result corresponding to the third type of target detection head; the third detection result includes the location of the workers in the semiconductor factory; and the safety warning information of the work scenario is determined based on the isolation index, including: determining the safety warning information of the work scenario based on the isolation index and the location of the workers in the semiconductor factory.

[0062] Optionally, the terminal determines safety warning information for the work scenario based on the isolation index and the location of workers within the semiconductor factory. For example, in an image of a work scenario in a semiconductor factory without designated areas of interest, if the worker locations within the factory are empty and the isolation index is less than a threshold, a first warning message is immediately generated and sent for that work scenario. If the worker locations within the factory are empty and the isolation index is greater than or equal to the threshold, a second warning message is generated and sent. The first warning message has a higher warning level than the second warning message.

[0063] Optionally, for example, in the initial work scene image of a semiconductor factory that divides the area of ​​concern, the work scene image is also the image within the area of ​​concern. The isolation index and the location of the workers in the semiconductor factory are obtained. Combined with the location of the workers obtained from the image outside the area of ​​concern, the safety warning information is determined based on the isolation index, the location of the workers within the area of ​​concern, and the location of the workers outside the area of ​​concern.

[0064] If the isolation index is less than or equal to the threshold, and the locations of workers within the area of ​​concern are empty, and the locations of workers outside the area of ​​concern are also empty, then the third warning information in the safety warning information for this work scenario will be generated and sent immediately.

[0065] If the isolation index is less than or equal to the threshold, the location of the workers in the area of ​​concern is empty, and there are workers outside the area of ​​concern, then the fourth warning information in the safety warning information for this work scenario will be generated and sent immediately.

[0066] If the isolation index is greater than the threshold, the location of the workers in the area of ​​concern is empty, and there are workers outside the area of ​​concern, then the fifth warning information in the safety warning information for this work scenario will be generated and sent immediately.

[0067] If the isolation index is greater than the threshold, and the locations of workers within the area of ​​concern are empty, and the locations of workers outside the area of ​​concern are also empty, then the sixth warning message in the safety warning information for this work scenario will be generated and sent immediately.

[0068] If the isolation index is greater than the threshold, the location of the workers in the area of ​​concern is empty, and there are workers outside the area of ​​concern, then the seventh warning information in the safety warning information for this work scenario will be generated and sent immediately.

[0069] If the isolation index is greater than the threshold and there are workers in the area of ​​concern, no safety warning information will be sent for that work scenario.

[0070] It should be noted that the third, fourth, fifth, sixth, and seventh warning messages are unrelated to the first and second warning messages. The warning level of the third warning message is higher than that of the fourth warning message, the fourth warning message is higher than that of the fifth warning message, the fifth warning message is higher than that of the sixth warning message, and the sixth warning message is higher than that of the seventh warning message.

[0071] In this embodiment, by comprehensively judging and determining the safety warning information of the work scenario based on the location of the workers and the isolation index within the semiconductor factory, the accuracy of the safety warning information can be achieved.

[0072] In an exemplary embodiment, determining the isolation index based on the first detection result and the second detection result includes: processing the location of warning signs within the semiconductor factory using a boundary fitting algorithm to obtain the dynamic boundary of the warning signs; and determining the isolation index based on the dynamic boundary and the location of holes in the semiconductor factory floor.

[0073] The first test result includes the location of warning signs within the semiconductor factory; the second test result includes the location of holes in the ground within the semiconductor factory.

[0074] Boundary fitting algorithms include the α-shape algorithm, which is an algorithm that extracts the "shape" or "boundary" of a set of discrete points. It is seen as a generalization of the concept of "convex hull" and can generate concave boundaries, even those with holes, whose "concavity" is controlled by a key parameter α.

[0075] Optionally, the terminal calculates the positions of warning signs within the semiconductor factory using an α-shape algorithm to obtain the dynamic boundaries of the warning signs; for example... Figure 2 and Figure 3 The alpha shape geometric profile shown determines the isolation index based on the geometric positional relationship between the location of holes in the semiconductor factory grounds and the dynamic boundary.

[0076] In this embodiment, the isolation index can be determined by the location of warning signs within the semiconductor factory and the location of holes in the ground of the semiconductor factory.

[0077] In one exemplary embodiment, such as Figure 4 As shown, the isolation index is determined based on dynamic boundaries and the location of holes in the semiconductor factory site, including the following steps S402 to S408. Wherein:

[0078] Step S402: Determine the distance from the location of the hole in the semiconductor factory ground to the dynamic boundary.

[0079] The Isolation Score (IS) is a comprehensive geometric metric calculated based on the dynamic boundary generated by the α-shape and the positional relationship of the hole. It is used to quantify the physical isolation effect of the warning sign on the hole. IS measures the degree to which the center H of the hole is isolated by the dynamic boundary formed by the warning sign. The value is usually between 0 and 1 (or 0% to 100%), with a higher value indicating a better isolation effect.

[0080] Optionally, the terminal calculates the shortest distance from each side of the hole H in the semiconductor factory ground to the dynamic boundary of the polygon generated by α-shape, denoted as min_dist. The closer the distance, the higher the score.

[0081] Step S404: Determine the compact terms of the dynamic boundary.

[0082] The compact term is determined based on the ratio of the area to the perimeter of the dynamic boundary, and can be denoted as: This measures the compactness and continuity of the dynamic boundary. A larger ratio indicates a more complete and coherent dynamic boundary, rather than a sparse and dispersed one.

[0083] Optionally, the terminal obtains the area and perimeter of the dynamic boundary, and calculates the ratio of the area to the perimeter of the dynamic boundary to obtain the compact term.

[0084] Step S406: Determine the location of holes in the semiconductor factory grounds and the coverage metric of the dynamic boundary.

[0085] The coverage metric, cover_ratio, represents the measure of how much the hole H in the semiconductor factory ground is covered by the dynamic boundary of the polygon.

[0086] Optionally, the terminal can determine whether a point is within the dynamic boundary of a polygon using the ray casting method, or calculate the ratio of the distance from the hole to the center of the dynamic boundary of the polygon to the equivalent radius of the dynamic boundary of the polygon to obtain the coverage metric.

[0087] Step S408: Determine the isolation index based on distance, compactness, and coverage metrics.

[0088] Optionally, the terminal, based on the aforementioned distance min_dist, compacts the term. The isolation index is determined by the coverage metric cover_ratio, as described in formula (2).

[0089] Formula (2)

[0090] In the formula, , , and These are all hyperparameters. Based on testing and statistical analysis of real-world scenario data, the hyperparameter settings are as follows: =0.001, = 0.3, =0.2, =0.5. The larger the IS value, the better the warning sign can warn of holes in the semiconductor factory grounds. Let's assume the threshold is set to 8.5 (which can be adjusted according to actual conditions).

[0091] according to Figure 2 The minimum distance was calculated to be 203.93, the area perimeter rate to be 76.82, and the cover ratio to be 0.11. From this, the IS Index was calculated to be 15.66. A value greater than the threshold indicates that the warning sign effectively alerts the semiconductor factory to surface holes.

[0092] according to Figure 3 Calculate the parameters: Min distance: 779.29, Area perimeter rate: 17.16, Cover ratio: 0, and calculate IS Index: 3.57.

[0093] In this embodiment, a multi-dimensional "isolation index (IS)" that integrates minimum distance, boundary compactness, and coverage is proposed. This enables a concrete and accurate mathematical modeling of abstract security rules, transforms discrete detection targets into a quantitative assessment of "effective isolation," and improves the accuracy of determining "whether effective isolation has been formed."

[0094] In an exemplary embodiment, the multi-scale feature map includes a first-scale feature map and a second-scale feature map; the target detection model includes a first feature extraction branch and a second feature extraction branch; the first feature extraction branch is used to extract global features of the work scene image to obtain a first-scale feature map; the second feature extraction branch is used to extract local features of the work scene image to obtain a second-scale feature map.

[0095] In this model, the first-scale feature map is larger than the second-scale feature map. The first-scale feature map is a large-scale feature map, and the second-scale feature map is a small-scale feature map.

[0096] The enhanced backbone network includes a two-branch feature extraction backbone network, consisting of a first feature extraction branch and a second feature extraction branch. The first feature extraction branch, also known as the main branch, uses standard convolutional layers to extract global (semantic) features.

[0097] The second feature extraction branch is a small target enhancement branch that runs parallel to the main branch. The second feature extraction branch first performs local contrast adaptive enhancement preprocessing on the input task scene image, and then uses a series of small-sized convolutional kernels and dilated convolutional layers to specifically capture weak local texture anomalies and edge discontinuities. The output of the small target enhancement branch is specifically used to enhance the feature response to small-sized, low-contrast targets.

[0098] The principle behind designing a dual-branch feature extraction backbone network is as follows: In the special lighting environment of a semiconductor factory, the target size of holes in the semiconductor factory floor is very small, and the contrast with the color and texture of the floor is very low. There is a lack of stable edge and corner features. If only large-scale feature maps obtained through global (semantic) features are used, the error rate of detecting holes in the semiconductor factory floor will be very high.

[0099] In this embodiment, a dual-branch feature extraction backbone network is used to extract multi-scale feature maps, enabling accurate detection of holes in semiconductor factory floors. Through the second feature extraction branch and a dual-attention detection head, the average detection accuracy of holes in semiconductor factory floors is significantly improved on the semiconductor factory floor hole dataset, fundamentally transforming automated hazard perception in this scenario from "infeasible" to "highly reliable."

[0100] In an exemplary embodiment, the first type of target detection head is connected to the first feature extraction branch; the second type of target detection head is connected to the second feature extraction branch.

[0101] The first type of target detection head is the warning sign detection head in the semiconductor factory; the second type of target detection head is the ground hole detection head in the semiconductor factory.

[0102] The hole detection head for semiconductor factory ground surfaces uniquely connects to the output features of the small target enhancement branch. The head integrates a channel and spatial dual attention mechanism module, enabling it to autonomously focus on areas in the feature map that have subtle differences from the ground texture, significantly suppressing uniform background noise.

[0103] The warning sign detection head in the semiconductor factory is connected to the first feature extraction branch, which is mainly connected to the deep semantic features of the main branch. During the training process of the warning sign detection head in the semiconductor factory, a spatial prior fusion loss function is introduced. This function not only calculates the error between the predicted bounding box and the ground truth bounding box, but also adds an additional constraint to encourage the model to learn that "there is a statistical correlation in the spatial distribution of the predicted locations of warning signs and holes," enabling the model to implicitly learn the association logic of targets within the scene.

[0104] The operator inspection head in the semiconductor factory is connected to the first feature extraction branch, which is mainly connected to the deep semantic features of the main branch.

[0105] In this embodiment, by deploying three decoupled but parameter-sharing dedicated detection heads, and by connecting the ground hole detection head of the semiconductor factory only to the second feature extraction branch, stable and high-precision detection of "invisible" small targets in industrial scenarios is achieved.

[0106] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0107] Based on the same inventive concept, this application also provides a safety warning device for a semiconductor factory operation scenario, which implements the safety warning method for the semiconductor factory operation scenario described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the safety warning device for a semiconductor factory operation scenario provided below can be found in the limitations of the safety warning method for a semiconductor factory operation scenario described above, and will not be repeated here.

[0108] In one exemplary embodiment, such as Figure 5 As shown, a safety early warning device for a semiconductor factory operation scenario is provided, comprising: an acquisition module 501, a processing module 502, a first determination module 503, and a second determination module 504, wherein:

[0109] The acquisition module 501 is used to acquire images of the working scene in the semiconductor factory.

[0110] The processing module 502 is used to input the work scene image into the target detection model and receive the detection result of the target detection model. The target detection model includes an enhanced backbone network and a detection head. The enhanced backbone network is used to extract features from the work scene image to obtain a multi-scale feature map. The detection head is used to obtain the detection result corresponding to the detection head based on the multi-scale feature map. The detection head includes a first-type target detection head and a second-type target detection head, and the first-type target detection head and the second-type target detection head establish spatial correlation constraints during training. The detection result includes a first detection result corresponding to the first-type target detection head and a second detection result corresponding to the second-type target detection head.

[0111] The first determining module 503 is used to determine the isolation index based on the first detection result and the second detection result.

[0112] The second determining module 504 is used to determine the safety warning information of the work scenario based on the isolation index.

[0113] In an exemplary embodiment, the detection head includes a third type of target detection head; the detection result also includes a third detection result corresponding to the third type of target detection head; the third detection result includes the location of the workers in the semiconductor factory; the second determination module 504 is used to determine safety warning information for the work scenario based on the isolation index and the location of the workers in the semiconductor factory.

[0114] In an exemplary embodiment, the first detection result includes the location of warning signs within the semiconductor factory; the second detection result includes the location of holes in the semiconductor factory floor; the first determination module 503 is used to process the location of warning signs within the semiconductor factory using a boundary fitting algorithm to obtain the dynamic boundary of the warning signs; and to determine the isolation index based on the dynamic boundary and the location of holes in the semiconductor factory floor.

[0115] In an exemplary embodiment, a first determining module 503 is configured to: determine the distance from the location of a hole in the semiconductor factory ground to a dynamic boundary; determine a compact term of the dynamic boundary; wherein the compact term is determined based on the ratio of the area to the perimeter of the dynamic boundary; determine a coverage metric between the location of the hole in the semiconductor factory ground and the dynamic boundary; and determine an isolation index based on the distance, the compact term, and the coverage metric.

[0116] In one exemplary embodiment, the isolation index is calculated using the following formula:

[0117]

[0118] In the formula, IS represents the isolation index; min_dist represents the distance from the location of the hole in the semiconductor factory ground to the dynamic boundary; The compact term represents the dynamic boundary; cover_ratio represents the coverage measure of the location of holes in the semiconductor factory site relative to the dynamic boundary. , , and This is a hyperparameter.

[0119] In an exemplary embodiment, the multi-scale feature map includes a first-scale feature map and a second-scale feature map; the target detection model includes a first feature extraction branch and a second feature extraction branch; the first feature extraction branch is used to extract global features of the work scene image to obtain a first-scale feature map; the second feature extraction branch is used to extract local features of the work scene image to obtain a second-scale feature map.

[0120] In an exemplary embodiment, the first type of target detection head is connected to the first feature extraction branch; the second type of target detection head is connected to the second feature extraction branch.

[0121] In one embodiment, the acquisition module 501 is further configured to acquire an initial work scene image of the semiconductor factory; and divide the initial work scene image according to a preset area of ​​interest to obtain a work scene image of the semiconductor factory.

[0122] The various modules in the safety early warning device for the aforementioned semiconductor factory operation scenario can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0123] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores safety warning information data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a safety warning method for a semiconductor factory operation scenario.

[0124] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0126] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0127] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A safety early warning method for semiconductor factory operation scenarios, characterized in that, The method includes: Acquire images of the operational scene in a semiconductor factory; The work scene image is input into the target detection model, and the detection results of the target detection model are received. The target detection model includes an enhanced backbone network and a detection head. The enhanced backbone network is used to extract features from the work scene image to obtain a multi-scale feature map. The detection head is used to obtain a detection result corresponding to the multi-scale feature map. The detection head includes a first-type target detection head and a second-type target detection head, and the first-type target detection head and the second-type target detection head establish spatial association constraints during training. The detection results include a first detection result corresponding to the first-type target detection head and a second detection result corresponding to the second-type target detection head. The isolation index is determined based on the first and second test results; Based on the isolation index, safety warning information for the work scenario is determined.

2. The method according to claim 1, characterized in that, The detection head includes a third type of target detection head; the detection result also includes a third detection result corresponding to the third type of target detection head; the third detection result includes the location of the workers in the semiconductor factory; The step of determining safety warning information for the work scenario based on the isolation index includes: Based on the isolation index and the location of workers within the semiconductor factory, safety warning information for the work scenario is determined.

3. The method according to claim 1, characterized in that, The first detection result includes the location of warning signs within the semiconductor factory; the second detection result includes the location of holes in the semiconductor factory floor; the step of determining the isolation index based on the first and second detection results includes: The location of warning signs within the semiconductor factory is processed using a boundary fitting algorithm to obtain the dynamic boundaries of the warning signs. The isolation index is determined based on the dynamic boundary and the location of the holes in the semiconductor factory grounds.

4. The method according to claim 3, characterized in that, The determination of the isolation index based on the dynamic boundary and the location of holes in the semiconductor factory site includes: Determine the distance from the location of the perforation in the semiconductor factory ground surface to the dynamic boundary; Determine the compact term of the dynamic boundary; wherein the compact term is determined based on the ratio of the area to the perimeter of the dynamic boundary; Determine the coverage metric between the location of the perforations in the semiconductor factory ground surface and the dynamic boundary; The isolation index is determined based on the distance, the compactness term, and the coverage metric.

5. The method according to claim 4, characterized in that, The isolation index is calculated using the following formula: In the formula, IS represents the isolation index; min_dist represents the distance from the location of the hole in the semiconductor factory ground to the dynamic boundary; The compact term represents the dynamic boundary; cover_ratio represents the coverage measure of the location of the holes in the semiconductor factory grounds relative to the dynamic boundary. , , and This is a hyperparameter.

6. The method according to claim 1, characterized in that, The multi-scale feature map includes a first-scale feature map and a second-scale feature map; the target detection model includes a first feature extraction branch and a second feature extraction branch; the first feature extraction branch is used to extract global features of the work scene image to obtain a first-scale feature map; the second feature extraction branch is used to extract local features of the work scene image to obtain a second-scale feature map.

7. The method according to claim 6, characterized in that, The first type of target detection head is connected to the first feature extraction branch; the second type of target detection head is connected to the second feature extraction branch.

8. The method according to claim 1, characterized in that, The acquisition of images of the semiconductor factory's operational scene includes: Acquire initial operational scene images of a semiconductor factory; The semiconductor factory's work scene image is obtained by dividing the initial work scene image according to the preset area of ​​interest.

9. A safety early warning device for semiconductor factory operation scenarios, characterized in that, The device includes: The acquisition module is used to acquire images of the operational scene in a semiconductor factory. A processing module is used to input the work scene image into a target detection model and receive the detection results from the target detection model. The target detection model includes an enhanced backbone network and a detection head. The enhanced backbone network is used to extract features from the work scene image to obtain a multi-scale feature map. The detection head is used to obtain a detection result corresponding to the detection head based on the multi-scale feature map. The detection head includes a first-type target detection head and a second-type target detection head, and the first-type target detection head and the second-type target detection head establish spatial association constraints during training. The detection results include a first detection result corresponding to the first-type target detection head and a second detection result corresponding to the second-type target detection head. The first determining module is used to determine the isolation index based on the first detection result and the second detection result; The second determining module is used to determine the safety warning information of the work scenario based on the isolation index.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images