Behavior identification alarm method, system, device and medium for semiconductor smart factory

Abstract

The application discloses a kind of behavior identification alarm method, system, equipment and medium of semiconductor intelligent factory, method includes receiving the real-time data obtained by real-time shooting of collection equipment to job operation behavior;The real-time data is analyzed and processed;Call pre-trained detection model to predict the image data after analysis and processing, generate prediction result;Based on the prediction result and pre-defined business rules, fusion decision is made, and decision output is generated;Based on the decision output, alarm information is generated, and the alarm information is output to alarm system;Receive the artificial re-judgment input sent by audit terminal, and based on the artificial re-judgment input and the prediction result, statistical analysis is carried out, and statistical report is generated to be used for the iterative optimization of the detection model.The application can efficiently and accurately identify irregular behavior, and continuously optimize through fusion decision and feedback mechanism to improve identification reliability.

Description

Technical Field

[0001] This invention belongs to the field of semiconductor intelligent manufacturing technology, and in particular relates to a behavior recognition alarm method, system, equipment and medium for semiconductor intelligent factories. Background Technology

[0002] Semiconductor smart factories are a new manufacturing model that integrates advanced technologies such as the Internet of Things (IoT), Artificial Intelligence (AI), digital twins, and edge computing to achieve intelligent manufacturing across the entire semiconductor manufacturing chain, from design and production to the supply chain. In semiconductor factories, operational safety in tank truck scenarios is paramount. Accurate, real-time, and efficient detection of violations such as fence integrity, PPE compliance, and first-aid kit configuration in tank truck scenarios is a crucial aspect of semiconductor plant safety management. Existing technologies typically rely on manual review of photographic records of operational steps (such as images or videos taken by cameras). This method is inefficient, error-prone, and cannot meet the demands of real-time monitoring and rapid response. Although visual algorithms have been introduced for anomaly detection, recognition methods based on single AI models have low accuracy and lack a mechanism to integrate model predictions (such as confidence scores) with business rules (such as threshold judgments) for decision-making, resulting in poor reliability and frequent false alarms or missed alarms. Furthermore, existing technologies lack an effective mechanism to automatically collect and statistically analyze human feedback to continuously optimize the recognition model, making it difficult for the overall recognition system to adapt to dynamically changing environments and achieve long-term stable performance improvements. Therefore, there is an urgent need for a method that can efficiently and accurately identify violations and support continuous optimization through integrated decision-making and feedback mechanisms to improve the reliability of identification. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a behavior recognition alarm method, system, device and medium for semiconductor smart factories, which can efficiently and accurately identify violations and support continuous optimization through integrated decision and feedback mechanisms to improve the reliability of recognition.

[0004] The first aspect of this invention discloses a behavior recognition and alarm method for a semiconductor smart factory, comprising the following steps:

[0005] Step 1: Receive real-time data, wherein the real-time data is obtained by the acquisition device from real-time shooting of the operation behavior, and the real-time data includes video streams or image files;

[0006] Step 2: The real-time data is parsed and processed. When the real-time data is a video stream, frame extraction is performed based on a preset frame extraction frequency to generate multiple image data. When the real-time data is an image file, the image data is directly obtained.

[0007] Step 3: Call the pre-trained detection model to predict the parsed image data and generate prediction results, wherein the prediction results include behavior recognition labels and confidence scores;

[0008] Step 4: Based on the prediction results and predefined business rules, perform fusion decision-making to generate decision output, wherein the fusion decision-making includes comparing the confidence score with the threshold of the business rule and determining the final behavior recognition state based on the comparison result;

[0009] Step 5: Generate alarm information based on the decision output, and output the alarm information to the alarm system;

[0010] Step six: Receive manual review input from the audit terminal, wherein the manual review input is a correction feedback of the alarm information, and perform statistical analysis based on the manual review input and the prediction results to generate a statistical report for iterative optimization of the detection model.

[0011] The behavior recognition and alarm method for the aforementioned semiconductor smart factory, in step four, specifically includes the following process for the fusion decision:

[0012] When the confidence score is higher than the first threshold of the business rule, the behavior identification status is directly determined to be a violation;

[0013] When the confidence score is lower than the second threshold of the business rule, the behavior recognition status is directly determined to be compliant;

[0014] When the confidence score is between the first threshold and the second threshold, a weighted calculation is performed based on the logical conditions of the business rules to determine the behavior recognition status.

[0015] The above-mentioned behavior recognition and alarm method for a semiconductor smart factory includes the following training process for the detection model:

[0016] Obtain a labeled dataset, wherein the labeled dataset is generated by manually annotating the violation images, including behavior recognition labels;

[0017] The initial detection model is trained using the labeled dataset, and the model parameters are iteratively optimized until a preset accuracy threshold is reached.

[0018] Release the trained detection model as an application programming interface (API).

[0019] In the aforementioned behavior recognition and alarm method for a semiconductor smart factory, step six, the specific process of statistical analysis includes:

[0020] Collect the manually reviewed inputs and corresponding prediction results, and calculate the prediction accuracy and false alarm rate.

[0021] The statistical report is generated based on the metrics, and the statistical report is used to adjust the training parameters of the detection model or update the business rules.

[0022] The aforementioned behavior recognition and alarm method for a semiconductor smart factory includes model confidence calibration parameters in the statistical report, which are generated through the following steps:

[0023] (1) Calculate the deviation δ between the manual review result and the prediction result, defined as follows: Let y i human Let y be the probability value of the violation in the manual review of the i-th sample. i pred To predict the probability of violation for the i-th sample, then:

[0024]

[0025] Where K is the number of re-judged samples;

[0026] (2) When δ>γ, γ is the calibration threshold, and the confidence score output by the detection model is compressed and transformed:

[0027]

[0028] Among them, Score raw λ represents the original confidence score output by the model, and λ is the calibration intensity coefficient.

[0029] In the above-mentioned behavior recognition alarm method for semiconductor smart factories, the frame sampling frequency is dynamically adjusted according to the real-time data type in step two.

[0030] The aforementioned behavior recognition and alarm method for a semiconductor smart factory, in the fusion decision-making process of step four:

[0031] When the prediction results of N consecutive frames are detected to have all entered the threshold buffer, a dynamic rule strengthening mechanism is activated, including:

[0032] (a) Automatically increase the weighting factor α of the business rule, using the following formula:

[0033]

[0034] Where, α new The adjusted weighting factor is α, where C is the current consecutive frame number, β is the adjustment coefficient, and α is the weighting factor. default As the default weighting factor, α max This is a preset upper limit value;

[0035] (b) Recalculate the behavior recognition state based on the enhanced weighted factors.

[0036] The second aspect of this invention discloses a behavior recognition and alarm system for a semiconductor smart factory, including a data receiving module, a data parsing module, a model prediction module, a fusion decision module, an alarm generation module, and a report generation module;

[0037] The data receiving module is used to receive real-time data, wherein the real-time data is obtained by the acquisition device from real-time shooting of the operation behavior, and the real-time data includes video streams or image files;

[0038] The data parsing module is used to parse and process the real-time data. When the real-time data is a video stream, it performs frame extraction operation based on a preset frame extraction frequency to generate multiple image data. When the real-time data is an image file, it directly obtains the image data.

[0039] The model prediction module is used to call a pre-trained detection model to predict the parsed image data and generate prediction results, wherein the prediction results include behavior recognition labels and confidence scores.

[0040] The fusion decision module is used to make fusion decisions based on the prediction results and predefined business rules, and generate decision outputs. The fusion decision includes comparing the confidence score with the threshold of the business rule and determining the final behavior recognition state based on the comparison result.

[0041] An alarm generation module is used to generate alarm information based on the decision output and output the alarm information to the alarm system.

[0042] The report generation module is used to receive manual review input sent by the review terminal, wherein the manual review input is a correction feedback of the alarm information, and to perform statistical analysis based on the manual review input and the prediction results to generate a statistical report for iterative optimization of the detection model.

[0043] A third aspect of the present invention discloses an electronic device, comprising: a memory and a processor, wherein the processor and the memory are connected;

[0044] The memory is used to store programs;

[0045] The processor invokes a program stored in the memory to execute the method provided by the first aspect embodiment and / or any possible embodiment in combination with the first aspect embodiment.

[0046] The fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a computer to perform the methods provided by the embodiments of the first aspect and / or any possible embodiments in combination with the embodiments of the first aspect.

[0047] Compared with existing technologies, this invention has the following advantages: It effectively solves the problems of low efficiency, low accuracy, poor reliability, and lack of continuous optimization mechanisms in existing technologies. Specifically, the fusion decision-making mechanism in step four (i.e., comparing the confidence score in the prediction results with the threshold of predefined business rules to determine the final behavior recognition status) significantly improves the accuracy and reliability of identifying violations, reduces false alarms or missed alarms caused by a single AI model, thereby reducing the incidence of danger in tank truck scenarios. Simultaneously, the manual review input and statistical analysis function in step six (i.e., receiving manual correction feedback and generating statistical reports for iterative optimization of the detection model) realizes the system's self-learning closed loop, continuously optimizing the model through dynamic feedback collection, and improving long-term recognition performance. Furthermore, the overall method automates the processing of real-time data (steps one to three) and generates alarm information (step five), replacing the tedious process of manually reviewing photo records, significantly improving monitoring efficiency and response speed.

[0048] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0049] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention.

[0050] Figure 2 This is a system module diagram of Embodiment 2 of the present invention. Detailed Implementation

[0051] Example 1

[0052] like Figure 1 As shown, a behavior recognition and alarm method for a semiconductor smart factory includes the following steps:

[0053] Step 1: Receive real-time data, wherein the real-time data is obtained by the acquisition device from real-time shooting of the operation behavior, and the real-time data includes video streams or image files;

[0054] During implementation, the system receives data from acquisition devices (such as industrial cameras) via a data interface (e.g., a network protocol). These acquisition devices are installed in the tank truck operating area to capture operational activities in real time. The data includes video streams (continuous frame sequences) or image files (single photographs). During implementation, the system verifies the data format (e.g., H.264 video or JPEG images) to ensure compatibility.

[0055] During tank truck PPE inspections, for example, a camera captures the operator putting on the PPE, and the system receives the video stream as input; or it receives uploaded photographic image files to record the valve status.

[0056] Step 2: The real-time data is parsed and processed. When the real-time data is a video stream, frame extraction is performed based on a preset frame extraction frequency to generate multiple image data. When the real-time data is an image file, the image data is directly obtained.

[0057] During implementation, the system parses real-time data: if the input is a video stream, keyframes are extracted based on a preset frame extraction frequency (e.g., 1 frame per second) to generate multiple image data; if the input is an image file, the file content is directly read as image data. The frame extraction frequency can be set (e.g., via a configuration file) to ensure coverage of critical action moments. Video parsing uses libraries (e.g., FFmpeg) for frame extraction, and image processing uses standard decoding.

[0058] By standardizing the raw data into an image format, it becomes easier for the model to process the data and reduces redundant data.

[0059] For example: when inputting a video stream, the system extracts one frame every 5 seconds to generate a sequence image of the tanker operation; when inputting an image file, it directly obtains the complete image of the fence taken at the time.

[0060] Step 3: Call the pre-trained detection model to predict the parsed image data and generate prediction results, wherein the prediction results include behavior recognition labels and confidence scores;

[0061] During implementation, the system calls the API of a pre-trained deep learning model (such as YOLO or Faster R-CNN) to process image data. The model takes an image as input and outputs predictions including a behavior recognition label (such as "PPE not worn") and a confidence score (a value from 0 to 1). The model is optimized during training and then called in real time via the API after deployment. The prediction process includes image preprocessing (scaling, normalization) and model inference.

[0062] By automatically identifying violations and providing quantified confidence levels, human intervention is reduced.

[0063] In practice, the model processes frame-by-frame images, such as detecting that a tanker truck operator is not wearing gloves, and outputs the label "PPE violation" with a confidence level of 0.75.

[0064] Step 4: Based on the prediction results and predefined business rules, perform fusion decision-making to generate decision output, wherein the fusion decision-making includes comparing the confidence score with the threshold of the business rule and determining the final behavior recognition state based on the comparison result;

[0065] During implementation, the system loads predefined business rules (such as XML file storage thresholds: first threshold 0.7, second threshold 0.3). It compares the confidence score with the thresholds: if higher than 0.7, it directly determines a violation; if lower than 0.3, it determines compliance; if in between, it performs a weighted calculation based on rule logic (such as "fire extinguishers must be configured") (weights are based on rule importance). The decision output is the final state (violation or compliance). In implementation, conditional judgments and weighted algorithms are used. By integrating AI and rules, false positives and false negatives are reduced, improving decision accuracy.

[0066] In actual use, the confidence level of 0.65 (fire extinguisher testing) is between 0.3 and 0.7. The system combines the rule "fire extinguishers must be qualified" with weighted calculation (such as weight 0.6) and judges it as a violation decision.

[0067] Step 5: Generate alarm information based on the decision output, and output the alarm information to the alarm system;

[0068] During implementation, if the decision output indicates a violation, the system generates an alarm message (such as a JSON message containing the violation type, image ID, and timestamp) and outputs it to the alarm system (such as a monitoring dashboard or app) via a message queue (such as RabbitMQ). The alarm system displays the information and triggers notifications (such as sound or SMS). Real-time alarms facilitate timely risk handling and improve security response.

[0069] In actual use, for example, if the decision output fence is missing or violated, the system generates an alarm message "Tank truck area fence violation" and sends it to the plant monitoring APP, and the operator receives the notification.

[0070] Step six: Receive manual review input from the audit terminal, wherein the manual review input is a correction feedback of the alarm information, and perform statistical analysis based on the manual review input and the prediction results to generate a statistical report for iterative optimization of the detection model.

[0071] During implementation, the system provides a web interface for operators to manually review alarms on the audit terminal (e.g., by clicking "confirm" or "correct"). It receives review input (e.g., correcting to compliance) and collects corresponding prediction results. Statistical analysis includes calculating metrics (accuracy, false alarm rate) and generating statistical reports (CSV or PDF files). These reports are used for model optimization: such as adjusting training parameters or updating business rules. Through a closed-loop feedback system, the model is optimized via manual input, reducing long-term errors.

[0072] In practical use, for example, if an operator re-determines an alarm as a false alarm (when in fact PPE is worn in compliance), the system calculates the accuracy of the prediction and generates a report showing a high false alarm rate, which is then used for the next round of model training and optimization.

[0073] In summary, during implementation, the system receives real-time video streams from cameras in the tanker truck operation area or uploaded image files (such as fence detection images). If the input is video, it extracts frames at a preset frequency (such as 1 frame every 5 seconds) to generate an image sequence. It then calls a pre-trained YOLO model to identify behavioral labels (such as "PPE not worn") and confidence scores (such as 0.75) in the images. Combined with predefined business rules (such as "PPE wearing compliance threshold 0.7"), it makes a fusion decision: if the confidence score is higher than 0.7, it directly judges the violation and generates an alarm message that is pushed to the plant monitoring APP. The human operator reviews the alarm results on the terminal (such as correcting false alarms). The system statistically analyzes the review data to generate a false alarm rate report (such as 15%), which is used to optimize the model training parameters, thereby improving the accuracy of tanker truck operation review and reducing the incidence of danger.

[0074] In this embodiment, the specific process of the fusion decision in step four includes:

[0075] When the confidence score is higher than the first threshold of the business rule, the behavior identification status is directly determined to be a violation;

[0076] When the confidence score is lower than the second threshold of the business rule, the behavior recognition status is directly determined to be compliant;

[0077] When the confidence score is between the first threshold and the second threshold, a weighted calculation is performed based on the logical conditions of the business rules to determine the behavior recognition status.

[0078] During implementation, in the fusion decision-making process, the system sets a first threshold of 0.7 (violation line) and a second threshold of 0.3 (compliance line). When the fire extinguisher detection confidence level is 0.55 (between 0.3 and 0.7), the system automatically calls the business rule logic (such as "fire extinguisher must be qualified") to perform weighted calculation (weight coefficient 0.6) and outputs the enhanced violation decision, avoiding missed judgments due to ambiguity of confidence level and improving the reliability of high-risk equipment detection.

[0079] In this embodiment, the training process of the detection model includes:

[0080] Obtain a labeled dataset, wherein the labeled dataset is generated by manually annotating the violation images, including behavior recognition labels;

[0081] The initial detection model is trained using the labeled dataset, and the model parameters are iteratively optimized until a preset accuracy threshold is reached.

[0082] Release the trained detection model as an application programming interface (API).

[0083] During implementation, the system acquires a dataset of violation images from tanker truck operation scenarios (such as 2000 images labeled "fence missing" and "PPE not worn"), generates structured training data through manual annotation, trains an initial YOLOv5 model using the PyTorch framework, iteratively optimizes the convolutional layer parameters until the test set accuracy reaches over 95% (a preset threshold), and finally encapsulates the model as a RESTful API interface for application-stage calls. For example, during the training of the tanker truck fire extinguisher detection model, 800 images labeled "fire extinguisher missing" were iterated 20 times, and the output model API was deployed to the server to achieve end-to-end violation recognition capabilities, providing basic predictive support for fusion decision-making.

[0084] In this embodiment, the specific process of statistical analysis in step six includes:

[0085] Collect the manually reviewed inputs and corresponding prediction results, and calculate the prediction accuracy and false alarm rate.

[0086] The statistical report is generated based on the metrics, and the statistical report is used to adjust the training parameters of the detection model or update the business rules.

[0087] During implementation, the system collects manual review data (such as operators correcting 10 PPE false alarms) and corresponding model prediction results, calculates the current model false alarm rate (such as 18%), and generates a statistical report to guide the update of business rules (adjusting the threshold from 0.7 to 0.65), thereby reducing the frequency of false alarms in tank truck safety audits.

[0088] In this embodiment, the statistical report includes model confidence calibration parameters, which are generated through the following steps:

[0089] (1) Calculate the deviation δ between the manual review result and the prediction result, defined as follows: Let y i human Let y be the probability value of the violation in the manual review of the i-th sample. i pred To predict the probability of violation for the i-th sample, then:

[0090]

[0091] Where K is the number of re-judged samples;

[0092] (2) When δ>γ, γ is the calibration threshold, and the confidence score output by the detection model is compressed and transformed:

[0093]

[0094] Among them, Score rawλ represents the original confidence score output by the model, and λ is the calibration intensity coefficient.

[0095] Calculate the deviation δ between manual review and model prediction (e.g., the average deviation of first aid kit detection is 0.12). When δ exceeds the calibration threshold γ (0.1), apply a compression formula (e.g., Score) to the original confidence level. calibrated =0.8 / (1+0.5×0.12)≈0.75), the calibrated score is closer to the level of human judgment, solving the problem of model overconfidence in low-risk scenarios (such as the displacement of the first aid kit).

[0096] In this embodiment, in step two, the frame sampling frequency is dynamically adjusted according to the real-time data type.

[0097] During implementation, the frame extraction strategy is dynamically adjusted based on the real-time data type: during the high-risk phase of tank truck unloading, the frame extraction frequency is automatically increased to 2 frames per second (default 1 frame per second) to ensure the capture of instantaneous violations such as valve leakage, optimize the allocation of computing resources, and improve the recognition rate of key actions.

[0098] In this embodiment, during the fusion decision process in step four:

[0099] When the prediction results of N consecutive frames are detected to have all entered the threshold buffer, a dynamic rule strengthening mechanism is activated, including:

[0100] (a) Automatically increase the weighting factor α of the business rule, using the following formula:

[0101]

[0102] Where, α new The adjusted weighting factor is α, where C is the current consecutive frame number, β is the adjustment coefficient, and α is the weighting factor. default As the default weighting factor, α max This is a preset upper limit value;

[0103] (b) Recalculate the behavior recognition state based on the enhanced weighted factors.

[0104] During implementation, when the confidence level of the tank truck pipeline status detection is within the threshold range (e.g., 0.4-0.6) for three consecutive frames, the dynamic reinforcement mechanism is activated: according to formula α new =min(0.5+0.2×3 / 3, 0.8) increases the weighting factor to 0.7, recalculates and outputs violation decisions, solving the problem of inconsistent sequence judgment caused by single-frame ambiguity (such as the continuous risk judgment of pipeline micro-leakage).

[0105] Example 2

[0106] like Figure 2As shown, a behavior recognition and alarm system for a semiconductor smart factory includes a data receiving module, a data parsing module, a model prediction module, a fusion decision module, an alarm generation module, and a report generation module.

[0107] The data receiving module is used to receive real-time data, wherein the real-time data is obtained by the acquisition device from real-time shooting of the operation behavior, and the real-time data includes video streams or image files;

[0108] The data parsing module is used to parse and process the real-time data. When the real-time data is a video stream, it performs frame extraction operation based on a preset frame extraction frequency to generate multiple image data. When the real-time data is an image file, it directly obtains the image data.

[0109] The model prediction module is used to call a pre-trained detection model to predict the parsed image data and generate prediction results, wherein the prediction results include behavior recognition labels and confidence scores.

[0110] The fusion decision module is used to make fusion decisions based on the prediction results and predefined business rules, and generate decision outputs. The fusion decision includes comparing the confidence score with the threshold of the business rule and determining the final behavior recognition state based on the comparison result.

[0111] An alarm generation module is used to generate alarm information based on the decision output and output the alarm information to the alarm system.

[0112] The report generation module is used to receive manual review input sent by the review terminal, wherein the manual review input is a correction feedback of the alarm information, and to perform statistical analysis based on the manual review input and the prediction results to generate a statistical report for iterative optimization of the detection model.

[0113] The behavior recognition alarm system for a semiconductor smart factory provided in this embodiment has the same implementation principle and technical effect as the method embodiment in Embodiment 1. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in Embodiment 1.

[0114] Example 3

[0115] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a computer to perform the behavior recognition and alarm method for a semiconductor smart factory described in Embodiment 1 above.

[0116] Example 4

[0117] An electronic device includes: a memory and a processor, wherein the processor and the memory are connected;

[0118] The memory is used to store programs;

[0119] The processor invokes a program stored in the memory to execute a behavior recognition alarm method for a semiconductor smart factory as described in Embodiment 1.

[0120] It should be noted that the electronic device mentioned may be, but is not limited to, personal computers (PCs), tablet computers, mobile internet devices (MIDs), etc.

[0121] It should be noted that processors, memory, and other components that may be present in electronic devices are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, processors, memory, and other components may be electrically connected to each other via one or more communication buses or signal lines.

[0122] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0124] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0125] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, laptop, server, mobile phone, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A behavior recognition and alarm method for a semiconductor smart factory, characterized in that, Includes the following steps: Step 1: Receive real-time data, wherein the real-time data is obtained by the acquisition device from real-time shooting of the operation behavior, and the real-time data includes video streams or image files; Step 2: The real-time data is parsed and processed. When the real-time data is a video stream, frame extraction is performed based on a preset frame extraction frequency to generate multiple image data. When the real-time data is an image file, the image data is directly obtained. Step 3: Call the pre-trained detection model to predict the parsed image data and generate prediction results, wherein the prediction results include behavior recognition labels and confidence scores; Step 4: Based on the prediction results and predefined business rules, perform fusion decision-making to generate decision output, wherein the fusion decision-making includes comparing the confidence score with the threshold of the business rule and determining the final behavior recognition state based on the comparison result; Step 5: Generate alarm information based on the decision output, and output the alarm information to the alarm system; Step 6: Receive manual review input from the audit terminal, wherein the manual review input is a correction feedback of the alarm information, and perform statistical analysis based on the manual review input and the prediction results to generate a statistical report for iterative optimization of the detection model; In step six, the specific process of statistical analysis includes: Collect the manually reviewed inputs and corresponding prediction results, and calculate the prediction accuracy and false alarm rate. The statistical report is generated based on the indicators, and the statistical report is used to adjust the training parameters of the detection model or update the business rules. The statistical report includes model confidence calibration parameters, which are generated through the following steps: (1) Calculate the deviation δ between the manual review result and the prediction result, defined as follows: Let y i human Let y be the probability value of the violation in the manual review of the i-th sample. i pred To predict the probability of violation for the i-th sample, then: Where K is the number of re-judged samples; (2) When δ>γ, γ is the calibration threshold, and the confidence score output by the detection model is compressed and transformed: Among them, Score raw λ represents the original confidence score output by the model, and λ is the calibration intensity coefficient. In the fusion decision-making process of step four: When the prediction results of N consecutive frames are detected to have all entered the threshold buffer, a dynamic rule strengthening mechanism is activated, including: (a) Automatically increase the weighting factor α of the business rule, using the following formula: Where, α new The adjusted weighting factor is α, where C is the current consecutive frame number, β is the adjustment coefficient, and α is the weighting factor. default As the default weighting factor, α max This is a preset upper limit value; (b) Recalculate the behavior recognition state based on the enhanced weighted factors.

2. The behavior recognition and alarm method for a semiconductor smart factory according to claim 1, characterized in that, In step four, the specific process of the fusion decision includes: When the confidence score is higher than the first threshold of the business rule, the behavior identification status is directly determined to be a violation; When the confidence score is lower than the second threshold of the business rule, the behavior recognition status is directly determined to be compliant; When the confidence score is between the first threshold and the second threshold, a weighted calculation is performed based on the logical conditions of the business rules to determine the behavior recognition status.

3. The behavior recognition and alarm method for a semiconductor smart factory according to claim 1, characterized in that, The training process of the detection model includes: Obtain a labeled dataset, wherein the labeled dataset is generated by manually annotating the violation images, including behavior recognition labels; The initial detection model is trained using the labeled dataset, and the model parameters are iteratively optimized until a preset accuracy threshold is reached. Release the trained detection model as an application programming interface (API).

4. The behavior recognition and alarm method for a semiconductor smart factory according to claim 1, characterized in that, In step two, the frame sampling frequency is dynamically adjusted according to the real-time data type.

5. A behavior recognition and alarm system for a semiconductor smart factory, characterized in that, It includes a data receiving module, a data parsing module, a model prediction module, a fusion decision module, an alarm generation module, and a report generation module; The data receiving module is used to receive real-time data, wherein the real-time data is obtained by the acquisition device from real-time shooting of the operation behavior, and the real-time data includes video streams or image files; The data parsing module is used to parse and process the real-time data. When the real-time data is a video stream, it performs frame extraction operation based on a preset frame extraction frequency to generate multiple image data. When the real-time data is an image file, it directly obtains the image data. The model prediction module is used to call a pre-trained detection model to predict the parsed image data and generate prediction results, wherein the prediction results include behavior recognition labels and confidence scores. The fusion decision module is used to make fusion decisions based on the prediction results and predefined business rules, and generate decision outputs. The fusion decision includes comparing the confidence score with the threshold of the business rule and determining the final behavior recognition state based on the comparison result. In the process of integrated decision-making: When the prediction results of N consecutive frames are detected to have all entered the threshold buffer, a dynamic rule strengthening mechanism is activated, including: (a) Automatically increase the weighting factor α of the business rule, using the following formula: Where, α new The adjusted weighting factor is α, where C is the current consecutive frame number, β is the adjustment coefficient, and α is the weighting factor. default As the default weighting factor, α max This is a preset upper limit value; (b) Recalculate the behavior recognition state based on the enhanced weighted factors; An alarm generation module is used to generate alarm information based on the decision output and output the alarm information to the alarm system. The report generation module receives manual review input from the review terminal, where the manual review input is feedback for correcting the alarm information. Based on the manual review input and the prediction results, the module performs statistical analysis to generate a statistical report for iterative optimization of the detection model. The specific process of the statistical analysis includes: Collect the manually reviewed inputs and corresponding prediction results, and calculate the prediction accuracy and false alarm rate. The statistical report is generated based on the indicators, and the statistical report is used to adjust the training parameters of the detection model or update the business rules. The statistical report includes model confidence calibration parameters, which are generated through the following steps: (1) Calculate the deviation δ between the manual review result and the prediction result, defined as follows: Let y i human Let y be the probability value of the violation in the manual review of the i-th sample. i pred To predict the probability of violation for the i-th sample, then: Where K is the number of re-judged samples; (2) When δ>γ, γ is the calibration threshold, and the confidence score output by the detection model is compressed and transformed: Among them, Score raw λ represents the original confidence score output by the model, and λ is the calibration intensity coefficient.

6. An electronic device, characterized in that, include: A memory and a processor, wherein the processor and the memory are connected; The memory is used to store programs; The processor invokes a program stored in the memory to execute the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer program, which is executed by a computer to perform the method as described in any one of claims 1-4.

Images