Tank car operation behavior analysis method based on improved YOLO network for semiconductor factory service system

Abstract

The invention provides a tank car operation behavior analysis method based on an improved YOLO network for a semiconductor factory service system, and belongs to the field of semiconductor manufacturing, and the method specifically comprises the steps: collecting a real-time image of a tank car operation region; marking a target area corresponding to multiple view angles of each detection target in the real-time image through an improved YOLO network; generating a real-time behavior sequence according to the target area of each detection target marked by the real-time image; judging whether the real-time behavior sequence conforms to a specification or not; judging whether the operation behavior of the tank car occurs at a specified position or not by combining a mapping relation between the marked target area of each detection target and the area position of the shooting scene; and outputting a real-time analysis result corresponding to the tank car operation behavior according to the real-time behavior sequence and a judgment result of the tank car operation behavior. Through the processing scheme provided by the invention, all-around intelligent supervision of tank car operation from physical operation to flow compliance is realized, and the safety and management efficiency of a high-risk operation scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of semiconductor manufacturing, in particular to a tank truck operation behavior analysis method based on an improved YOLO network for a semiconductor plant management system. BACKGROUND

[0002] In a semiconductor smart plant management system, tank truck operation as a high-risk chemical transfer link has very high requirements for operation safety. Traditional tank truck safety monitoring mainly relies on manual video patrol or simple motion sensing devices. These methods have obvious defects: 1. Manual monitoring is prone to fatigue and omission, and does not have the ability to continuously monitor high-frequency operations or night operations; 2. Sensor monitoring is difficult to cover complex operation details, such as whether the equipment is placed in place, whether the key is inserted, whether the operation sequence is standardized, etc., and a single point cannot form a complete closed loop.

[0003] In recent years, target detection algorithms (such as the YOLO series) have been gradually applied to industrial scenarios to achieve intelligent identification of operation behavior. However, existing solutions generally have the following problems: 1. Inaccurate detection of small targets (such as keys, hand movements, etc.), which is prone to missed detection; 2. Lack of comprehensive judgment ability for operation process compliance, making it difficult to achieve multi-index collaborative analysis; 3. High false positive rate, increasing the response burden of monitoring personnel.

[0004] Therefore, there is an urgent need for a more intelligent and accurate solution to achieve all-weather, closed-loop intelligent monitoring of tank truck operation behavior, thereby reducing the risk of safety accidents. SUMMARY

[0005] Therefore, in order to overcome the shortcomings of the prior art, the semiconductor plant management system based on the improved YOLO network for the tank truck operation behavior analysis method, device, computer equipment and storage medium provided by the present application are deployed in an all-weather video monitoring network to form a full-process closed-loop monitoring system of "image collection-target detection-behavior analysis-rule judgment-alarm response", achieving all-round intelligent supervision of tank truck operation from physical operation to process compliance, and significantly improving the safety and management efficiency of high-risk operation scenarios.

[0006] In order to achieve the above object, the application provides a method for analyzing tank truck operation behavior based on an improved YOLO network in a semiconductor plant management system, comprising: collecting real-time images of a tank truck operation area through a high-definition video monitoring camera; analyzing each real-time image through an improved YOLO network and marking a target area corresponding to each detection target multi-view in the real-time image, wherein the improved YOLO network is generated by introducing a lightweight attention mechanism into a YOLO network, and the lightweight attention mechanism performs enhancement processing on a spatial area corresponding to each detection target in the real-time image; generating a real-time behavior sequence according to the target area corresponding to each detection target multi-view marked in the real-time image; judging whether the real-time behavior sequence conforms to a standard based on a multi-target time sequence recognition logic constructed based on a standard operation specification; judging whether the tank truck operation behavior occurs at a specified position in combination with a region position mapping relationship between the target area corresponding to each detection target multi-view and a shooting scene; and outputting a real-time analysis result corresponding to the tank truck operation behavior according to the real-time behavior sequence and the judgment result of the tank truck operation behavior.

[0007] In some embodiments, the method for constructing the improved YOLO network comprises: constructing a lightweight attention mechanism for performing global average pooling and maximum pooling on an image, generating channel weights, and then generating a spatial attention map through a convolution operation to output a spatial area corresponding to each detection target after weighting; in a feature extraction path of a YOLO network, selecting a plurality of key feature layers as embedding points of the lightweight attention mechanism; inputting an image at a position corresponding to the embedding point in the YOLO network to the lightweight attention mechanism, and continuing to pass the image weighted by the lightweight attention mechanism to subsequent network layers in the YOLO network to output a spatial area corresponding to each detection target after weighting.

[0008] In some embodiments, the lightweight attention mechanism comprises: performing global average pooling on each channel of a feature map of each detection target to obtain global features of each channel; performing global maximum pooling on each channel of the feature map of each detection target to obtain maximum features of each channel; concatenating the global features and the maximum features, and generating enhancement weights of each channel through a shared multi-layer perception; enhancing important channel features corresponding to the detection target in the real-time image through the enhancement weights; performing average and maximum pooling in the channel dimension, generating a spatial attention map through a convolution operation after concatenation, and highlighting a spatial area corresponding to the detection target.

[0009] In some embodiments, the improved YOLO network training method comprises: learning and training feature maps of each detection target in the data set, and analyzing the real boundary box in the training data through a clustering algorithm to automatically determine the anchor box size and proportion most suitable for the data set; inputting the feature maps of each detection target corresponding to the embedding point position in the YOLO network to the lightweight attention mechanism, and continuing to pass the feature maps of each detection target weighted processed by the lightweight attention mechanism to the subsequent network layer in the YOLO network to output the spatial region corresponding to each detection target after weighting.

[0010] In some embodiments, the real-time behavior sequence is generated according to the target region corresponding to the multi-view of each detection target marked in the real-time image, comprising: sorting the real-time image according to the real-time time and real-time camera position information corresponding to the real-time image to obtain an image generation order; extracting the target region corresponding to each detection target of the real-time image according to the image generation order, and determining each operation event corresponding to the detection target; sorting the operation events according to the real-time time to generate a real-time behavior sequence.

[0011] In some embodiments, the generation method of the multi-target time sequence identification logic comprises: splitting the standard operation specification into a plurality of sequentially occurring behavior events through a semantic analysis method to obtain a behavior event sequence; splitting the behavior events into a plurality of ordered state transition rules; and generating the multi-target time sequence identification logic according to the behavior event sequence and the state transition rule.

[0012] In some embodiments, the real-time analysis result is further sent to a plant management system to realize detection log storage and abnormal feedback archiving.

[0013] The device comprises a collection module for collecting real-time images of a tank truck operation area through a high-definition video monitoring camera; a region analysis module for analyzing each real-time image through an improved YOLO network and marking target regions corresponding to each detection target multi-angle in the real-time image, wherein the improved YOLO network is generated by introducing a lightweight attention mechanism into the YOLO network, and the lightweight attention mechanism performs enhancement processing on the spatial region corresponding to each detection target in the real-time image; an operation sequence generation module for generating a real-time behavior sequence according to the target regions corresponding to each detection target multi-angle marked in the real-time image; an event judgment module for judging whether the real-time behavior sequence conforms to the standard based on multi-target time sequence recognition logic constructed based on the standard operation specification; a position judgment module for judging whether the tank truck operation behavior occurs at a specified position in combination with the region position mapping relationship between the target regions corresponding to each detection target multi-angle and the shooting scene; and an analysis result output module for outputting a real-time analysis result corresponding to the tank truck operation behavior according to the real-time behavior sequence and the judgment result of the tank truck operation behavior.

[0014] A computer device comprises a memory and a processor, and the memory stores a computer program, wherein the processor implements the steps of the above method when executing the computer program.

[0015] A computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the above method.

[0016] Compared with the prior art, the advantages of the present application are that the enhanced YOLO network fused with the lightweight attention mechanism enhances the features of small targets such as hands and keys in the operation process, optimizes the target detection model in combination with multi-angle real-time images, analyzes the spatial regions of the enhanced detection targets, constructs a key action operation judgment mechanism and an operation process logic reasoning system, and realizes automatic, high-precision, interpretable compliance discrimination and abnormal behavior early warning for the whole process of tank truck loading and unloading operation. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1is a flowchart of a method for analyzing tank truck operation behavior based on an improved YOLO network by a semiconductor factory management system in an embodiment of the present application; Figure 2 is a structural block diagram of a device for analyzing tank truck operation behavior based on an improved YOLO network by a semiconductor factory management system in an embodiment of the present application; and Figure 3 is an internal structure diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present application will be described in detail below with reference to the drawings.

[0020] The above embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. The present application can be implemented or applied in other different specific embodiments, and each detail herein can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0021] It should be noted that the various aspects of the embodiments described below are within the scope of the present application. It should be apparent that the aspects described herein can be embodied in a wide variety of forms and that any specific structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art should appreciate that an aspect described herein can be implemented independently of any other aspects and that two or more aspects can be combined in various ways. For example, an apparatus can be implemented or a method can be practiced using any number of the aspects described herein. In addition, such an apparatus can be implemented or such a method can be practiced using other structure and / or functionality in addition to or other than one or more of the aspects described herein.

[0022] It should also be noted that the drawings included in the following embodiments are only schematically illustrating the basic concept of the present application, and only the components related to the present application are shown in the drawings, not the number, shape and size of the components when actually implemented. The actual implementation of each component can be a random change in shape, number and proportion, and the layout pattern of the components can be more complex.

[0023] Also in the following description, specific details are provided to facilitate thorough understanding of examples. However, one skilled in the relevant art will appreciate that the described aspects can be practiced without these specific details.

[0024] The embodiment of the present application provides a method for analyzing operation behavior of a tank car based on an improved YOLO network of a semiconductor plant system, which is applied to a server or a terminal. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable smart devices, and the server can be implemented by an independent server or a server cluster composed of multiple servers. As shown in the figure, the method for analyzing operation behavior of the tank car based on the improved YOLO network of the semiconductor plant system includes the following steps: Figure 1 As shown in the figure, the method for analyzing operation behavior of the tank car based on the improved YOLO network of the semiconductor plant system includes the following steps: Step 101, collecting real-time images of a tank car operation area through a high-definition video monitoring camera.

[0025] The server can collect real-time images of the tank car operation area from all-weather and multiple angles through the high-definition video monitoring camera. Multiple monitoring cameras can be arranged in the same operation area, so that different angles of the same position can be shot.

[0026] Step 102, analyzing each real-time image through the improved YOLO network, and marking target regions corresponding to each detection target in the real-time image from multiple angles. The improved YOLO network is generated by introducing a lightweight attention mechanism into the YOLO network, and the lightweight attention mechanism performs enhancement processing on the spatial region corresponding to each detection target in the real-time image.

[0027] The detection target can be a hand, a body of a worker, or a key, a valve or other components, or a vehicle. There can be multiple detection targets in the real-time image, and the improved YOLO network can mark all detection targets in the same real-time image.

[0028] YOLO (You Only Look Once) network regards the target detection task as a single regression problem, which directly maps from image pixels to bounding box coordinates and class probabilities. The YOLO network is a target detection model, which is mainly composed of three parts: backbone, neck and detection head. The main function of the backbone is to extract basic features from the input image. The function of the neck is to fuse and enhance the features extracted by the backbone, so as to better process targets of different scales. The function of the detection head is to classify and locate targets by using the fused features. The improved YOLO network is generated by introducing a lightweight attention mechanism into the YOLO network. The lightweight attention mechanism enhances the spatial regions corresponding to each detection target in real-time images. The lightweight attention mechanism can be set at the output end of the deep CSP module (such as the C3 module) in the backbone network or on the high-level feature maps (such as the P3, P4, and P5 layers) before multi-scale feature fusion in the neck network, to enhance the feature responses of detection targets in different real-time images.

[0029] The improved YOLO network is generated by introducing a lightweight attention mechanism into the YOLO network. The lightweight attention mechanism enhances the spatial regions corresponding to each detection target in real-time images. In an embodiment, the lightweight attention mechanism is a CBAM (Convolutional Block Attention Module) module. The CBAM module enhances the expression ability of the convolutional neural network (CNN) for key features by combining channel attention and spatial attention. The channel attention submodule assigns different weights to each channel in the feature map, thereby enhancing important channel features and suppressing unimportant channel features. The spatial attention submodule assigns different weights to each spatial position of the feature map, thereby enhancing important spatial regions and suppressing unimportant spatial regions. The server can enhance the detection region of small targets by improving the YOLO network, thereby increasing the recognition probability of small targets.

[0030] The server analyzes each real-time image by using the improved YOLO network, and marks the target regions corresponding to the multi-view of each detection target in the real-time image.

[0031] In step 103, real-time behavior sequences are generated according to the target regions corresponding to the multi-view of each detection target marked in the real-time image.

[0032] The server generates a real-time behavior sequence according to a target region corresponding to each detection target marked in the real-time image and a multi-view angle. There can be multiple detection targets in the same real-time image, and there can be multiple real-time images of different angles at the same time for the same detection target. The server can summarize the target regions of the same detection target at the same time, and then determine the behavior event corresponding to the detection target according to the time sequence, the summarized target region, and the state transition rule stored in advance. The server sorts the sequence of the behavior event according to the real-time time to obtain the real-time behavior sequence. The real-time image is used to shoot the behavior of the worker in the tank car in a specific region. The server can simultaneously determine the real-time behavior sequence occurring at the same time in different tank cars in the same scene, but the real-time behavior sequences of different tank cars are independent of each other and do not affect each other.

[0033] In step 104, the multi-target time sequence recognition logic constructed based on the standard operation specification is used to determine whether the real-time behavior sequence conforms to the specification.

[0034] The server determines whether the real-time behavior sequence conforms to the specification based on the multi-target time sequence recognition logic constructed based on the standard operation specification. The multi-target time sequence recognition logic is an operation process of the worker determined by the server in combination with the time sequence of the business rule. The server determines whether the tank car operation process corresponding to the real-time behavior sequence is consistent with the multi-target time sequence recognition logic, and generates a judgment result corresponding to the real-time behavior sequence. The server converts the standard operation procedure (SOP) into an ordered state transition rule library, combines the time constraints and action dependency relationship, and dynamically checks the operation sequence, time sequence compliance, and key step integrity. The server continuously compares the actual behavior sequence with the preset rule path, and once detects abnormal behaviors such as skipping, reverse sequence, omission, or timeout, triggers a multi-level alarm mechanism.

[0035] For example, the operation process of the worker in the multi-target time sequence recognition logic is: after the driver stops the car, the driver needs to stop at a specified position first, then put the triangular wood behind the wheel, and finally put the car key in the specified position, and the time sequence of the three operations cannot be incorrect. The three operations can be different behavior events detected by different cameras and then analyzed by the server. The server compares the behavior events in the real-time behavior sequence with the event sequence in the multi-target time sequence recognition logic. When the comparison result is consistent, the server determines that the real-time behavior sequence conforms to the specification; when the comparison result is inconsistent, the server determines that the real-time behavior sequence does not conform to the specification.

[0036] In one embodiment, the server obtains a detection confidence corresponding to each detection target marked, and the server comprehensively determines based on the detection confidence of the detection target and the event sequence in the multi-target time sequence recognition logic to suppress false positives and omissions, and generates an alarm combined with the event-level trigger logic.

[0037] Step 105, in combination with the target region corresponding to the multi-view of each detection target marked and the region position mapping relationship of the shooting scene, it is judged whether the tank truck operation behavior occurs at the specified position.

[0038] The region position mapping relationship is the scene position of each behavior event occurrence stored in advance, which is the mapping of behavior event and scene position. The server can determine the electronic fence of each detection target through the training of the scene image shot by the camera in advance, and combine the behavior event in the real-time behavior sequence in S104 to judge whether the position of each detection target is accurate.

[0039] The server combines the target region corresponding to the multi-view of each detection target marked and the region position mapping relationship of the shooting scene to judge whether the tank truck operation behavior occurs at the specified position. For example, the server can improve the YOLO network to detect the key targets such as operation personnel, tank truck, valve, ground wire and operation tool and their spatial position relationship in real time, and determine whether the operations such as "opening valve", "connecting pipeline" and "removing ground wire" occur.

[0040] Step 106, according to the real-time behavior sequence and the judgment result of the tank truck operation behavior, output the real-time analysis result corresponding to the tank truck operation behavior.

[0041] The server outputs the real-time analysis result corresponding to the tank truck operation behavior according to the real-time behavior sequence and the judgment result of the tank truck operation behavior.

[0042] The server can output the operation compliance judgment result in real time; it can also issue a high confidence alarm for abnormal behavior, and store the warning information and event video segment correspondingly.

[0043] The above method enhances the features of key parts (such as hands and keys) and other small targets in the operation process through the enhanced YOLO network with light attention mechanism, optimizes the target detection model combined with multi-angle real-time image, and then analyzes the spatial region of the enhanced detection target to construct the key action operation judgment mechanism and operation process logic reasoning system, realizes the automatic, high-precision and explainable compliance discrimination and abnormal behavior early warning for the whole process of tank truck loading and unloading operation.

[0044] In one embodiment, the method further comprises: sending the real-time analysis result to the plant management system to realize detection log storage and abnormal feedback archiving.

[0045] The server can connect the tank operation behavior analysis method and the plant management system, realize detection log storage and abnormal feedback archiving; and the server can also configure different alarm levels according to real-time analysis results, so as to provide the alarm window of the plant management system in time, assist the on-site personnel to respond and intervene in time and prevent accidents, and significantly reduce the personnel casualties and equipment damage in the tank operation.

[0046] In one embodiment, the method for improving the construction of the YOLO network comprises the following steps: A lightweight attention mechanism is constructed, which is used for global average pooling and maximum pooling of an image, generation of channel weights, generation of a spatial attention map through convolution operation, and output of spatial regions corresponding to each detection target after weighting; In the feature extraction path of the YOLO network, a plurality of key feature layers are selected as embedding points of the lightweight attention mechanism; The image at the position corresponding to the embedding point in the YOLO network is input to the lightweight attention mechanism, and the image processed by the lightweight attention mechanism is further transmitted to the subsequent network layer in the YOLO network, and the spatial regions corresponding to each detection target after weighting are output.

[0047] The server can construct a lightweight attention mechanism, which is used for global average pooling and maximum pooling of an image, generation of channel weights, generation of a spatial attention map through convolution operation, and output of spatial regions corresponding to each detection target after weighting.

[0048] The server can select several key feature layers in the feature extraction path of the YOLO network as embedding points of the lightweight attention mechanism. The server selects several key feature layers in the feature extraction path of the YOLO network as embedding points of the CBAM. The server can connect the input end of the lightweight attention mechanism with the output end of the deep CSP module (such as the C3 module) in the deep layer of the backbone network. The server can also connect the input end of the lightweight attention mechanism with the high-level feature map (such as the P3, P4, and P5 layers) before multi-scale feature fusion in the neck network. In an embodiment, the server preferentially deploys on a high-resolution feature layer (such as P3) used for small target detection to enhance fine-grained feature response. Taking the YOLO network as the YOLO v8 network and introducing the lightweight attention mechanism (CBAM) on the YOLO v8 network architecture as an example, the server performs feature enhancement for the detection of key body parts (such as hands and keys) and other small target detection problems in the job process. Specifically, the CBAM module is selectively embedded after the deep C2f module in the backbone network of YOLOv8. The processing logic is as follows: first, the input feature map is subjected to global average pooling and maximum pooling in the channel dimension, respectively, and the channel attention weight is generated through a shared multi-layer perception to achieve adaptive enhancement of the key feature channels; then, average and maximum pooling are performed along the channel direction in the spatial dimension, and the spatial attention map is generated through convolution operation after splicing, focusing on the key operation area in the image. This attention mechanism is integrated in a plug-and-play manner without changing the original detection head structure and loss function, significantly improving the detection sensitivity and positioning accuracy of low-resolution, occluded, or distant targets, while keeping the model lightweight to meet the real-time inference requirements of edge devices.

[0049] The server can input the image at the position corresponding to the embedding point in the YOLO network to the lightweight attention mechanism, and continue to pass the image processed by the lightweight attention mechanism to the subsequent network layer in the YOLO network to output the spatial region corresponding to each detection target after weighting.

[0050] The server inputs the feature map at the selected position in the initial YOLO network to the lightweight attention mechanism; the feature map processed by the lightweight attention mechanism is continuously passed to the subsequent network layer; the detection head, loss function, anchor box design, and prediction logic in the initial YOLO network remain unchanged; the server introduces the lightweight attention mechanism into the YOLO network in a "plug-and-play" manner without reconstructing the backbone of the YOLO network or modifying the output structure.

[0051] The above method integrates the YOLO detection structure and the attention mechanism to improve the detection accuracy of small targets.

[0052] In an embodiment, the lightweight attention mechanism includes: The global average pooling is performed on each channel of the feature map of each detection target to obtain a global feature of each channel; The global maximum pooling is performed on each channel of the feature map of each detection target to obtain a maximum feature of each channel; The global feature and the maximum feature are spliced, and an enhanced weight of each channel is generated through a shared multi-layer perception; through the enhanced weight, important channel features corresponding to the detection target in the real-time image are enhanced; The average and maximum pooling are performed on the channel dimension, and a spatial attention map is generated through convolution operation after splicing, so as to highlight the spatial region corresponding to the detection target.

[0053] The lightweight attention mechanism is an attention mechanism module combining channel attention and spatial attention, and is used to enhance the perception ability of a convolutional neural network to key features. The lightweight attention mechanism includes two processing contents. One is to generate channel weights through a shared multi-layer perception by performing global average pooling and maximum pooling on the input feature map respectively, so as to enhance important feature channels. The other is to generate a spatial attention map through convolution operation after splicing, so as to highlight key spatial regions, by performing average and maximum pooling on the channel dimension. The server connects the two processing contents of the lightweight attention mechanism in series, and processes the feature map gradually from channel to space, and outputs the weighted feature map.

[0054] The server can perform global average pooling on each channel of the feature map to obtain a global feature of each channel. The server can also perform global maximum pooling on each channel of the feature map to obtain a maximum feature of each channel. Then the server splices the results of the global average pooling and the global maximum pooling, and outputs through a shared multi-layer perception (MLP), and generates a weight of each channel through a Sigmoid function. The server can multiply each channel weight with the real-time image to enhance important channel features.

[0055] The server models the importance of each spatial position of the feature map, enhances important spatial regions, and suppresses unimportant spatial regions. The server performs average pooling and maximum pooling on each channel of the feature map to obtain two channel-aggregated feature maps; splices the two channel-aggregated feature maps together to generate a spatial attention map through a convolution layer; and obtains the spatial attention map through a Sigmoid function of the output of the convolution layer. The server multiplies the spatial attention map with the real-time image to enhance important spatial regions.

[0056] The server enhances the important channel features in the real-time image and enhances the important spatial regions in the real-time image, and finally realizes the enhancement of the target regions of each detection target in the real-time image.

[0057] In one embodiment, the training method of the YOLO network is improved, comprising: The feature maps of each detection target in the data set are learned and trained, and the real boundary boxes in the training data are analyzed by a clustering algorithm to automatically determine the anchor box size and proportion most suitable for the data set; The feature maps of each detection target corresponding to the embedding point position in the YOLO network are input to the lightweight attention mechanism, and the feature maps of each detection target after weighted processing by the lightweight attention mechanism are further transmitted to the subsequent network layer in the YOLO network, and the spatial regions corresponding to each detection target after weighting are output.

[0058] During the training process, the server analyzes the real boundary boxes in the training data by a clustering algorithm (such as K-means), dynamically adjusts the parameters of the anchor box according to the difference between the predicted box and the real box, and automatically determines the anchor box size and proportion most suitable for the current data set. The server adds an adaptive anchor box generation mechanism, and data statistics-cluster calculation-dynamic allocation realize the individualized adaptation of the anchor box size, and improve the generalization ability on devices of different sizes.

[0059] The server inputs the feature maps of each detection target corresponding to the embedding point position in the YOLO network to the lightweight attention mechanism, and further transmits the feature maps of each detection target after weighted processing by the lightweight attention mechanism to the subsequent network layer in the YOLO network, and outputs the spatial regions corresponding to each detection target after weighting.

[0060] The above method adds an adaptive anchor box generation mechanism to realize the individualized adaptation of the anchor box size and improve the generalization ability of the YOLO network on devices of different sizes.

[0061] In one embodiment, the real-time behavior sequence is generated according to the target regions corresponding to the multi-view of each detection target marked in the real-time image, comprising: According to the real-time time and real-time camera position information corresponding to the real-time image, the real-time image is sorted to obtain an image generation order; According to the image generation order, the target regions corresponding to each detection target in the real-time image are extracted, and each operation event corresponding to the detection target is determined; The operation events are sorted according to the real-time time to generate a real-time behavior sequence.

[0062] The server sorts the real-time images according to real-time time and real-time camera position information corresponding to the real-time images, to obtain an image generation order. The server uses multiple cameras to capture the same scene position, and the server will sort the image generation order of the real-time images captured by each camera according to real-time time and real-time camera position information. The same scene position stores a plurality of corresponding image generation orders.

[0063] The server extracts a target region corresponding to each detection target of the real-time images according to the image generation order, and determines each operation event corresponding to the detection target. The server extracts a target region corresponding to each detection target of the real-time images according to each image generation order, and determines each operation event corresponding to the detection target. When the operation events determined by the multiple image generation orders in the same scene position are consistent, the server determines that the image generation order corresponds to the operation event, otherwise, the server will generate inconsistent results, and send the real-time image to the verification terminal for secondary determination, and receive the operation event determined by the verification terminal.

[0064] The server sorts the operation events according to real-time time to generate a real-time behavior sequence.

[0065] The above method verifies multiple factors of the same frame image by combining multi-target discrimination and operation timing logic judgment, reduces false positives and omissions, makes the alarm result more reliable, reduces invalid intervention, and improves the alarm accuracy.

[0066] In one embodiment, a method for generating multi-target timing recognition logic includes: Splitting a standard operating procedure into a plurality of sequentially occurring behavior events by a semantic analysis method to obtain a behavior event sequence; Splitting the behavior event into a plurality of ordered state transition rules; Generating multi-target timing recognition logic according to the behavior event sequence and the state transition rule.

[0067] The server splits a standard operating procedure (SOP specification) into a plurality of sequentially occurring behavior events by a semantic analysis method to obtain a behavior event sequence. The standard operating procedure (SOP specification) is a written guidance document used to standardize the execution steps of a specific task or process, to ensure the consistency and quality of the operation.

[0068] The server splits the behavior event into multiple ordered state transition rules. For example, the behavior event is "put the car key in the designated position", the server will sequentially split the behavior event into multiple state transition rules such as "hand holding the car key", "hand putting the car key in the designated position", "the key is in the designated position and the hand leaves the designated position", etc. The server splits the behavior event into a specific number and type of state transition rules. The server uses a deep learning model to analyze multiple video data, automatically identifies and labels different independent behaviors; and then the server verifies the split independent behaviors to ensure the accuracy and integrity of each independent behavior.

[0069] The server generates a multi-target time sequence recognition logic according to the behavior event sequence and the state transition rule.

[0070] The above method combines business rules to detect the operation process of the worker in time sequence, judges whether the tank truck operation process meets the specification, and realizes a multi-target time sequence combination logic judgment mechanism.

[0071] In one embodiment, as shown in Figure 2 A device for analyzing tank truck operation behavior based on an improved YOLO network in a semiconductor plant management system is provided, which includes a collection module 201, a region analysis module 202, an operation sequence generation module 203, an event judgment module 204, a position judgment module 205, and an analysis result output module 206.

[0072] The collection module 201 is configured to collect real-time images of a tank truck operation area through a high-definition video monitoring camera.

[0073] The region analysis module 202 is configured to analyze each real-time image through an improved YOLO network and mark the target regions corresponding to each detection target multi-view in the real-time image. The improved YOLO network is generated by introducing a lightweight attention mechanism into the YOLO network, and the lightweight attention mechanism performs enhancement processing on the spatial regions corresponding to each detection target in the real-time image.

[0074] The operation sequence generation module 203 is configured to generate a real-time behavior sequence according to the target regions corresponding to each detection target multi-view that are marked in the real-time image.

[0075] The event judgment module 204 is configured to judge whether the real-time behavior sequence meets the specification based on a multi-target time sequence recognition logic constructed based on a standard operation specification.

[0076] The position judgment module 205 is configured to judge whether the tank truck operation behavior occurs at a specified position in combination with the region position mapping relationship between the target regions corresponding to each detection target multi-view that are marked and the shooting scene.

[0077] The analysis result output module 206 is configured to output real-time analysis results corresponding to the tank truck operation behavior according to the real-time behavior sequence and the judgment result of the tank truck operation behavior.

[0078] The specific limitations of the device for analyzing the tank truck operation behavior of the semiconductor plant management system based on the improved YOLO network can refer to the limitations of the method for analyzing the tank truck operation behavior of the semiconductor plant management system based on the improved YOLO network in the foregoing, which will not be repeated here. Each module in the device for analyzing the tank truck operation behavior of the semiconductor plant management system based on the improved YOLO network can be realized by software, hardware, and combinations thereof, in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0079] In one embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 3 The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store data such as standard operation specifications. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a method for analyzing the tank truck operation behavior of a semiconductor plant management system based on an improved YOLO network.

[0080] Those skilled in the art can understand that Figure 3 The structure shown in the foregoing

[0081] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program: collecting real-time images of a tank car operation area from all-weather and multi-angle through a high-definition video monitoring camera; analyzing each real-time image through an improved YOLO network, and marking target regions corresponding to multi-angle views of each detection target in the real-time image, the improved YOLO network being generated by introducing a lightweight attention mechanism into the YOLO network, the lightweight attention mechanism performing enhancement processing on spatial regions corresponding to each detection target in the real-time image; generating a real-time behavior sequence according to the target regions corresponding to multi-angle views of each detection target marked in the real-time image; determining whether the real-time behavior sequence conforms to a standard based on multi-target time sequence recognition logic constructed based on a standard operation specification; determining whether a tank car operation behavior occurs at a specified position based on a region position mapping relationship between the target regions corresponding to multi-angle views of each detection target marked and a shooting scene; and outputting a real-time analysis result corresponding to the tank car operation behavior according to the determination results of the real-time behavior sequence and the tank car operation behavior.

[0082] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program: collecting real-time images of a tank car operation area from all-weather and multi-angle through a high-definition video monitoring camera; analyzing each real-time image through an improved YOLO network, and marking target regions corresponding to multi-angle views of each detection target in the real-time image, the improved YOLO network being generated by introducing a lightweight attention mechanism into the YOLO network, the lightweight attention mechanism performing enhancement processing on spatial regions corresponding to each detection target in the real-time image; generating a real-time behavior sequence according to the target regions corresponding to multi-angle views of each detection target marked in the real-time image; determining whether the real-time behavior sequence conforms to a standard based on multi-target time sequence recognition logic constructed based on a standard operation specification; determining whether a tank car operation behavior occurs at a specified position based on a region position mapping relationship between the target regions corresponding to multi-angle views of each detection target marked and a shooting scene; and outputting a real-time analysis result corresponding to the tank car operation behavior according to the determination results of the real-time behavior sequence and the tank car operation behavior.

[0083] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which shall be covered within the protection scope of the present application.

Claims

1. A method for analyzing tank car operation behavior based on an improved YOLO network in a semiconductor plant management system, characterized in that, The method comprises the following steps: Collecting real-time images of the tank car operation area through a high-definition video monitoring camera; Analyzing each real-time image through an improved YOLO network, and marking the target area corresponding to each detection target multi-view in the real-time image, wherein the improved YOLO network is generated by introducing a lightweight attention mechanism into the YOLO network, and the lightweight attention mechanism performs enhancement processing on the spatial area corresponding to each detection target in the real-time image; Generating a real-time behavior sequence according to the target area corresponding to each detection target multi-view marked in the real-time image; Judging whether the real-time behavior sequence conforms to the standard through a multi-target time sequence recognition logic constructed based on a standard operation specification; Judging whether the tank car operation behavior occurs at a specified position in combination with the region position mapping relationship between the target area corresponding to each detection target multi-view and the shooting scene; Outputting a real-time analysis result corresponding to the tank car operation behavior according to the judgment result of the real-time behavior sequence and the tank car operation behavior.

2. The method of claim 1, wherein, The construction method of the improved YOLO network comprises the following steps: Constructing a lightweight attention mechanism, which is used for global average pooling and maximum pooling of an image, generating channel weights, and then generating a spatial attention map through convolution operation to output the spatial area corresponding to each detection target after weighting; In the feature extraction path of the YOLO network, a plurality of key feature layers are selected as embedding points of the lightweight attention mechanism; The image at the position corresponding to the embedding point in the YOLO network is input to the lightweight attention mechanism, and the image processed by the lightweight attention mechanism is further transmitted to the subsequent network layer in the YOLO network to output the spatial area corresponding to each detection target after weighting.

3. The method of claim 2, wherein, The lightweight attention mechanism comprises the following steps: Global average pooling is performed on each channel of the feature map of each detection target to obtain the global feature of each channel; Global maximum pooling is performed on each channel of the feature map of each detection target to obtain the maximum feature of each channel; After the global feature and the maximum feature are spliced, a shared multi-layer perception machine is used to generate an enhanced weight of each channel; through the enhanced weight, important channel features corresponding to the detection target in the real-time image are enhanced; Average and maximum pooling are performed in the channel dimension, and a spatial attention map is generated through convolution operation after splicing, so as to highlight the spatial area corresponding to the detection target.

4. The method of claim 2, wherein, The training method of the improved YOLO network comprises the following steps: Learning and training the feature map of each detection target in the data set, and analyzing the real boundary box in the training data through a clustering algorithm to automatically determine the anchor box size and proportion most suitable for the data set; The feature map of each detection target at the position corresponding to the embedding point in the YOLO network is input to the lightweight attention mechanism, and the feature map of each detection target processed by the lightweight attention mechanism is further transmitted to the subsequent network layer in the YOLO network to output the spatial area corresponding to each detection target after weighting.

5. The method of claim 1, wherein, The real-time behavior sequence is generated according to the target region corresponding to each detection target in multiple perspectives of the real-time image, and the real-time behavior sequence includes: The real-time image is sorted according to real-time time and real-time camera position information corresponding to the real-time image, and an image generation order is obtained; According to the image generation order, the target region corresponding to each detection target of the real-time image is extracted, and each operation event corresponding to the detection target is determined; The operation events are sorted according to the real-time time, and a real-time behavior sequence is generated.

6. The method of claim 1, wherein, The generation method of the multi-target time sequence identification logic includes: The standard operation specification is split into a plurality of sequentially occurring behavior events by a semantic analysis method, and a behavior event sequence is obtained; The behavior events are split into a plurality of ordered state transition rules; The multi-target time sequence identification logic is generated according to the behavior event sequence and the state transition rule.

7. The method of claim 1, wherein, Further comprising: The real-time analysis result is sent to a plant management system, and detection log storage and abnormal feedback archiving are realized.

8. A device for analyzing the operation behavior of a tank car based on an improved YOLO network in a semiconductor plant management system, characterized in that, The device includes: The acquisition module is configured to acquire real-time images of a tank car operation area through a high-definition video monitoring camera; The region analysis module is configured to analyze each real-time image through an improved YOLO network, and mark the target region corresponding to each detection target in multiple perspectives of the real-time image, wherein the improved YOLO network is generated by introducing a lightweight attention mechanism into the YOLO network, and the lightweight attention mechanism performs enhancement processing on the spatial region corresponding to each detection target in the real-time image; The operation sequence generation module is configured to generate a real-time behavior sequence according to the target region corresponding to each detection target in multiple perspectives of the real-time image; The event judgment module is configured to judge whether the real-time behavior sequence conforms to the specification based on the multi-target time sequence identification logic constructed based on the standard operation specification; The position judgment module is configured to judge whether the tank car operation behavior occurs at a specified position in combination with a region position mapping relationship between the target region corresponding to each detection target in multiple perspectives and a shooting scene; The analysis result output module is configured to output a real-time analysis result corresponding to the tank car operation behavior according to the real-time behavior sequence and the judgment result of the tank car operation behavior. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to realize the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method of any one of claims 1 to 7.

Images