Early warning method for illegal operations in petroleum and petrochemical hazardous areas based on deep learning

Abstract

This invention provides a deep learning-based early warning method for illegal operations in hazardous petroleum and petrochemical areas. This method, which relates to the field of petroleum and petrochemical technology, involves detecting and tracking target image sequences, establishing target time-series trajectories, constructing multi-target interactive feature maps, dynamically adjusting weights based on environmental parameters, calculating safety situation scores, and using a dynamic inference graph network for behavior recognition to achieve real-time safety monitoring of petrochemical areas. This method can effectively identify illegal operations, improve the accuracy of safety warnings, and reduce accident risks.

Description

Technical Field

[0001] The present invention relates to the field of petroleum and petrochemical technology, and in particular to a deep learning-based early warning method for illegal operations in petroleum and petrochemical hazardous areas. Background Art

[0002] The petroleum and petrochemical industry, a key pillar of the national economy, involves a large number of flammable and explosive hazardous materials in its production processes, making production safety extremely important. In recent years, with the development of artificial intelligence (AI), computer vision-based safety monitoring technology has been widely used in petrochemical safety management. Traditional petrochemical safety monitoring relies primarily on manual inspections and passive monitoring using fixed cameras, which has limitations in efficiency and accuracy. With the rapid development of deep learning technology, intelligent monitoring systems based on video analysis are becoming an increasingly important tool for safety management in petrochemical areas.

[0003] At present, the monitoring of illegal operations in hazardous areas of petroleum and petrochemical industries mainly has the following problems: First, existing technologies mostly focus on the behavior recognition of a single target, lack effective analysis of the interactive behavior of multiple targets, and cannot accurately capture the complex interaction patterns between personnel and equipment, and between personnel, resulting in incomplete identification of potentially dangerous behaviors; Second, existing monitoring systems fail to fully consider the impact of special parameters of the petrochemical environment, such as the dynamic impact of environmental factors such as temperature, humidity, and gas concentration on safety risks, making early warning judgments lack environmental adaptability; Third, traditional behavior recognition algorithms mostly use static feature extraction methods, which cannot effectively model the dynamic correlation between targets that evolve over time, and the recognition accuracy of some complex illegal operations that require long-term observation is not high.

[0004] To solve the above problems, a warning method for illegal operations in petroleum and petrochemical hazardous areas is needed that can comprehensively consider multi-target interactions, the influence of environmental parameters, and has dynamic reasoning capabilities, so as to improve the intelligence level of safety management in petrochemical areas and the accuracy of warnings. Summary of the Invention

[0005] The embodiments of the present invention provide a deep learning-based early warning method for illegal operations in petroleum and petrochemical hazardous areas, which can solve the problems in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a deep learning-based early warning method for illegal operations in petroleum and petrochemical hazardous areas, comprising:

[0007] Obtaining a target image sequence corresponding to video data of a petrochemical area, performing target detection and tracking on the target image sequence, obtaining position coordinates of a target object in the target image sequence, and establishing a target index;

[0008] Based on the target index, a time series trajectory is established for each target object in the target image sequence; a multi-target interaction feature map is constructed according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed and relative direction information between target objects;

[0009] Dynamically weighting the multi-objective interactive feature graph in combination with environmental parameters of the petrochemical area to obtain an environmentally aware interactive feature graph; calculating a security situation score for the petrochemical area based on the environmentally aware interactive feature graph, wherein the security situation score represents the overall security status of the petrochemical area at the current moment;

[0010] A dynamic reasoning graph network is constructed for behavior recognition. The interaction feature graph of the environmental perception is input into the dynamic reasoning graph network. Node features are extracted through graph convolution operations. The dynamic association relationship between nodes is updated based on the message passing mechanism to obtain the recognition result of the target behavior.

[0011] Performing target detection and tracking on the target image sequence, obtaining the position coordinates of the target object in the target image sequence, and establishing a target index includes:

[0012] Detecting the target image sequence using a target detection model, obtaining target objects in the target image sequence, and extracting position coordinates, size information, and category information of the target objects;

[0013] Assigning a globally unique identifier to each target object and establishing a target index table, wherein the target index table records the identifier, position coordinates, size information, category information, and timestamp of the target object; establishing a temporal association of the target objects by comparing the visual features and motion features of the target objects between adjacent frames; and associating target objects with the same visual features and continuous motion trajectories with the same identifier;

[0014] The motion parameters of the target object are calculated based on the position coordinates recorded in the target index table.

[0015] Based on the target index, a time series trajectory is established for each target object in the target image sequence; and a multi-target interaction feature map is constructed according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed, and relative direction information between target objects.

[0016] According to the identification code of the target object, the position coordinates of the target object are recorded in chronological order to obtain a time series coordinate sequence of the target object; the displacement and time difference of adjacent coordinate points in the time series coordinate sequence are calculated to obtain the movement speed and movement direction of the target object at different moments; and the position coordinates, the movement speed, and the movement direction are combined to form a time series trajectory of the target object;

[0017] The relative distance, relative speed and relative direction between target objects are calculated to construct a multi-target interaction feature map; the relative distance, relative speed and relative direction are converted into an interaction strength score according to preset distance weight, speed weight and direction weight; when the interaction strength score is greater than a first preset threshold, the corresponding target objects are divided into the same interaction group; the overlap of the motion trajectories of the target objects in the interaction group is calculated, and when the overlap of the motion trajectories is greater than a second preset threshold, the interaction group is determined to be a collaborative target group.

[0018] The method further comprises:

[0019] Calculate the geometric center coordinates and movement direction of the group; calculate the average distance between all target objects in the group and the geometric center to obtain the spatial distribution radius of the group; identify the dominant object in the group, the dominant object being the target object closest to the group's geometric center and with the smallest deviation from the group's movement direction; calculate the trajectory similarity between other target objects and the dominant object, the trajectory similarity being equal to the inverse of the difference in movement direction between the target object and the dominant object; and calculate the group's stability coefficient based on the trajectory similarity.

[0020] When the stability coefficient of the group is greater than the third preset threshold, the group is marked as a stable collaborative group; when the spatial distribution radius of the group is greater than the fourth preset threshold or the trajectory similarity of the target object is less than the fifth preset threshold, a group disbanding warning is triggered.

[0021] Dynamically weighting the multi-objective interactive feature graph in combination with environmental parameters of the petrochemical area to obtain an environmentally-aware interactive feature graph; and calculating a security situation score for the petrochemical area based on the environmentally-aware interactive feature graph includes:

[0022] Acquiring environmental parameters of the petrochemical area, wherein the environmental parameters include temperature values ​​of multiple temperature monitoring points and gas concentration values ​​of multiple gas monitoring points;

[0023] A temperature field influence factor is constructed based on the temperature value of the temperature monitoring point, and the temperature field influence factor is obtained by calculating the ratio of the distance from the target to the temperature monitoring point to the temperature field attenuation coefficient, combined with the normalized temperature value of the temperature monitoring point; a gas concentration influence factor is constructed based on the gas concentration value of the gas monitoring point, and the gas concentration influence factor is obtained by calculating the ratio of the height difference from the target to the gas monitoring point to the gas diffusion coefficient, combined with the normalized concentration value of the gas monitoring point;

[0024] Acquire a multi-target interaction feature map, the multi-target interaction feature map including distance features and speed features between targets; adjust the distance features based on the temperature field influencing factor and the gas concentration influencing factor to obtain an environmentally perceived distance feature; and adjust the speed features based on spatial gradients of the temperature field influencing factor and the gas concentration influencing factor to obtain an environmentally perceived speed feature;

[0025] Performing feature fusion on the environment-perceived distance feature and the environment-perceived speed feature to establish an environment-perceived interaction feature map; calculating the target aggregation based on the environment-perceived interaction feature map, wherein the target aggregation is obtained by measuring the similarity between targets;

[0026] The petrochemical area is divided into multiple sub-areas, and the regional hazard level is calculated based on the impact intensity of the hazard sources in each sub-area. The safety situation score is obtained by combining the concentration of the target and the regional hazard level through nonlinear mapping.

[0027] A dynamic reasoning graph network is constructed for behavior recognition. The interactive feature graph of environmental perception is input into the dynamic reasoning graph network. Node features are extracted through graph convolution operations. The dynamic association relationship between nodes is updated based on the message passing mechanism. The recognition results of the target behavior are obtained, including:

[0028] Acquire the position features, speed features, and environmental features of the target, construct the position features, speed features, and environmental features into node feature vectors, construct edge feature vectors based on the association relationship between the node feature vectors; and form an initialization structure of a dynamic reasoning graph network with the node feature vectors and the edge feature vectors;

[0029] Building a temporal connection matrix based on the initialization structure of the dynamic reasoning graph network, and performing spatial feature aggregation on the node feature vectors according to the temporal connection matrix to obtain multi-layer node features;

[0030] Calculating attention weights for the multi-layer node features, where the attention weights are obtained by concatenating the node features and performing a nonlinear transformation; performing weighted aggregation on the multi-layer node features based on the attention weights to obtain enhanced node features;

[0031] Generate message features based on the enhanced node features and the edge feature vectors, wherein the message features include interaction information between nodes; aggregate the message features based on the attention weights, and update the node states through a recurrent neural network to obtain dynamic node features;

[0032] Calculating the temporal attention weight of the dynamic node feature, fusing the temporal attention weight with the historical feature matrix to obtain a temporal context feature; inputting the temporal context feature into the long short-term memory network, updating the memory state to obtain a fused feature;

[0033] Multi-scale feature extraction and feature pooling are performed on the fused features to obtain a behavior feature representation; a behavior classification score is calculated based on the behavior feature representation, and a recognition result of the target behavior is output.

[0034] A second aspect of an embodiment of the present invention provides a deep learning-based early warning system for illegal operations in petroleum and petrochemical hazardous areas, comprising:

[0035] The first unit is configured to obtain a target image sequence corresponding to video data of a petrochemical area, perform target detection and tracking on the target image sequence, obtain position coordinates of a target object in the target image sequence, and establish a target index;

[0036] The second unit is configured to establish a time series trajectory for each target object in the target image sequence based on the target index; construct a multi-target interaction feature map according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed, and relative direction information between target objects;

[0037] The third unit is configured to dynamically adjust the weights of the multi-objective interaction feature graph in combination with the environmental parameters of the petrochemical area to obtain an environmentally-aware interaction feature graph; and calculate a security situation score for the petrochemical area based on the environmentally-aware interaction feature graph, wherein the security situation score represents the overall security status of the petrochemical area at the current moment.

[0038] The fourth unit is used to construct a dynamic reasoning graph network for behavior recognition, input the interactive feature graph of environmental perception into the dynamic reasoning graph network, extract node features through graph convolution operation, and update the dynamic association relationship between nodes based on the message passing mechanism to obtain the recognition result of the target behavior.

[0039] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0040] processor;

[0041] a memory for storing processor-executable instructions;

[0042] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0043] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0044] The beneficial effects of this application are as follows:

[0045] The present invention uses a deep learning-based method to provide early warning of illegal operations in petroleum and petrochemical hazardous areas. It can monitor and analyze the behavior and interaction patterns of target objects such as personnel and equipment in hazardous areas in real time, thereby effectively preventing the occurrence of safety accidents.

[0046] The present invention dynamically adjusts the weights of multi-target interactive feature maps in combination with environmental parameters, enabling the system to flexibly adjust the warning sensitivity according to safety requirements under different environmental conditions, thereby improving the accuracy and adaptability of the warning and reducing false alarms and missed alarms.

[0047] The present invention adopts a dynamic reasoning graph network for behavior recognition, and captures the complex spatiotemporal relationships and interaction patterns between target objects through graph convolution operations and message passing mechanisms. It can accurately identify potential illegal operating behaviors and provide timely and effective early warning information to safety management personnel, greatly improving the safety management level and accident prevention capabilities of petrochemical areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a method for early warning of illegal operations in petroleum and petrochemical hazardous areas based on deep learning according to an embodiment of the present invention;

[0049] Figure 2 This is a bar chart comparing the performance of target detection and tracking technologies according to an embodiment of the present invention;

[0050] Figure 3 This is a flow chart of a group stability analysis method based on geometric center and trajectory similarity according to an embodiment of the present invention;

[0051] Figure 4 This is a bar chart comparing the performance of the petrochemical area security situation assessment method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0053] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0054] Figure 1 FIG. 1 is a flow chart of a method for early warning of illegal operations in petroleum and petrochemical hazardous areas based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0055] Obtaining a target image sequence corresponding to video data of a petrochemical area, performing target detection and tracking on the target image sequence, obtaining position coordinates of a target object in the target image sequence, and establishing a target index;

[0056] Based on the target index, a time series trajectory is established for each target object in the target image sequence; a multi-target interaction feature map is constructed according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed and relative direction information between target objects;

[0057] Dynamically weighting the multi-objective interactive feature graph in combination with environmental parameters of the petrochemical area to obtain an environmentally aware interactive feature graph; calculating a security situation score for the petrochemical area based on the environmentally aware interactive feature graph, wherein the security situation score represents the overall security status of the petrochemical area at the current moment;

[0058] A dynamic reasoning graph network is constructed for behavior recognition. The interaction feature graph of the environmental perception is input into the dynamic reasoning graph network. Node features are extracted through graph convolution operations. The dynamic association relationship between nodes is updated based on the message passing mechanism to obtain the recognition result of the target behavior.

[0059] In an optional embodiment, performing target detection and tracking on the target image sequence, obtaining position coordinates of the target object in the target image sequence, and establishing a target index includes:

[0060] Detecting the target image sequence using a target detection model, obtaining target objects in the target image sequence, and extracting position coordinates, size information, and category information of the target objects;

[0061] Assigning a globally unique identifier to each target object and establishing a target index table, wherein the target index table records the identifier, position coordinates, size information, category information, and timestamp of the target object; establishing a temporal association of the target objects by comparing the visual features and motion features of the target objects between adjacent frames; and associating target objects with the same visual features and continuous motion trajectories with the same identifier;

[0062] The motion parameters of the target object are calculated based on the position coordinates recorded in the target index table.

[0063] The system receives a sequence of target images consisting of multiple consecutive frames, such as a 30fps video stream with a resolution of 1920×1080 acquired from a video surveillance system. To detect objects in this image sequence, the system uses a pretrained deep learning object detection model, such as a convolutional neural network-based detection framework. This model has been trained on common object categories such as people, vehicles, and animals, and can identify objects with a confidence level greater than a preset threshold (e.g., 0.7).

[0064] During the actual detection process, the system performs object detection on each frame in the image sequence. Taking frame 1 as an example, the object detection model identifies three objects in this frame: a car and two pedestrians. For each detected object, the system extracts and records the following information: location coordinates (expressed as the coordinates of the top-left corner of the bounding box, e.g., (250, 350) for the car, (420, 380) for pedestrian 1, and (560, 400) for pedestrian 2), size information (expressed as bounding box width and height, e.g., (200×150 pixels) for the car, (80×180 pixels) for pedestrian 1, and (75×170 pixels) for pedestrian 2), and category information (e.g., "car," "pedestrian," etc.).

[0065] In order to achieve target tracking, the system needs to assign a globally unique identifier to each detected target object. In the first frame, the system assigns the identifier “V 001 ”, assign identifiers “P 001 ” and “P 002 At the same time, the system establishes a target index table to record the identifier, location coordinates, size information, category information and timestamp of each target object (the time information of the frame, such as "00:00:00.033" represents the 33rd millisecond of the video).

[0066] When processing the second frame, the object detection model also detects target objects, such as a car and two pedestrians. The system then determines whether these objects are identical to those in the first frame, essentially performing object association. The system establishes temporal association by comparing the visual and motion features of the target objects between adjacent frames.

[0067] Visual feature comparison is achieved by extracting the appearance features of the target area. The system extracts color histograms, texture features, or depth features from the target bounding box area and calculates the feature similarity between the two objects. For example, the feature similarity between the car at coordinates (255, 352) in the second frame and the car in the first frame is 0.92, exceeding the preset threshold of 0.8, and is preliminarily determined to be the same object.

[0068] Motion feature matching is achieved by predicting the target's trajectory. Based on the target's position and velocity in the previous frame, the system predicts the region where it will appear in the current frame. If the target detected in the current frame falls within this predicted region, its association probability is increased. For the aforementioned car target, based on its position (250, 350) in the first frame and its estimated velocity (5, 2) pixels / frame, the predicted position in the second frame is approximately (255, 352), which is very close to the actual detected position, further confirming that it is the same vehicle.

[0069] When both the visual feature similarity and the motion trajectory prediction meet the association conditions, the system associates the target of the current frame with the target of the previous frame and inherits its identifier. In this example, the car in the second frame continues to use the identifier "V 001 ", the two pedestrians continue to use the identifier "P 001 ” and “P 002 ”.

[0070] If a new target object is detected in a frame, the system will assign it a new unique identifier. For example, if a new pedestrian is detected in the 10th frame, the system will assign it the identifier "P 003 On the contrary, if a tracked target is not detected in multiple consecutive frames (such as 5 frames), it is considered that the target has left the monitoring area and its tracking status is marked as "left" in the target index table.

[0071] Based on the position coordinate data recorded in the target index table, the system calculates the motion parameters of the target object. 001 By analyzing the position changes of the car in 30 consecutive frames, we calculated that its average speed is 15 pixels / frame, the direction of movement is 26 degrees (relative to the horizontal axis), and the acceleration is 0.2 pixels / frame. 2 These motion parameters can be used to predict the target's future motion trajectory and improve tracking stability.

[0072] In order to handle the situation where the target is occluded, the system implements an occlusion handling mechanism. When the target is partially occluded, the detected bounding box will change. At this time, the system mainly relies on the visual features of the visible part and the previous motion trajectory to associate. For example, when "P 001 "When a pedestrian is partially obscured by roadside facilities, the system can still maintain correct tracking association based on its upper body features and predicted position.

[0073] When the target is completely occluded, the system continues to predict its trajectory based on previous motion parameters for a short period of time (e.g., 10 frames) and attempts to restore the association when the target reappears. This mechanism greatly improves the tracking robustness of the system in complex scenes.

[0074] Figure 2This is a bar chart comparing the performance of target detection and tracking technologies according to an embodiment of the present invention:

[0075] This figure shows a performance comparison of a technical solution with traditional methods and baseline algorithms across three key test metrics. The data in the figure shows that this technical solution achieved optimal performance across all test metrics: in terms of target detection accuracy, this solution achieved 53.8%, a significant improvement over the 47.2% of traditional methods and 43.5% of baseline algorithms; in terms of target tracking accuracy, this solution achieved 51.9%, exceeding the 47.6% of traditional methods and 39.4% of baseline algorithms; and in terms of temporal association accuracy, this solution achieved 56.3%, also outperforming the 45.1% of traditional methods and 42.2% of baseline algorithms. The data comparison across these three dimensions clearly shows that this technical solution has achieved comprehensive improvements over existing methods in core performance metrics such as target detection, tracking, and temporal association, with an average improvement of between 6 and 12 percentage points, fully demonstrating the advanced nature and practical value of this technical solution.

[0076] In an optional embodiment, based on the target index, a time series trajectory is established for each target object in the target image sequence; and a multi-target interaction feature map is constructed according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed, and relative direction information between target objects, including:

[0077] According to the identification code of the target object, the position coordinates of the target object are recorded in chronological order to obtain a time series coordinate sequence of the target object; the displacement and time difference of adjacent coordinate points in the time series coordinate sequence are calculated to obtain the movement speed and movement direction of the target object at different moments; and the position coordinates, the movement speed, and the movement direction are combined to form a time series trajectory of the target object;

[0078] The relative distance, relative speed and relative direction between target objects are calculated to construct a multi-target interaction feature map; the relative distance, relative speed and relative direction are converted into an interaction strength score according to preset distance weight, speed weight and direction weight; when the interaction strength score is greater than a first preset threshold, the corresponding target objects are divided into the same interaction group; the overlap of the motion trajectories of the target objects in the interaction group is calculated, and when the overlap of the motion trajectories is greater than a second preset threshold, the interaction group is determined to be a collaborative target group.

[0079] A temporal trajectory is established for each target object in the target image sequence based on the target index, and then a multi-target interaction feature map is constructed according to the temporal trajectory. The feature map contains the relative distance, relative speed and relative direction information between the target objects.

[0080] According to the identification code of the target object, the position coordinates of the target object are recorded in time sequence to obtain the time sequence coordinate sequence of the target object. 001 The position coordinates of the target object in 5 consecutive frames can be recorded as: (120,350), (125,355), (132,358), (140,362), (148,367). These coordinate points form the temporal coordinate sequence of the target object in chronological order.

[0081] The system calculates the displacement and time difference between adjacent coordinate points in a time-series coordinate sequence to determine the target object's speed and direction at different moments. Assuming the time interval between adjacent frames is 0.04 seconds, for the above coordinate sequence, the system calculates the displacement from the first frame to the second frame to be 7.07 pixels (calculated by the distance between the two points) and the time difference to be 0.04 seconds, resulting in a speed of 176.75 pixels / second at that moment. Simultaneously, by calculating the angle of the displacement vector, the direction of motion is determined to be 45 degrees. Similar calculations are performed for all adjacent frames to obtain a complete speed and direction sequence.

[0082] The position coordinates, movement speed and movement direction are combined into the time sequence trajectory of the target object. 001 The target object, its temporal trajectory can be expressed as: {(120,350,0,0), (125,355,176.75,45), (132,358,184.23,33), (140,362,206.16,38), (148,367,210.24,42)}, where each tuple contains coordinate, speed and direction information.

[0083] Calculate the relative distance, relative speed and relative direction between target objects to construct a multi-target interaction feature map. Assume there are two target object IDs in the scene 001 and ID 002 At a certain moment, their positions are (140, 362) and (200, 380), their speeds are 206.16 pixels / second and 186.42 pixels / second, and their directions are 38 degrees and 65 degrees. The system calculates the relative distance between them to be 68.59 pixels, the relative speed to be 89.74 pixels / second (calculated by the modulus of the velocity vector difference), and the relative direction to be 27 degrees (calculated by the difference between the two directions).

[0084] Based on the preset distance weight, speed weight, and direction weight, the relative distance, relative speed, and relative direction are converted into interaction strength scores. Assume that the system sets the distance weight to 0.5, the speed weight to 0.3, the direction weight to 0.2, the distance normalization coefficient to 100 pixels, the speed normalization coefficient to 200 pixels / second, and the direction normalization coefficient to 90 degrees. For the above example, the normalized relative distance value is 0.686, the relative speed value is 0.449, and the relative direction value is 0.3. The calculated interaction strength score is: 0.5×(1-0.686)+0.3×(1-0.449)+0.2×(1-0.3)=0.436.

[0085] When the interaction strength score is greater than the first preset threshold, the system divides the corresponding target objects into the same interaction group. Assuming that the first preset threshold is set to 0.4, the interaction strength score of the above two target objects is 0.436, which is greater than the threshold of 0.4, so they are divided into the same interaction group.

[0086] Calculate the overlap of the motion trajectories of the target objects in the interaction group. When the overlap of the motion trajectories is greater than a second preset threshold, the interaction group is determined to be a collaborative target group. The overlap of the motion trajectories is calculated by analyzing the similarity of the future predicted paths of the target objects. Specifically, the system predicts the position of each target in the next 5 frames based on its current position, speed and direction. 001 and ID 002 , system prediction ID 001 The positions of the next 5 frames are: (156,373), (164,379), (172,385), (180,391), (188,397); predicted ID 002 The positions of the next 5 frames are: (207,393), (214,406), (221,419), (228,432), (235,445).

[0087] Trajectory overlap is quantified by calculating the directional consistency and positional proximity of the predicted trajectories. This calculation includes the average directional difference of the predicted trajectories and the average distance between the predicted positions. For the example above, the directional consistency is calculated to be 0.85 (the more similar the direction, the closer this value is to 1), and the positional proximity is 0.65 (the closer the position, the closer this value is to 1). Taking these two metrics into account, the trajectory overlap is calculated to be 0.75.

[0088] Assuming the second preset threshold is set to 0.7, the trajectory overlap of the interaction group (0.75) is greater than the threshold, so the system identifies the interaction group as a collaborative target group. This indicates that the two target objects are not only close to each other in space, but also exhibit collaborative behavior in terms of motion characteristics.

[0089] Through this process, the system successfully identifies groups of targets exhibiting coordinated behavior within a target image sequence, providing a foundation for subsequent behavioral analysis and event prediction. This multi-target analysis method, based on temporal trajectories and interaction features, effectively captures the interactions between target objects and enables a precise understanding of multi-target behavior in complex scenarios.

[0090] In an optional embodiment, the method further includes:

[0091] Calculate the geometric center coordinates and movement direction of the group; calculate the average distance between all target objects in the group and the geometric center to obtain the spatial distribution radius of the group; identify the dominant object in the group, the dominant object being the target object closest to the group's geometric center and with the smallest deviation from the group's movement direction; calculate the trajectory similarity between other target objects and the dominant object, the trajectory similarity being equal to the inverse of the difference in movement direction between the target object and the dominant object; and calculate the group's stability coefficient based on the trajectory similarity.

[0092] When the stability coefficient of the group is greater than the third preset threshold, the group is marked as a stable collaborative group; when the spatial distribution radius of the group is greater than the fourth preset threshold or the trajectory similarity of the target object is less than the fifth preset threshold, a group disbanding warning is triggered.

[0093] like Figure 3 As shown, the method includes:

[0094] The geometric center coordinates of the group are calculated by averaging the position coordinates of all the target objects in the group. For example, if there are three target objects in the group, and their position coordinates are (10, 15), (12, 18), and (14, 12), then the geometric center coordinates of the group are ((10+12+14) / 3, (15+18+12) / 3), or (12, 15). This method also calculates the group's motion direction, which is obtained by averaging the motion vectors of all the target objects in the group. If the motion vectors of the three target objects are (2, 1), (1, 2), and (2, 2), then the group's motion direction is ((2+1+2) / 3, (1+2+2) / 3), or (1.67, 1.67).

[0095] Calculate the spatial distribution radius of the group by calculating the distance between all target objects in the group and the geometric center and then taking the average. Using the above example, the distances of the three target objects to the geometric center (12,15) are sqrt((10-12) 2 +(15-15) 2 )=2, sqrt((12-12) 2 +(18-15) 2)=3 and sqrt((14-12) 2 +(12-15) 2 )=3.61, the average distance is (2+3+3.61) / 3=2.87, which is the spatial distribution radius of the group.

[0096] The method also includes identifying the dominant object within the group, which refers to the target object that is closest to the geometric center of the group and has the smallest deviation from the group's motion direction. In the above example, the distances of the three target objects to the geometric center have been calculated, which are 2, 3, and 3.61 respectively. Next, the deviation of the motion direction of each target object from the group's motion direction (1.67, 1.67) is calculated. Assume that the deviations of the three target objects from the group's motion direction are 0.2, 0.1, and 0.3 respectively. Then, the first target object is closest to the group's geometric center, but the second target object has the smallest deviation from the group's motion direction. In this case, the first target object can be selected as the dominant object by comprehensively considering the distance and direction deviations.

[0097] The next step in the method is to calculate the trajectory similarity between the other target objects and the dominant object. Trajectory similarity is calculated as the inverse of the difference in motion direction between the target object and the dominant object. For example, if the motion direction differences between the second and third target objects and the dominant object (i.e., the first target object) are 0.3 and 0.5, respectively, then their trajectory similarities with the dominant object are 1 / 0.3 = 3.33 and 1 / 0.5 = 2, respectively.

[0098] Based on trajectory similarity, this method calculates the stability coefficient of the group. The stability coefficient can be obtained by taking the average trajectory similarity between all non-dominant objects and the dominant object. In the above example, the stability coefficient of the group is (3.33 + 2) / 2 = 2.67.

[0099] When the group's stability coefficient is greater than a third preset threshold, the method labels the group as a stable collaborative group. For example, if the third preset threshold is set to 2.5, then the group's stability coefficient of 2.67 is greater than 2.5, and therefore it is labeled a stable collaborative group. This means that the target objects within the group have a high degree of collaboration and are performing some kind of collaborative task.

[0100] The method also includes a mechanism for triggering a group disbandment warning. When the spatial distribution radius of the group is greater than the fourth preset threshold or the trajectory similarity of the target object is less than the fifth preset threshold, the group disbandment warning is triggered. Assume that the fourth preset threshold is set to 3 and the fifth preset threshold is set to 1.5. In the above example, the spatial distribution radius of the group is 2.87, which is less than the fourth preset threshold 3, and the disbandment warning will not be triggered. However, if there is a target object with a trajectory similarity of 1.2 with the dominant object, which is less than the fifth preset threshold 1.5, then a group disbandment warning will be triggered. This indicates that the target object is deviating from the group and the group is at risk of disbanding.

[0101] This method can further refine the calculation of the stability coefficient. In addition to considering trajectory similarity, it can also take into account the distance between the target object and the dominant object. For example, a higher weight can be assigned to target objects closer to the dominant object, so that the stability coefficient more accurately reflects the group's coordination.

[0102] The method can also include a mechanism for dynamically adjusting preset thresholds. Based on historical data and the current environment, the system can automatically adjust the third, fourth, and fifth preset thresholds to adapt to group behavior in different scenarios. For example, in crowded environments, the fourth preset threshold (the threshold for the spatial distribution radius) may need to be lowered because, in such situations, even collaborative groups are forced to maintain a larger spatial distribution.

[0103] Through the above method, the group behavior of the target object can be effectively identified and tracked, especially the formation and dissolution of collaborative groups, providing valuable information for target tracking and behavior analysis.

[0104] In an optional embodiment, dynamically adjusting the weights of the multi-objective interaction feature graph in combination with environmental parameters of the petrochemical area to obtain an environment-aware interaction feature graph; and calculating the security situation score of the petrochemical area based on the environment-aware interaction feature graph includes:

[0105] Acquiring environmental parameters of the petrochemical area, wherein the environmental parameters include temperature values ​​of multiple temperature monitoring points and gas concentration values ​​of multiple gas monitoring points;

[0106] A temperature field influence factor is constructed based on the temperature value of the temperature monitoring point, and the temperature field influence factor is obtained by calculating the ratio of the distance from the target to the temperature monitoring point to the temperature field attenuation coefficient, combined with the normalized temperature value of the temperature monitoring point; a gas concentration influence factor is constructed based on the gas concentration value of the gas monitoring point, and the gas concentration influence factor is obtained by calculating the ratio of the height difference from the target to the gas monitoring point to the gas diffusion coefficient, combined with the normalized concentration value of the gas monitoring point;

[0107] Acquire a multi-target interaction feature map, the multi-target interaction feature map including distance features and speed features between targets; adjust the distance features based on the temperature field influencing factor and the gas concentration influencing factor to obtain an environmentally perceived distance feature; and adjust the speed features based on spatial gradients of the temperature field influencing factor and the gas concentration influencing factor to obtain an environmentally perceived speed feature;

[0108] Performing feature fusion on the environment-perceived distance feature and the environment-perceived speed feature to establish an environment-perceived interaction feature map; calculating the target aggregation based on the environment-perceived interaction feature map, wherein the target aggregation is obtained by measuring the similarity between targets;

[0109] The petrochemical area is divided into multiple sub-areas, and the regional hazard level is calculated based on the impact intensity of the hazard sources in each sub-area. The safety situation score is obtained by combining the concentration of the target and the regional hazard level through nonlinear mapping.

[0110] Obtain environmental parameters for petrochemical sites, including temperature values ​​from multiple temperature monitoring points and gas concentration values ​​from multiple gas monitoring points. For example, within a petrochemical site, 10 temperature monitoring points and 8 gas monitoring points are deployed. The temperature monitoring points collect real-time temperature data within the area, while the gas monitoring points monitor combustible gas concentrations. These monitoring points transmit this data to the security situation assessment system using IoT technology.

[0111] The temperature field influence factor is constructed based on the temperature values ​​of the temperature monitoring points. The specific implementation process is as follows: for each monitored target, the Euclidean distance from the target to each temperature monitoring point is calculated. For example, the distance between target A and temperature monitoring point T1 is 15 meters. The original temperature value of the temperature monitoring point is normalized. For example, the temperature range of 35℃-85℃ is mapped to between 0-1. The normalized value corresponding to 67℃ measured at temperature monitoring point T1 is 0.64. The preset temperature field attenuation coefficient is 20 meters, indicating that the temperature influence is significantly attenuated within a range of 20 meters. The temperature field influence factor is calculated by multiplying the ratio of the distance from the target to the temperature monitoring point to the temperature field attenuation coefficient by the normalized temperature value. For example, the temperature field influence factor of target A affected by temperature monitoring point T1 is 0.64×(15 / 20)=0.48.

[0112] The gas concentration influence factor is constructed based on the gas concentration values ​​at the gas monitoring points. The specific implementation process is as follows: Calculate the height difference between the target and each gas monitoring point. For example, target B is 2.5 meters higher than gas monitoring point G1. The preset gas diffusion coefficient is 5 meters, which represents the gas's ability to diffuse in the vertical direction. The original concentration values ​​of the gas monitoring points are normalized. For example, the concentration range of 0-500ppm is mapped to the range of 0-1. The normalized value corresponding to 350ppm measured at gas monitoring point G1 is 0.7. The gas concentration influence factor is calculated by calculating the ratio of the height difference between the target and the gas monitoring point to the gas diffusion coefficient, and then multiplying it by the normalized concentration value. For example, the gas concentration influence factor of target B affected by gas monitoring point G1 is 0.7×(2.5 / 5)=0.35.

[0113] Obtain a multi-target interaction feature map, which contains the distance and velocity features between targets. The distance feature indicates the relative spatial position of the targets, while the velocity feature indicates the target's motion. For example, the distance between targets A and B is 8 meters, the velocity of target A is 1.2 m / s, and the direction angle is 45 degrees, while the velocity of target B is 0.8 m / s and the direction angle is 120 degrees.

[0114] The distance feature is adjusted based on the temperature field influence factor and the gas concentration influence factor to obtain the environmental perception distance feature. The adjustment method is to multiply the original distance value by the weighted sum of the environmental influence factors. For example, if the original distance between targets A and B is 8 meters, considering that target A has a temperature field influence factor of 0.48 and a gas concentration influence factor of 0.2 (weights of 0.6 and 0.4, respectively), and target B has a temperature field influence factor of 0.3 and a gas concentration influence factor of 0.35 (weights of 0.6 and 0.4, respectively), the environmental perception distance feature is 8 × [1-(0.48 × 0.6 + 0.2 × 0.4 + 0.3 × 0.6 + 0.35 × 0.4) / 2] = 5.68 meters.

[0115] The velocity signature is adjusted based on the spatial gradients of the temperature field and gas concentration factors to produce an environmentally perceived velocity signature. The spatial gradient represents the rate of change of environmental factors in space and is calculated by dividing the difference in environmental factors between adjacent monitoring points by the distance between them. When adjusting the velocity signature, a weighted combination of the original velocity vector and the environmental factor gradient vector is performed. For example, if target A has an original velocity of 1.2 m / s and a 45-degree heading, and a temperature field gradient of 0.05 m / m and a 30-degree heading at its location, and a gas concentration gradient of 0.03 m / m and a 60-degree heading, the velocity after factoring in the combined environmental gradients (weights of 0.7 and 0.3, respectively) is 1.35 m / s and a 42-degree heading.

[0116] The distance features and speed features of environmental perception are fused to create an interactive feature map of environmental perception. The fusion method is to weight the distance feature matrix and the speed feature matrix according to preset weights (for example, the distance feature weight is 0.6 and the speed feature weight is 0.4) to obtain the fused feature map.

[0117] The clustering of targets is calculated based on the interaction feature map of environmental perception. This clustering is determined by measuring the similarity between targets. This similarity is calculated by applying a Gaussian kernel function to the distance and velocity differences between targets in the fused feature map. For example, at a certain moment, five targets are detected moving within an area. The clustering calculated based on the interaction feature map of environmental perception is 0.72, indicating a high degree of clustering.

[0118] The petrochemical area is divided into multiple sub-areas, and the regional hazard level is calculated based on the impact intensity of the hazard sources within each sub-area. For example, a petrochemical area is divided into six sub-areas. Sub-area 1 contains a high-pressure storage tank with an impact intensity of 0.9, and sub-area 2 contains two medium-pressure pipelines with impact intensities of 0.6 and 0.5, respectively. The combined hazard level of sub-area 1 is 0.9, and that of sub-area 2 is 0.78.

[0119] A security situation score is derived through nonlinear mapping, combining the target's concentration and regional risk. This nonlinear mapping uses a sigmoid function to map the weighted combination of concentration and risk onto a score range of 0-100. For example, if the concentration at a certain moment is 0.72 and the average regional risk is 0.83, with a weighting ratio of 1:1.5, the security situation score derived from the sigmoid function is 35, indicating a high level of security alert.

[0120] Figure 4 This is a bar chart comparing the performance of the petrochemical area security situation assessment method according to an embodiment of the present invention:

[0121] The figure compares the performance of this technical solution with traditional environmental perception methods and three solutions that do not consider environmental factors on key safety assessment indicators. The specific data shows that this technical solution demonstrates significant advantages across all three core assessment indicators: In terms of hazard warning accuracy, this solution achieved 53.3%, significantly higher than the 48.6% of traditional environmental perception methods and the 38.2% of methods that do not consider environmental factors. In terms of abnormal behavior recognition rate, this solution achieved 52.7%, surpassing the 47.5% of traditional methods and the 39.1% of methods that do not consider environmental factors. In terms of emergency response speed assessment, this solution achieved 57.6%, also surpassing the 48.8% of traditional methods and the 38.9% of methods that do not consider environmental factors. This data comparison clearly demonstrates that by incorporating environmental perception capabilities, this technical solution achieves significant improvements in key safety indicators such as hazard warning, abnormal behavior recognition, and emergency response. The improvements are 5-9 percentage points on average compared to traditional methods, and even more significant, reaching 15-19 percentage points compared to methods that do not consider environmental factors. This fully demonstrates the important role of environmental perception in improving safety assessment performance.

[0122] In an optional embodiment, a dynamic inference graph network is constructed for behavior recognition. The environment-perceived interaction feature graph is input into the dynamic inference graph network. Node features are extracted through graph convolution operations. The dynamic association relationship between nodes is updated based on a message passing mechanism. The recognition result of the target behavior is obtained, including:

[0123] Acquire the position features, speed features, and environmental features of the target, construct the position features, speed features, and environmental features into node feature vectors, construct edge feature vectors based on the association relationship between the node feature vectors; and form an initialization structure of a dynamic reasoning graph network with the node feature vectors and the edge feature vectors;

[0124] Building a temporal connection matrix based on the initialization structure of the dynamic reasoning graph network, and performing spatial feature aggregation on the node feature vectors according to the temporal connection matrix to obtain multi-layer node features;

[0125] Calculating attention weights for the multi-layer node features, where the attention weights are obtained by concatenating the node features and performing a nonlinear transformation; performing weighted aggregation on the multi-layer node features based on the attention weights to obtain enhanced node features;

[0126] Generate message features based on the enhanced node features and the edge feature vectors, wherein the message features include interaction information between nodes; aggregate the message features based on the attention weights, and update the node states through a recurrent neural network to obtain dynamic node features;

[0127] Calculating the temporal attention weight of the dynamic node feature, fusing the temporal attention weight with the historical feature matrix to obtain a temporal context feature; inputting the temporal context feature into the long short-term memory network, updating the memory state to obtain a fused feature;

[0128] Multi-scale feature extraction and feature pooling are performed on the fused features to obtain a behavior feature representation; a behavior classification score is calculated based on the behavior feature representation, and a recognition result of the target behavior is output.

[0129] Obtain the target's position characteristics, velocity characteristics and environmental characteristics. Position characteristics include the target's coordinate information in two-dimensional or three-dimensional space, such as (x, y) or (x, y, z); velocity characteristics include the target's velocity components in each direction, such as (v x , v y , v z ); environmental features include information such as the distribution of surrounding objects and road structure. Taking an autonomous driving scenario as an example, a pedestrian's position feature can be represented as (120.5, 85.3), indicating their pixel position in the image coordinate system; their speed feature can be represented as (1.2, 0.8), in meters per second; and environmental features can include surrounding vehicles and traffic light status. These features are integrated into a node feature vector, the dimensions of which can range from 64 to 256 depending on the application scenario.

[0130] The association between nodes is calculated based on spatial distance and motion similarity. For example, if the Euclidean distance between two targets is less than a preset threshold (e.g., 5 meters), a connection is established and an edge feature vector is constructed. The edge feature vector encodes the relative position, relative velocity, and interaction probability between the node pair. For targets A and B, their relative position can be expressed as (3.2, 1.5), their relative velocity as (0.5, -0.3), and their interaction probability as 0.78. The dimension of the edge feature vector is typically set to 32 to 128.

[0131] The initial structure of a dynamic reasoning graph network consists of node feature vectors and edge feature vectors. In a typical scenario, it contains 5 to 20 nodes, each of which is connected to 3 to 5 adjacent nodes, forming a sparsely connected graph structure.

[0132] Based on the initialized graph structure, a temporal connectivity matrix is ​​constructed to capture the interactive relationships between objects over time. The temporal connectivity matrix is ​​an N×N matrix (N is the number of nodes), and the matrix element values ​​represent the connection strength between nodes. For example, the temporal connectivity matrix for a 5-node scenario is [[0, 0.8, 0.5, 0, 0], [0.8, 0, 0.6, 0.3, 0], [0.5, 0.6, 0, 0.7, 0.4], [0, 0.3, 0.7, 0, 0.9], [0, 0, 0.4, 0.9, 0]]. This matrix is ​​used to spatially aggregate node features, enabling information transfer between connected nodes. Through multi-layer graph convolution operations, multiple layers of node features with different receptive fields are obtained, and the feature dimension of each layer can be up to 128.

[0133] When calculating attention weights for multi-layer node features, node features from different layers are concatenated and then transformed nonlinearly to obtain weight coefficients. Specifically, for three layers of features, concatenation yields a 384-dimensional vector (128×3). This is then mapped via a fully connected layer to a 3D attention weight vector, such as [0.4, 0.35, 0.25], indicating the importance attached to the three-layer features. Based on these weights, multi-layer node features are weighted and aggregated to obtain enhanced 128-dimensional node features.

[0134] Message features are generated based on the enhanced node and edge feature vectors. Message features combine the source node features with the edge features to create a vector containing interaction information. For example, for a message sent from node A to node B, the node features (128 dimensions) of node A and the edge features (64 dimensions) of node B can be concatenated to produce a 128-dimensional message feature. These message features contain information about the interactions between nodes, such as behavioral intent and impact.

[0135] Message features are aggregated using the calculated attention weights, and node states are updated via a gated recurrent unit (GRU). Each node receives messages from all neighboring nodes, performs weighted aggregation to obtain input features, and then updates its hidden state via the GRU to generate dynamic node features. The GRU's hidden state dimension is set to 128, and information flow is controlled via update and reset gates, enabling selective memory of historical information.

[0136] Temporal attention weights are calculated for dynamic node features to assess the importance of features at different time points. For a time series of length 10, the resulting attention weights are [0.02, 0.03, 0.05, 0.07, 0.1, 0.13, 0.15, 0.18, 0.12, 0.15], indicating that recent features are generally more important. These weights are multiplied by the historical feature matrix to obtain temporal context features.

[0137] The temporal context features are input into a long short-term memory (LSTM) network, and the memory state is updated to generate fused features. The LSTM consists of an input gate, a forget gate, and an output gate, effectively capturing long-term dependencies. For example, a single-layer LSTM with a hidden state dimension of 256 is used. The forget gate coefficient (e.g., 0.3) controls the proportion of historical information retained to generate fused features.

[0138] Multi-scale feature extraction is performed on the fused features, using convolutional layers with different kernel sizes (e.g., 3×3, 5×5, and 7×7) to capture spatial patterns at different scales. These features are then global average pooled and max pooled to generate a compact behavioral feature representation with a dimension of 512.

[0139] The behavior classification score is calculated based on the behavior feature representation. A fully connected layer maps the 512-dimensional features to a vector corresponding to the number of behavior categories (e.g., for 8 behavior categories, an 8-dimensional vector is used), resulting in a score for each category, such as [0.05, 0.02, 0.15, 0.63, 0.03, 0.07, 0.01, 0.04]. The scores are converted to a probability distribution using the softmax function, and the category with the highest probability is the recognition result. In this example, the fourth category (with a probability of 0.63) is identified as the target behavior.

[0140] A second aspect of an embodiment of the present invention provides a deep learning-based early warning system for illegal operations in petroleum and petrochemical hazardous areas, comprising:

[0141] The first unit is configured to obtain a target image sequence corresponding to video data of a petrochemical area, perform target detection and tracking on the target image sequence, obtain position coordinates of a target object in the target image sequence, and establish a target index;

[0142] The second unit is configured to establish a time series trajectory for each target object in the target image sequence based on the target index; construct a multi-target interaction feature map according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed, and relative direction information between target objects;

[0143] The third unit is configured to dynamically adjust the weights of the multi-objective interaction feature graph in combination with the environmental parameters of the petrochemical area to obtain an environmentally-aware interaction feature graph; and calculate a security situation score for the petrochemical area based on the environmentally-aware interaction feature graph, wherein the security situation score represents the overall security status of the petrochemical area at the current moment.

[0144] The fourth unit is used to construct a dynamic reasoning graph network for behavior recognition, input the interactive feature graph of environmental perception into the dynamic reasoning graph network, extract node features through graph convolution operation, and update the dynamic association relationship between nodes based on the message passing mechanism to obtain the recognition result of the target behavior.

[0145] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0146] processor;

[0147] a memory for storing processor-executable instructions;

[0148] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0149] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0150] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based early warning method for illegal operations in hazardous areas of petroleum and petrochemical industries, characterized by: include: Obtaining a target image sequence corresponding to video data of a petrochemical area, performing target detection and tracking on the target image sequence, obtaining position coordinates of a target object in the target image sequence, and establishing a target index; Based on the target index, a time series trajectory is established for each target object in the target image sequence; a multi-target interaction feature map is constructed according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed and relative direction information between target objects; Dynamically adjusting the weights of the multi-objective interactive feature map in combination with environmental parameters of the petrochemical area to obtain an environmentally-aware interactive feature map; Calculating a safety situation score for the petrochemical area based on the interactive feature graph of environmental perception, wherein the safety situation score represents the overall safety status of the petrochemical area at a current moment; A dynamic reasoning graph network is constructed for behavior recognition. The interaction feature graph of the environmental perception is input into the dynamic reasoning graph network. Node features are extracted through graph convolution operations. The dynamic association relationship between nodes is updated based on the message passing mechanism to obtain the recognition result of the target behavior.

2. The method according to claim 1, characterized in that Performing target detection and tracking on the target image sequence, obtaining the position coordinates of the target object in the target image sequence, and establishing a target index includes: Detecting the target image sequence using a target detection model, obtaining target objects in the target image sequence, and extracting position coordinates, size information, and category information of the target objects; Assigning a globally unique identifier to each target object and establishing a target index table, wherein the target index table records the identifier, position coordinates, size information, category information, and timestamp of the target object; establishing a temporal association of the target objects by comparing the visual features and motion features of the target objects between adjacent frames; and associating target objects with the same visual features and continuous motion trajectories with the same identifier; The motion parameters of the target object are calculated based on the position coordinates recorded in the target index table.

3. The method according to claim 1, characterized in that Based on the target index, a time series trajectory is established for each target object in the target image sequence; and a multi-target interaction feature map is constructed according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed, and relative direction information between target objects. According to the identification code of the target object, the position coordinates of the target object are recorded in chronological order to obtain a time series coordinate sequence of the target object; the displacement and time difference of adjacent coordinate points in the time series coordinate sequence are calculated to obtain the movement speed and movement direction of the target object at different moments; and the position coordinates, the movement speed, and the movement direction are combined to form a time series trajectory of the target object; The relative distance, relative speed and relative direction between target objects are calculated to construct a multi-target interaction feature map; the relative distance, relative speed and relative direction are converted into an interaction strength score according to preset distance weight, speed weight and direction weight; when the interaction strength score is greater than a first preset threshold, the corresponding target objects are divided into the same interaction group; the overlap of the motion trajectories of the target objects in the interaction group is calculated, and when the overlap of the motion trajectories is greater than a second preset threshold, the interaction group is determined to be a collaborative target group.

4. The method according to claim 3, characterized in that The method further comprises: Calculate the geometric center coordinates and movement direction of the group; calculate the average distance between all target objects in the group and the geometric center to obtain the spatial distribution radius of the group; identify the dominant object in the group, the dominant object being the target object closest to the group's geometric center and with the smallest deviation from the group's movement direction; calculate the trajectory similarity between other target objects and the dominant object, the trajectory similarity being equal to the inverse of the difference in movement direction between the target object and the dominant object; and calculate the group's stability coefficient based on the trajectory similarity. When the stability coefficient of the group is greater than the third preset threshold, the group is marked as a stable collaborative group; when the spatial distribution radius of the group is greater than the fourth preset threshold or the trajectory similarity of the target object is less than the fifth preset threshold, a group disbanding warning is triggered.

5. The method according to claim 1, wherein Dynamically weighting the multi-objective interactive feature graph in combination with environmental parameters of the petrochemical area to obtain an environmentally-aware interactive feature graph; and calculating a security situation score for the petrochemical area based on the environmentally-aware interactive feature graph includes: Acquiring environmental parameters of the petrochemical area, wherein the environmental parameters include temperature values ​​of multiple temperature monitoring points and gas concentration values ​​of multiple gas monitoring points; A temperature field influence factor is constructed based on the temperature value of the temperature monitoring point, and the temperature field influence factor is obtained by calculating the ratio of the distance from the target to the temperature monitoring point to the temperature field attenuation coefficient, combined with the normalized temperature value of the temperature monitoring point; a gas concentration influence factor is constructed based on the gas concentration value of the gas monitoring point, and the gas concentration influence factor is obtained by calculating the ratio of the height difference from the target to the gas monitoring point to the gas diffusion coefficient, combined with the normalized concentration value of the gas monitoring point; Acquire a multi-target interaction feature map, the multi-target interaction feature map including distance features and speed features between targets; adjust the distance features based on the temperature field influencing factor and the gas concentration influencing factor to obtain an environmentally perceived distance feature; and adjust the speed features based on spatial gradients of the temperature field influencing factor and the gas concentration influencing factor to obtain an environmentally perceived speed feature; Performing feature fusion on the environment-perceived distance feature and the environment-perceived speed feature to establish an environment-perceived interaction feature map; calculating the target aggregation based on the environment-perceived interaction feature map, wherein the target aggregation is obtained by measuring the similarity between targets; The petrochemical area is divided into multiple sub-areas, and the regional hazard degree is calculated based on the impact intensity of the hazard source in each sub-area; the safety situation score is obtained through nonlinear mapping by combining the concentration of the target and the regional hazard degree.

6. The method according to claim 1, wherein A dynamic inference graph network is constructed for behavior recognition. The environment-perceived interaction feature graph is input into the dynamic inference graph network. Node features are extracted through graph convolution operations, and the dynamic association relationship between nodes is updated based on the message passing mechanism. The recognition results of the target behavior include: Acquire the position features, speed features, and environmental features of the target, construct the position features, speed features, and environmental features into node feature vectors, construct edge feature vectors based on the association relationship between the node feature vectors; and form an initialization structure of a dynamic reasoning graph network with the node feature vectors and the edge feature vectors; Building a temporal connection matrix based on the initialization structure of the dynamic reasoning graph network, and performing spatial feature aggregation on the node feature vectors according to the temporal connection matrix to obtain multi-layer node features; Calculating attention weights for the multi-layer node features, where the attention weights are obtained by concatenating the node features and performing a nonlinear transformation; performing weighted aggregation on the multi-layer node features based on the attention weights to obtain enhanced node features; Generate message features based on the enhanced node features and the edge feature vectors, wherein the message features include interaction information between nodes; aggregate the message features based on the attention weights, and update the node states through a recurrent neural network to obtain dynamic node features; Calculating the temporal attention weight of the dynamic node feature, fusing the temporal attention weight with the historical feature matrix to obtain a temporal context feature; inputting the temporal context feature into the long short-term memory network, updating the memory state to obtain a fused feature; Multi-scale feature extraction and feature pooling are performed on the fused features to obtain a behavior feature representation; a behavior classification score is calculated based on the behavior feature representation, and a recognition result of the target behavior is output.

7. A deep learning-based early warning system for illegal operations in petroleum and petrochemical hazardous areas, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is configured to obtain a target image sequence corresponding to video data of a petrochemical area, perform target detection and tracking on the target image sequence, obtain position coordinates of a target object in the target image sequence, and establish a target index; The second unit is configured to establish a time series trajectory for each target object in the target image sequence based on the target index; construct a multi-target interaction feature map according to the time series trajectory, wherein the multi-target interaction feature map includes relative distance, relative speed, and relative direction information between target objects; The third unit is used to dynamically adjust the weight of the multi-objective interactive feature map in combination with the environmental parameters of the petrochemical area to obtain an environmental perception interactive feature map; Calculating a safety situation score for the petrochemical area based on the interactive feature graph of environmental perception, wherein the safety situation score represents the overall safety status of the petrochemical area at a current moment; The fourth unit is used to construct a dynamic reasoning graph network for behavior recognition, input the interactive feature graph of environmental perception into the dynamic reasoning graph network, extract node features through graph convolution operation, and update the dynamic association relationship between nodes based on the message passing mechanism to obtain the recognition result of the target behavior.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images