A method and system for detecting abnormal behavior in large-scale intelligent agents

Abstract

The application provides a large-scale agent-oriented abnormal behavior detection method and system, relating to the technical field of artificial intelligence, the method comprising: collecting multi-source heterogeneous original data from distributed deployed agent nodes, performing data preprocessing on the collected original data to obtain a preprocessed standardized data set; extracting an internal region intersecting a clipping window from the preprocessed standardized data set to obtain a candidate abnormal region set in the spatial dimension; extracting a time sequence behavior feature from the candidate abnormal region set, calculating a reconstruction error of the time sequence behavior feature through a self-encoder, capturing an abnormal pattern in the data distribution, and obtaining an image abnormal region. Through a complete closed loop of data integration, feature extraction, multi-modal fusion and deep analysis, the application processes massive data of large-scale agents, and improves the detection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for detecting abnormal behavior of large-scale intelligent agents. Background Technology

[0002] In the construction of smart coal mines, a large number of heterogeneous intelligent agent nodes are deployed underground, including high-definition camera equipment, environmental sensors, personnel positioning beacons, and equipment operation monitoring terminals. These nodes collect video images of underground workers, equipment operating status, personnel movement trajectories, and environmental data in real time. The system aims to identify abnormal working behaviors of underground personnel in real time through multimodal data analysis and provide timely warnings to ensure safe production in coal mines.

[0003] However, existing monitoring systems face the following significant shortcomings in this scenario: existing systems often process video, sensor data, or positioning trajectories independently, lacking a unified fusion and analysis mechanism for visual behavior, equipment status, and spatial location information.

[0004] For example, the system may detect a gathering of people, but it cannot combine the sudden increase in gas sensor data with whether people have entered dangerous areas in violation of regulations to make a comprehensive risk assessment, resulting in delayed or missed warnings; uneven lighting, dust obstruction, equipment vibration and other interference factors in the mine seriously affect video quality; traditional image processing methods are not capable of detecting blurred, low-light or partially obscured targets, which easily leads to missed or false detections.

[0005] Existing methods often rely on single-frame images or short time segments for identification, making it difficult to capture the temporal dependencies and spatial relationships of behavior. For example, a system might detect that a person is not wearing a safety helmet at a certain moment, but it cannot combine this with whether the person is in a rest area or has just completed equipment maintenance to make a compliance judgment, leading to frequent false alarms. The system cannot identify the motives behind abnormal behavior or multiple people colluding to violate regulations. Traditional methods only output a binary result of whether it is abnormal, and cannot provide decision support information such as why it is abnormal or whether multiple people are involved in collusion, making it difficult to curb systemic risks at their root. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide an abnormal behavior detection method and system for large-scale intelligent agents. Through a complete closed loop of data integration, feature extraction, multimodal fusion, and deep analysis, it processes massive amounts of data from large-scale intelligent agents, thereby improving detection efficiency and accuracy.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0008] Firstly, a method for detecting abnormal behavior in large-scale intelligent agents, the method comprising:

[0009] Step 1: Collect multi-source heterogeneous raw data from distributed intelligent agent nodes, and preprocess the collected raw data to obtain a preprocessed standardized dataset.

[0010] Step 2: Extract the internal regions that intersect with the clipping window from the preprocessed standardized dataset to obtain a set of candidate abnormal regions in the spatial dimension;

[0011] Step 3: Extract temporal behavior features from the candidate abnormal region set, and use the temporal behavior features to calculate the reconstruction error through an autoencoder to capture abnormal patterns in the data distribution and obtain abnormal regions of the image;

[0012] Step 4: Based on the abnormal regions in the image, construct a planar subdivision structure, and use point query operations to spatially locate the abnormal regions in the image, determine the specific location and region affiliation of the abnormal regions in the polygon grid, and obtain the region features after localization.

[0013] Step 5: Weight the localized regional features, calculate the correlation weights between features to obtain weighted image features, and perform fine-grained fusion of the weighted image features with temporal behavioral features to form unified multimodal feature data.

[0014] Step 6: Analyze the abnormal crossover patterns of the behavioral trajectories in the multimodal feature data, and identify the abnormal interaction feature description vectors in the behavioral trajectories;

[0015] Step 7: Perform multi-dimensional analysis on the abnormal interaction feature description vector to achieve abnormal behavior detection of large-scale intelligent agents. The multi-dimensional analysis includes motivation analysis, operation method identification and collusion network construction.

[0016] Secondly, an abnormal behavior detection system for large-scale intelligent agents includes:

[0017] The data acquisition module is used to collect multi-source heterogeneous raw data from distributed intelligent agent nodes, and to preprocess the collected raw data to obtain a preprocessed standardized dataset.

[0018] The extraction module is used to extract the internal regions that intersect with the cropping window from the preprocessed standardized dataset to obtain a set of candidate abnormal regions in the spatial dimension; extract temporal behavior features from the set of candidate abnormal regions, and calculate the reconstruction error using an autoencoder to capture abnormal patterns in the data distribution and obtain abnormal regions in the image.

[0019] The fusion module is used to construct a planar subdivision structure based on the abnormal regions in the image, spatially locate the abnormal regions in the image through point query operations, determine the specific location and region affiliation of the abnormal regions in the polygon grid, and obtain the localized region features; the localized region features are weighted and the correlation weights between features are calculated to obtain weighted image features, and the weighted image features are fused with temporal behavior features in a fine-grained manner to form unified multimodal feature data;

[0020] The identification module is used to analyze abnormal crossover patterns in the behavioral trajectories of multimodal feature data and identify abnormal interaction feature description vectors in the behavioral trajectories. It performs multi-dimensional analysis on the abnormal interaction feature description vectors to achieve abnormal behavior detection of large-scale intelligent agents. The multi-dimensional analysis includes motivation analysis, operation method identification, and collusion network construction.

[0021] Thirdly, a computing device includes:

[0022] One or more processors;

[0023] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0024] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0025] The above-described solution of the present invention has at least the following beneficial effects:

[0026] By collecting heterogeneous raw data from multiple sources and performing standardized preprocessing, a unified integration of distributed intelligent agent node data was achieved. The standardized dataset provides a high-quality data foundation for subsequent feature extraction and anomaly detection. Edge-by-edge pruning was used to extract internal regions intersecting with the pruning window, accurately focusing on potential anomaly regions in the spatial dimension and avoiding redundant computation of irrelevant regions. Long Short-Term Memory (LSTM) networks were used to extract temporal behavioral features, combined with autoencoder reconstruction errors to capture abnormal patterns. This preserved the temporal dependencies of behaviors while accurately identifying anomaly regions through data distribution deviations. Furthermore, heterogeneous features were achieved by constructing a planar subdivision structure and point query operations. The system locates and assigns regions to polygonal grids, solving the problems of fuzzy spatial positioning and unclear region affiliation in traditional systems. It employs a self-attention mechanism to weight region features, highlighting the importance of key features. By finely fusing image and temporal features, it overcomes the limitations of independent processing of visual and temporal data in traditional systems. By calculating the relative orientation and intersection parameters of trajectory segments, it identifies abnormal intersection patterns, capturing potential risk correlations between behavioral trajectories. Combined with the CRIME theoretical framework, it analyzes abnormal behavior from dimensions such as motivation, operational methods, and collusion networks. This deep analysis breaks through the limitations of traditional systems that only output binary results. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an abnormal behavior detection method for large-scale intelligent agents provided by an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of an abnormal behavior detection system for large-scale intelligent agents provided by an embodiment of the present invention. Detailed Implementation

[0029] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0030] like Figure 1 As shown, an embodiment of the present invention proposes an abnormal behavior detection method for large-scale intelligent agents, the method comprising the following steps:

[0031] Step 1: Collect multi-source heterogeneous raw data from distributed intelligent agent nodes, and preprocess the collected raw data to obtain a preprocessed standardized dataset.

[0032] Step 2: Extract the internal regions that intersect with the clipping window from the preprocessed standardized dataset to obtain a set of candidate abnormal regions in the spatial dimension;

[0033] Step 3: Extract temporal behavior features from the candidate abnormal region set, and use the temporal behavior features to calculate the reconstruction error through an autoencoder to capture abnormal patterns in the data distribution and obtain abnormal regions of the image;

[0034] Step 4: Based on the abnormal regions in the image, construct a planar subdivision structure, and use point query operations to spatially locate the abnormal regions in the image, determine the specific location and region affiliation of the abnormal regions in the polygon grid, and obtain the region features after localization.

[0035] Step 5: Weight the localized regional features, calculate the correlation weights between features to obtain weighted image features, and perform fine-grained fusion of the weighted image features with temporal behavioral features to form unified multimodal feature data.

[0036] Step 6: Analyze the abnormal crossover patterns of the behavioral trajectories in the multimodal feature data, and identify the abnormal interaction feature description vectors in the behavioral trajectories;

[0037] Step 7: Perform multi-dimensional analysis on the abnormal interaction feature description vector to achieve abnormal behavior detection of large-scale intelligent agents. The multi-dimensional analysis includes motivation analysis, operation method identification and collusion network construction.

[0038] In this embodiment of the invention, multi-source heterogeneous data is collected to cover agent behavior and environmental information. After preprocessing, a standardized dataset is generated to reduce detection bias. Potential abnormal regions are extracted to reduce the scope of subsequent processing, improve detection efficiency, and focus on the abnormal spatial range. LSTM extracts temporal behavioral features to avoid missing anomalies due to ignoring long-term dependencies. The autoencoder distinguishes abnormal data through reconstruction error, without requiring a large amount of labeled data. It can adaptively capture abnormal patterns in the data distribution, balancing detection accuracy and data adaptability. Determining the location coordinates and region affiliation of anomalies within a polygonal grid provides spatial contextual information for anomalous behavior, facilitating rapid problem localization. Analysis of anomalous region correlations based on location attributes provides a basis for identifying multi-agent collaborative anomalies. A self-attention mechanism generates an attention weight matrix by calculating the query vector and key vector correlation of the localized region features, assigning high weights to key features and low weights to redundant background information, thus improving the effectiveness of image features. Unified multimodal feature data breaks down the separation between spatial and temporal features, characterizing the spatiotemporal integration of anomalous behavior. Calculating the relative orientation and intersection parameters of behavioral trajectory segments in the multimodal feature data analyzes anomalous trajectory intersection patterns, enabling the capture of interaction anomalies between multiple agents and the identification of trajectory conflicts. Generating anomalous interaction feature description vectors quantifies the degree of anomaly, upgrading anomalous behavior identification from qualitative judgment to quantitative analysis, and improving the reliability of detection results. Combining multi-dimensional analysis with the CRIME theoretical framework, it delves into the motivations and collusion networks of anomalous behavior, providing root-cause insights for anomaly governance.

[0039] In a preferred embodiment of the present invention, step 1 involves collecting multi-source heterogeneous raw data from distributed intelligent agent nodes, preprocessing the collected raw data to obtain a preprocessed standardized dataset, including:

[0040] By calling the interfaces of various intelligent agent nodes deployed in a distributed manner underground through a preset data acquisition protocol, multi-source heterogeneous raw data is obtained. Among them, high-definition camera equipment collects video image data of underground workers, including personnel posture, facial features and work actions; environmental sensors collect environmental parameters such as underground gas concentration, temperature, humidity, dust concentration and wind speed; personnel positioning beacons collect the real-time location coordinates and movement trajectory of workers; and equipment operation monitoring terminals collect operating parameters of equipment such as coal mining machines and conveyors, including speed, temperature, vibration and fault codes.

[0041] The collected raw data undergoes data cleaning. For video image data, blurred frames caused by dust obstruction and overexposed / underexposed frames caused by uneven lighting are removed. For environmental sensor data, instantaneous jump values ​​caused by equipment vibration are removed. For personnel positioning data, null values ​​and drift points caused by signal loss are removed. For equipment operation data, redundant records with repeated uploads are removed. The cleaned data is then converted into different formats: video image data is converted into a continuous frame sequence and the pixel matrix of each frame is extracted; environmental sensor data is converted into a time-stamped numerical sequence; personnel positioning data is converted into a set of trajectory points containing time coordinates; and equipment operation data is converted into a status parameter table associated with the equipment number.

[0042] The data after format conversion is standardized, and the pixel values ​​of video images are normalized to ensure that the pixel brightness is within a preset reasonable range. The numerical parameters of environmental sensors are calibrated to ensure that the measurement values ​​of different types of sensors are within a comparable range. The coordinate system of personnel positioning coordinates is unified so that the position data collected by different beacons are based on the same benchmark. Finally, a structured and standardized dataset is formed.

[0043] In a preferred embodiment of the present invention, step 2 includes:

[0044] Step 200: Define the polygonal boundary of the clipping window to determine the geometric constraints of the clipping region;

[0045] Step 201: Based on the geometric constraints of the clipping region, obtain the vertex sequence of the polygon to be clipped, initialize the output vertex list, traverse each boundary edge of the clipping window in sequence, use the current vertex list as input to perform visibility judgment, and obtain the positional relationship of the vertex relative to the current clipping edge.

[0046] Step 202: Based on the positional relationship of the vertices relative to the current clipping edge, preset vertex processing rules are established, and a new sequence of intermediate vertices is generated according to the vertex processing rules.

[0047] Step 203: Use the generated intermediate vertex sequence as the input vertex list for the next clipping edge, iterate through all clipping edges, and obtain the final vertex sequence that constitutes the clipped polygonal region, generating a set of candidate abnormal regions in the spatial dimension.

[0048] In this embodiment, when applied in a specific application, the specific implementation process of defining the polygon boundary of the clipping window in step 200 is as follows: Based on the underground safety monitoring requirements, the polygon boundary corresponding to the clipping window is determined; the boundary is set in a preset underground coordinate system based on the actual spatial structure of the underground roadway; for example, for the clipping area of ​​the tunneling face, the coordinates of each vertex of the polygon are determined by reading the engineering design drawing of the area. The vertices include the two endpoints at the entrance of the tunneling face, the turning point of the edge of the working face, and the dividing point connecting with the transport roadway; the vertices are connected sequentially according to the actual connection order of the underground space, first connecting the entrance endpoint of the tunneling face to the first edge turning point, then connecting each turning point sequentially, and finally connecting to the dividing point connecting with the transport roadway and returning to the entrance endpoint to form a closed polygon structure.

[0049] The geometric constraints of the cut-off region are determined, including that the length of each side of the polygon corresponds to the actual width of the roadway, the direction of each side is consistent with the extension direction of the roadway, and the size of each interior angle matches the turning angle of the roadway. Simultaneously, the relative positional relationship between the polygon boundary and surrounding equipment, such as the safe distance from the coal mining machine, is recorded. This determines the specific range and geometric characteristics of the cut-off region in the underground space.

[0050] In this embodiment, when specifically applied, the specific implementation process of step 201, obtaining the vertex sequence of the polygon to be clipped and determining its visibility, is as follows: Based on the geometric constraints of the clipping region, all vertices of the polygon to be clipped are extracted from the underground space data to be processed; these vertices include boundary points of the worker activity area, outline points of equipment distribution, and marker points of dangerous areas; the arrangement order of each vertex is determined according to the actual layout order of the underground space, for example, starting from the starting point of the left edge of the tunnel face, each edge point is recorded sequentially to the ending point of the right edge, forming the vertex sequence of the polygon to be clipped. An output vertex list is initialized, which is initially empty.

[0051] The processing order is determined according to the actual distribution of the polygon boundary edges of the clipping window. Processing begins with the first boundary edge at the entrance of the tunnel face and proceeds sequentially to the last boundary edge connecting to the transport roadway. Using the current vertex list as input, for each vertex in the list, its coordinates are compared with the spatial position of the current clipping edge. If the vertex is located on the side of the clipping edge closer to the interior of the tunnel face, it is determined to be on the inside; if it is located on the side farther from the interior of the tunnel face, it is determined to be on the outside; if the vertex falls exactly on the clipping edge, it is determined to be on the boundary. This process yields the positional relationship of each vertex relative to the current clipping edge.

[0052] In this embodiment, when applied in a specific way, the specific implementation process of the vertex processing rule preset and intermediate vertex sequence generation in step 202 is as follows: Based on the different positional relationships of vertices relative to the current trimming edge, vertex processing rules are preset; when a vertex is inside the current trimming edge, it indicates that the position corresponding to the vertex belongs to the normal working area of ​​the tunnel face, and its coordinates are added to the intermediate vertex sequence; when a vertex is outside the current trimming edge, it indicates that the position corresponding to the vertex exceeds the normal working area, and it is not included in the intermediate vertex sequence; when two adjacent vertices are inside and outside the current trimming edge respectively, it indicates that there is a situation where the positions corresponding to these two vertices cross the trimming edge, and at this time, by determining the specific position where the line connecting these two vertices intersects with the current trimming edge, the coordinates of the intersection point are obtained and the intersection point is included in the intermediate vertex sequence.

[0053] During processing, vertices are checked sequentially according to their order in the input list. The position attributes of adjacent vertices are compared one by one, and the corresponding processing operation is performed when they meet the above rules. The final intermediate vertex sequence is stored in the order of processing.

[0054] In this embodiment, when applied in a specific application, the specific implementation process of step 203, iterative processing and candidate abnormal region set generation is as follows: the intermediate vertex sequence generated in step 202 is used as the input vertex list of the next clipping edge. After clearing the intermediate vertex sequence, the next clipping edge is selected, and the position judgment operation in step 201 and the vertex processing operation in step 202 are repeated. Iterative processing is performed in the order of processing all boundary edges of the clipping window until all boundary edges are processed.

[0055] After the iteration, check whether the coordinates of the first and last vertices in the final vertex sequence are consistent. If they are inconsistent, calculate the intersection of the line connecting the last vertex and the first vertex and the corresponding boundary edge to make the final vertex sequence form a closed polygonal region. This polygonal region is the trimmed underground target region. Based on this region, filter according to the preset spatial anomaly judgment criteria. The judgment criteria include the number of people in the region exceeding the safe carrying capacity of the region, the time that people stay in the region exceeding the prescribed duration, the region overlapping with the region with excessive gas concentration, and abnormal fluctuations in the operating parameters of the equipment in the region. Regions that meet any of the above criteria are included in the candidate anomaly region set in the spatial dimension, and each candidate anomaly region is numbered, and its location range and associated anomaly characteristics are recorded.

[0056] In a preferred embodiment of the present invention, step 3 includes:

[0057] Step 300: Process the candidate abnormal region set, form regional sequence data in time series order, and use a long short-term memory network to perform time series modeling on the regional sequence data to extract the implicit time series behavior feature vectors.

[0058] Step 301: Input the temporal behavior feature vector into the autoencoder for encoding and decoding operations to reconstruct the corresponding output features; and calculate the reconstruction error between the original temporal behavior features and the reconstructed features based on the output features.

[0059] Step 302: Perform abnormal pattern recognition based on the reconstruction error value, and mark the regions with errors exceeding the threshold as abnormal regions of the image to obtain abnormal regions of the image.

[0060] In this embodiment of the invention, by organizing regional state data according to time series and performing time-series modeling, the dynamic correlation characteristics of regional state over time can be effectively captured, which can more comprehensively reflect the evolution process of abnormal behavior downhole. By using a long short-term memory network to extract implicit temporal behavior feature vectors, the potential temporal correlations between personnel activities, equipment operation, and environmental parameters can be accurately mined, improving the ability to identify complex collaborative abnormal behaviors and solving the defects of existing systems that independently process multi-source data and lack correlation analysis. Through the encoding and decoding operations of the autoencoder and the calculation of reconstruction error, the core features can be focused on and feature differences can be quantified, reducing the impact of interference factors such as downhole dust and vibration on the data, reducing the interference of noise data on anomaly judgment, and improving the identification of anomaly features. Anomaly marking is performed based on reconstruction error and differential threshold, and thresholds are set in combination with historical data and regional safety level, making the anomaly judgment of high-risk areas more stringent and low-risk areas more flexible, effectively reducing false alarms and missed alarms, and improving the pertinence and reliability of anomaly early warning.

[0061] In this embodiment, when applied in a specific application, the specific implementation process of step 300 above, which processes the candidate abnormal region set and performs time-series modeling to extract the implicit time-series behavioral feature vector, is as follows: the candidate abnormal region set in the spatial dimension is sorted out item by item, the timestamps corresponding to the status data of each region recorded by the downhole monitoring system are retrieved, and the status data of the same candidate abnormal region at different times are arranged in order from early to late according to the timestamps.

[0062] Each time point is divided at fixed intervals, with the interval duration set according to the rhythm of underground operations. The corresponding data includes the real-time number of personnel identified by high-definition camera equipment in the area, as well as their status information such as whether they are wearing safety helmets and carrying work tools; status parameters such as the rotation speed, temperature, vibration frequency, and current operating mode of the coal mining machine and conveyor uploaded by the equipment operation monitoring terminal; monitoring data such as gas concentration, carbon monoxide concentration, temperature, humidity, dust concentration, and wind speed collected by environmental sensors; and personnel positioning beacons recording the movement trajectory segments of the workers in the area, which include continuous position coordinates and movement direction.

[0063] The resulting regional sequence data is divided into preset time windows, the length of which is set based on the typical duration of the abnormal behavior. Data within each window is sequentially associated in chronological order. A Long Short-Term Memory (LSTM) network is used to process the regional sequence data within each window in stages. First, the correlation features between the first and second time nodes are extracted, including whether changes in personnel numbers conform to the flow patterns during shift changes; whether changes in coal mining machine speed are consistent with previous start-up or shutdown states; and whether fluctuations in gas concentration are within the reasonable range of influence of changes in the operating status of local ventilation fans.

[0064] Next, the cumulative correlation features between each subsequent time node and all preceding nodes are extracted sequentially. These include whether the direction of personnel movement trajectories at multiple consecutive nodes is consistent with the roadway direction; whether the temperature change trend of the conveyor matches the increase or decrease in conveying volume; and whether the dust concentration fluctuates accordingly with the change in the working intensity of the coal mining machine. These correlation features are integrated into three categories: personnel behavior, equipment status, and environmental parameters, forming a time-series behavioral feature vector. The feature value of each dimension in the vector corresponds to the specific manifestation of a certain type of correlation feature, such as the degree of consistency between personnel trajectory and roadway direction, the degree of matching between equipment temperature and load, and the fluctuation range of environmental parameters with equipment operation.

[0065] In this embodiment, when applied in a specific application, the specific implementation process of step 301 above, in which the autoencoder performs encoding and decoding operations and calculates the reconstruction error between the original temporal behavior features and the reconstructed features, is as follows: The extracted temporal behavior feature vector is input into the encoding part of the autoencoder. During the encoding process, based on the priority of underground safety management, the feature importance weight is preset. Among them, the behavioral features of personnel entering dangerous areas, the environmental features of sudden increases in gas concentration, and the state features of sudden changes in equipment vibration frequency have higher weights than other conventional parameter fluctuation features.

[0066] Based on these weights, the core features that have the greatest impact on anomaly detection are selected. At the same time, the feature dimensions are compressed, and the number of dimensions after compression is determined according to the type of core feature, forming coded features. The coded features are input into the decoding part of the autoencoder. During the decoding process, the features are expanded in reverse according to the feature weights at the time of encoding, and the original dimensional information corresponding to each core feature is gradually restored. During the restoration process, the normal value range of each parameter in the well is referenced to ensure that the reconstructed features conform to the actual operation scenario. Finally, the output features that are consistent with the dimensions of the original time-series behavioral feature vector are reconstructed.

[0067] The original time-series behavioral feature vector is compared with the reconstructed output features dimension by dimension. For each dimension, the absolute difference between the original feature value and the reconstructed feature value is calculated. For example, the original value for the personnel quantity dimension is 8, and the reconstructed value is 3, with a difference of 5; the original value for the gas concentration dimension is 0.09%, and the reconstructed value is 0.04%, with a difference of 0.05%; the original value for the equipment vibration frequency dimension is 30 Hz, and the reconstructed value is 45 Hz, with a difference of 15 Hz. Then, the absolute differences of all dimensions are added together and divided by the total number of dimensions to obtain the reconstruction error value reflecting the overall difference.

[0068] In this embodiment, when applied in a specific way, the specific implementation process of step 302 above, which identifies abnormal patterns based on reconstruction error values ​​and marks abnormal areas in the image, is as follows: Based on the time series data corresponding to historical abnormal events in the mine over the past three months, historical abnormal events include personnel illegally entering unventilated areas, equipment operating at excessive temperatures, and gas concentrations approaching alarm values. The distribution range of reconstruction errors under normal conditions is statistically analyzed, and combined with the preset thresholds for the safety levels of different areas, the threshold for the tunneling face as a high-risk area is set relatively low, the threshold for the transport roadway as a medium-risk area is set moderately, and the threshold for the rest area as a low-risk area is set relatively high.

[0069] The reconstruction error value calculated in step 301 is compared with the preset threshold of the corresponding area. If the error value exceeds the threshold, such as the error value of a certain time window of the tunneling face exceeding the set low threshold, it is determined that there is an abnormal mode in the area within the time period of the corresponding time window. The area determined to be abnormal is marked, and the spatial coordinate range of the area is recorded. The coordinate range is determined based on the polygon vertices of the underground coordinate system. The time window in which the abnormality occurred is recorded, and the time window is accurate to the start second and end second. The reconstruction error value and the specific dimensions that exceed the threshold are recorded. For example, the difference in the gas concentration dimension accounts for the largest proportion, followed by the difference in the number of personnel dimension, forming a detailed record of the abnormal area in the image.

[0070] Finally, all marked areas are summarized in chronological order of occurrence of the anomalies to form an image anomaly area set. Each area is labeled with the corresponding anomaly type, which includes abnormal personnel behavior such as not moving along the prescribed route, abnormal equipment status such as abnormal conveyor speed, and abnormal environmental parameters such as excessive dust concentration.

[0071] In a preferred embodiment of the present invention, step 4 includes:

[0072] Step 400: Obtain the image space and coordinate points of the abnormal region in the image;

[0073] Step 401: Divide the image space into non-overlapping trapezoidal units to construct a planar subdivision structure;

[0074] Step 402: Use the coordinates of the abnormal areas in the image as query points, perform point location query operation in the planar subdivision structure, and determine the specific trapezoidal unit and its corresponding polygonal grid area where each query point is located by traversing the trapezoidal diagram.

[0075] Step 403: Determine the specific spatial location and regional affiliation of the abnormal region in the polygon grid based on the point positioning query results, extract regional features based on the spatial positioning results, and generate the located regional features containing spatial location information.

[0076] In this embodiment of the invention, by accurately acquiring the image space and coordinate points of the abnormal image region, including boundary vertices, edge feature points, and internal key sampling points, precise raw data support is provided for subsequent spatial positioning. The image space is divided into non-overlapping trapezoidal units that match the cross-sectional features of the underground trapezoidal roadway. The constructed planar subdivision structure better conforms to the actual underground spatial morphology. This division method takes into account both the pixel distribution of the image space and the physical structure of the roadway, ensuring both the regularity and non-overlapping nature of the unit division and improving the adaptability of the spatial subdivision to the underground working area. By using the coordinate points of the abnormal image region as query points to perform positioning operations in the planar subdivision structure, combined with the trapezoidal units… The ordered traversal method can efficiently determine the specific trapezoidal unit and corresponding polygonal grid area to which each query point belongs. Based on the point positioning results, the specific spatial location and regional affiliation of the abnormal area are determined. By extracting spatial features including geometric shape, area, relative position with surrounding equipment, and safety control level, these features are fused with the original features of the abnormal area to generate the regional features after positioning. This achieves a deep integration of spatial location information and abnormal features. It not only determines the specific work zone and associated area where the abnormality occurred, but also provides rich spatial context information for the comprehensive analysis of abnormal behavior. This helps to accurately judge the impact range and risk level of the abnormality, and improves the pertinence of abnormal early warning and decision support capabilities.

[0077] In this embodiment, when applied in a specific application, the specific implementation process of step 400 above, obtaining the image space and coordinate points of the image abnormal area, is as follows: retrieve the monitoring video frame or image data corresponding to the image abnormal area. The image data comes from high-definition camera equipment at a specific location underground, such as the operation scene captured in real time by a fixed-point camera at the face of the tunneling face, or the environmental image collected by a mobile inspection camera in the transport tunnel according to a preset route.

[0078] Determine the two-dimensional image space corresponding to the image data. This space uses pixels as the basic unit. Its width range is the number of pixels covered by the horizontal field of view of the camera, such as 1920 pixels for the horizontal field of view of the tunnel face camera; its height range is the number of pixels covered by the vertical field of view, such as 1080 pixels for the vertical field of view of the transport tunnel camera; extract the coordinate points of abnormal areas in the image, including boundary vertices, edge feature points, and internal key sampling points.

[0079] Boundary vertices are identified by image edge detection technology, which identifies the corner points of the abnormal region's contour, such as the inflection points of the contour of an area where people illegally gather or the corner points of a coal mining machine operating abnormally. Edge feature points are selected along the abnormal region's contour at preset pixel intervals, such as selecting one point every 50 pixels to ensure complete coverage of the entire contour. Internal key sampling points are selected from representative locations within the abnormal region, such as the center of a gathering of people or the corresponding pixel point at the highest point of an abnormal equipment index. All coordinate points are recorded with the top left corner of the image as the origin and a pixel coordinate system with the horizontal X-axis pointing to the right and the vertical Y-axis pointing downwards. Each coordinate point is represented by a horizontal pixel value (e.g., X=850) and a vertical pixel value (e.g., Y=420).

[0080] In this embodiment, when applied in a specific application, the specific implementation process of step 401 above, which constructs the planar subdivision structure, is as follows: Based on the pixel size of the image space and the actual geometric parameters of the underground roadway, the size parameters of the trapezoidal unit are set; wherein, the length of the upper base of the trapezoid corresponds to the actual width of the top of the roadway (e.g., 3 meters), and is converted to pixel length (e.g., 300 pixels) according to the conversion ratio between pixels and actual size (e.g., 100 pixels corresponds to 1 meter); the length of the lower base corresponds to the actual width of the bottom of the roadway (e.g., 4 meters), and is converted to 400 pixels; the height corresponds to the actual height of the roadway (e.g., 2.5 meters), and is converted to 250 pixels, ensuring that the shape of the trapezoidal unit is consistent with the cross-sectional characteristics of the underground trapezoidal roadway.

[0081] Starting from the top-left pixel (X=0, Y=0) in the image space, the first row of trapezoidal units is divided horizontally: the top left endpoint of the first trapezoidal unit is (0, 0), the top right endpoint is (300, 0), the bottom right endpoint is (400, 250), and the bottom left endpoint is (100, 250). The top left endpoint of adjacent trapezoidal units is aligned with the top right endpoint of the previous unit (X coordinate incremented by 300 pixels), and the bottom left endpoint is aligned with the bottom right endpoint of the previous unit (X coordinate incremented by 300 pixels), ensuring no gaps or overlaps between units. After completing the first row, the process continues vertically downwards. The starting Y coordinate of the top of the next row of trapezoidal units is the bottom Y coordinate of the previous row (250 pixels). Subsequent rows are divided according to the same rules until the entire image space is covered (Y coordinate reaches 1080 pixels). Each trapezoidal unit is assigned a unique identifier, such as T-row number-column number (e.g., T-1-3 represents the third unit in the first row), forming a planar subdivision structure.

[0082] In this embodiment, when applied in a specific application, the specific implementation process of step 402 above, determining the specific trapezoidal unit and its corresponding polygonal grid region where each query point is located, is as follows: Each coordinate point in the image anomaly region is treated as an independent query point, and the retrieval function of the planar subdivision structure is invoked; starting from the initial trapezoidal unit (T-1-1) of the planar subdivision structure, the coordinates (X...) of the query point are... q Y q Compare X with the four boundaries of the current trapezoidal element: determine X q Is it located between the X-coordinates of the left and right endpoints of the bottom base of this cell? q Whether it is located between the upper and lower Y coordinates of the unit and inside the left and right sides (judged by the tilt direction of the left and right sides; the left side tilts upward and to the right from the lower left end point to the upper left end point, and the right side tilts downward and to the right from the upper right end point to the lower right end point).

[0083] If the query point is not in the current cell, it moves to the adjacent cell in the order from left to right and from top to bottom (e.g., if the current cell is T-1-1, it moves to T-1-2; if the cell is at the end of the row, it moves to the beginning of the next row, T-2-1). The comparison is repeated until a specific trapezoidal cell containing the query point is found. Based on the identifier of the cell (e.g., T-2-4), a preset mapping table is queried. This table records the correspondence between each trapezoidal cell and the polygonal grid area, including the grid area name (e.g., No. 2 working zone of the tunnel face), the area boundary coordinates, the work group to which it belongs, etc., thereby determining the polygonal grid area corresponding to the trapezoidal cell.

[0084] In this embodiment, when applied in a specific application, the specific implementation process of step 403 above, the specific spatial location and regional affiliation of the abnormal region in the polygonal grid, and the generation of the located regional features containing spatial location information is as follows: Integrate the polygonal grid region information where all query points are located, count the number of query points contained in each region, if a region contains more than 80% of the query points, it is determined to be the main affiliation region of the image abnormal region, and the remaining regions with a proportion of less than 20% are associated regions, and determine the overall distribution range of the abnormal region in the polygonal grid and the adjacency relationship between regions, such as the main region and the associated region being adjacent to each other on the left and right.

[0085] Based on the attribute data of the grid region, spatial features are extracted: the geometric shape is described by the coordinates of the polygon vertices of the region, such as a quadrilateral formed by (X1, Y1) to (X4, Y4); the area is calculated by multiplying the number of trapezoidal units contained in the grid region by the actual area of ​​a single unit, such as each trapezoidal unit corresponding to 7.5 square meters; the relative position with surrounding equipment is calculated by converting the coordinate difference into actual distance, such as the horizontal distance to the coal mining machine is the difference between the X coordinate of the query point and the X coordinate of the coal mining machine divided by the conversion ratio (100 pixels / meter); the orientation relationship is determined by the Y coordinate difference. (Yq greater than the equipment's Y coordinate indicates the front, otherwise it indicates the rear); the safety control level is determined according to the area type (e.g., the tunneling face is level A, the transport roadway is level B); these spatial features are fused with the original features of the abnormal area in the image (e.g., the abnormal type is personnel not wearing safety helmets, duration is 8 minutes) to generate the located area features, including the grid area number (e.g., G-2-2), spatial coordinate range (X1-X2, Y1-Y2), distance from the coal mining machine (5 meters), safety level (level A), abnormal type and duration, etc.

[0086] In a preferred embodiment of the present invention, step 5 includes:

[0087] Step 500: Project the localized region features into query vector, key vector, and value vector, and calculate the relevance score between the query vector and the key vector to generate an attention weight matrix;

[0088] Step 501: The value vector is weighted and summed using the attention weight matrix to obtain the weighted image feature representation;

[0089] Step 502: Align and stitch the weighted image features and temporal behavior features along the feature dimension, and perform nonlinear transformation and dimension mapping on the stitched features through a multilayer perceptron to obtain the transformed features.

[0090] Step 503: Standardize the transformed features using feature normalization techniques to obtain multimodal feature data.

[0091] In this embodiment of the invention, by projecting the located regional features into query vectors, key vectors, and value vectors, spatial positioning information can be transformed into a computable vector form, providing a foundation for feature correlation analysis. The attention weight matrix automatically mines the importance of different dimensions of information in the regional features, making the model more focused on spatial features that play a key role in anomaly detection, thus improving the feature's specificity and discriminative power. Using the attention weight matrix to perform weighted summation of the value vectors allows for dynamic adjustment of regional features based on feature importance, strengthening the expression of high-weight features and weakening redundant low-weight information, making the generated weighted image features better reflect the core spatial attributes of the anomaly region. The weighted image features are then compared with temporal behavioral features... By aligning and stitching the features dimensionally, spatial and temporal features are organically integrated, breaking down the barriers between their independent processing and enabling the model to simultaneously consider the spatial distribution and temporal evolution of anomalies. Through nonlinear transformation and dimensional mapping using a multilayer perceptron, the fused features can be converted into a higher-dimensional feature space more suitable for anomaly identification, enhancing the feature's ability to represent complex anomaly patterns. By standardizing the transformed features using feature normalization techniques, differences in numerical range and units among different modal features can be eliminated, ensuring that multimodal features are on the same scale and preventing a certain type of feature from dominating the model's judgment due to excessively large values. The standardized multimodal feature data has more stable distribution characteristics, which can improve the stability of subsequent model training and inference.

[0092] In this embodiment, when applied in a specific application, the specific implementation process of step 500 above, querying the vector, key vector, and value vector, and calculating the correlation score between the query vector and the key vector to generate the attention weight matrix, is as follows: Obtain the location-defined regional features, which include the grid number of the abnormal region, such as G-2-2 corresponding to the No. 2 working zone of the tunneling face; the spatial coordinate range, such as X1=500, Y1=300 to X2=800, Y2=600 corresponding to a range of 10 to 15 meters from the face of the underground tunneling face; the safety level, such as Level A corresponding to a high-risk area of ​​the tunneling face; the anomaly type, such as excessive gas concentration; the distance to associated equipment, such as a horizontal distance of 5 meters from the coal mining machine; and the duration of the anomaly, such as 12 minutes, etc.

[0093] Projection is performed according to preset feature projection rules, which define specific dimensions for each feature's corresponding query vector, key vector, and value vector. The query vector's dimension focuses on the association between safety level and anomaly type, such as the combined feature of excessive gas concentration in a Class A area. The key vector's dimension corresponds to the attribute identifier of each feature; for example, grid number G-2-2 corresponds to the attribute of working zone 2 of the tunneling face, spatial coordinate range corresponds to the specific location characteristics of the underground tunneling face, safety level A corresponds to the high-risk area attribute, and the anomaly type excessive gas concentration corresponds to anomalies in environmental parameters. The attributes include the spatial relationship between the distance to the associated equipment and the coal mining machine, and the duration of the abnormality corresponding to the duration of the gas exceeding the standard. The dimension of the value vector contains the specific values ​​of each feature, such as the numerical identifier of the grid number 2-2, the specific pixel value of the spatial coordinates X=500, Y=300 to X=800, Y=600, the quantification value of the safety level 1 (1 for level A, 2 for level B), the quantification value of the abnormality type 3 (3 for gas concentration exceeding the standard, 1 for personnel not wearing safety helmets), the actual distance to the associated equipment in meters 5, and the duration of the abnormality in minutes 12.

[0094] The relevance score is calculated by comparing the matching degree of each dimension in the query vector and the key vector. If the query vector focuses on the excessive gas concentration in the A-level area, the matching degree with the combination of the A-level area attribute and the gas concentration exceeding the standard attribute in the key vector is scored as 90 points, the matching degree with the combination of the A-level area attribute and the attribute of personnel not wearing safety helmets is scored as 40 points, and the matching degree with the combination of the B-level area attribute and the attribute of gas concentration exceeding the standard is scored as 60 points. According to the relevance scores of all dimensions, weight values ​​are allocated proportionally. In the total score of 190 points, 90 points account for 47.4%, 40 points account for 21.1%, and 60 points account for 31.5%, forming an attention weight matrix. Each element in the matrix corresponds to the weight percentage of the corresponding feature dimension.

[0095] In this embodiment, when applied in a specific application, the specific implementation process of step 501 above, which involves weighting and summing the value vectors using the attention weight matrix to obtain the weighted image feature representation, is as follows: Take out the attention weight matrix generated in step 500 above. The weight values ​​of each feature dimension in the matrix are: 47.4% for the security level and anomaly type association dimension, 21.1% for the security level and other anomaly types association dimension, and 31.5% for other security levels and target anomaly types association dimension.

[0096] These weight values ​​are associated with the feature values ​​of the corresponding dimensions in the value vector. For the value vector, the feature value corresponding to the dimension of safety level and anomaly type is the product of 1 (level A quantification value) and 3 (gas concentration exceeding the standard quantification value), multiplied by a weight of 47.4% to get 1.422; the feature value corresponding to the dimension of safety level and other anomaly type is the product of 1 and 1 (personnel not wearing safety helmets quantification value), multiplied by a weight of 21.1% to get 0.211; the feature value corresponding to the dimension of other safety level and target anomaly type is the product of 2 (level B quantification value) and 3, multiplied by a weight of 31.5% to get 1.89. The sum of these three weighted feature values ​​is 1.422 + 0.211 + 1.89 = 3.523, which is the comprehensive feature value sequence. This sequence is the weighted image feature representation, in which the safety level and target anomaly type feature with higher weights occupy a more significant numerical proportion in the sequence.

[0097] In this embodiment, when applied in a specific application, the specific implementation process of step 502 above, which aligns and stitches the weighted image features and temporal behavior features in the feature dimension, and performs nonlinear transformation and dimension mapping on the stitched features through a multilayer perceptron, is as follows: Extract the weighted image feature representation obtained in step 501, which includes the spatial correlation features of the abnormal area, such as grid number G-2-2, distance from the coal mining machine 5 meters, and safety level A; extract the temporal behavior features generated in step 300 above, which include personnel movement trends, such as moving deeper into the tunnel face at three consecutive time nodes; equipment status change patterns, such as the coal mining machine speed decreasing from 300 rpm to 150 rpm; and environmental parameter fluctuations, such as the gas concentration increasing from 0.05% to 0.09%.

[0098] The two types of features are aligned in dimensions. Image features contain 5 dimensions, and time-series features contain 5 dimensions. The security level dimension of image features and the device operation mode dimension of time-series features are grouped into security attribute dimensions. The associated device distance dimension of image features and the personnel movement distance dimension of time-series features are grouped into spatial movement dimensions. The missing time dimension in image features is supplemented by including the anomaly duration. The missing area identifier dimension in time-series features is supplemented by including the grid number, so that both types of features have 5 dimensions. The aligned features are then concatenated in the order of security attribute, spatial movement, anomaly type, time duration, and area identifier to form a combined feature sequence.

[0099] The combined feature sequence is input into a multilayer perceptron. The first layer filters and retains features that are strongly correlated with anomaly detection, such as Class A in safety attributes and equipment slowdown, 5 meters in spatial movement and personnel moving deeper, excessive gas concentration in anomaly type, 12 minutes in time duration, and G-2-2 in area identification. The middle layer converts 5 meters and 10 meters (personnel movement distance) in the spatial movement dimension into a relative value of 0.5 (5 meters / 10 meters), and 12 minutes and 30 minutes (standard duration) in the time duration dimension into a relative value of 0.4 (12 / 30). The last layer maps the feature dimensions from 5 to 8 target dimensions to obtain the transformed features.

[0100] In this embodiment, when applied in a specific application, the specific implementation process of standardizing the transformed features in step 503 is as follows: Feature normalization technology is used to process the transformed features from step 502. First, the numerical ranges of the eight dimensions are statistically analyzed. The first dimension, security attribute values, are distributed between 0.3 and 0.9; the second dimension, spatial movement relative values, are distributed between 0.2 and 0.8; and the remaining dimensions each have different ranges. All dimensions are mapped to the standard interval of 0 to 1 through linear adjustment. For the first dimension, the current value is calculated by subtracting 0.3 and then dividing by 0.9 minus 0.3 to obtain the adjusted value. For the second dimension, the current value is calculated by subtracting 0.2 and then dividing by 0.8 minus 0.2 to obtain the adjusted value. The remaining dimensions are calculated in the same way.

[0101] The adjusted features are standardized, and the average value of each dimension is calculated. For example, the average value of the first dimension is 0.6, and the average value of the second dimension is 0.5. The average value of the dimension is subtracted from each feature value of each dimension, so that the value of the first dimension fluctuates around 0, and the value of the second dimension fluctuates around 0. After processing, multimodal feature data is obtained. This data integrates features of spatial positioning, temporal behavior, and abnormal attributes. The values ​​of each dimension are all between -0.5 and 0.5, and are evenly distributed. It can be directly used for the comprehensive judgment of subsequent abnormal behavior.

[0102] In a preferred embodiment of the present invention, step 6 includes:

[0103] Step 600: Extract line segment representations of behavioral trajectories from multimodal feature data, calculate the relative orientational relationship between line segments of different behavioral trajectories, and generate an orientational feature matrix;

[0104] Step 601: Based on the orientation feature matrix and the principle of cross-section experiment, determine the intersection probability between line segments and screen potential intersecting line segment pairs.

[0105] Step 602: Calculate the precise intersection parameters for potential intersecting line segment pairs to determine the intersection coordinates and intersection type;

[0106] Step 603: Based on the intersection coordinates and intersection type, analyze the intersection distribution pattern and time series characteristics to identify abnormal crossover behavior patterns and generate an abnormal interaction feature description vector.

[0107] In this embodiment of the invention, by converting behavioral trajectories into line segment representations and calculating relative orientation relationships, a structured extraction of trajectory spatial correlations from multimodal features is achieved. The orientation feature matrix intuitively quantifies the spatial relationships such as the orientation and positional arrangement of different trajectory line segments, providing accurate spatial feature basis for subsequent intersection judgment. Based on the principle of cross-sectional experiments, potential intersecting line segment pairs are screened, which can quickly eliminate obviously non-intersecting line segment combinations in massive trajectory data, reducing the processing load of subsequent accurate calculations. Geometric topological rules are used to efficiently narrow the analysis scope, avoiding resource waste caused by indiscriminate full calculation, improving the efficiency of trajectory intersection analysis, and adapting to the limited computing power of downhole edge nodes. Precise parameter calculations of potential intersecting line segments can accurately obtain the spatial coordinates and intersection type of the intersection point. By quantifying the intersection point position and intersection shape, fine-grained features are provided to distinguish between normal and abnormal intersections, improving the accuracy of behavioral interaction analysis. Combining the intersection point distribution pattern and time series features, multi-dimensional identification of abnormal intersection behavior is achieved. By analyzing the density, distribution area, and temporal correlation of intersection points, abnormal patterns such as multiple people illegally gathering and equipment and personnel trajectory conflicts can be accurately identified. The generated abnormal interaction feature description vector integrates key information in the spatial and temporal dimensions, providing a structured decision-making basis for subsequent early warning and improving the pertinence of downhole safety management.

[0108] In this embodiment, when specifically applied, the specific implementation process of step 600 above—extracting the line segment representation of the behavioral trajectory, calculating the relative azimuth relationship between different behavioral trajectory line segments, and generating the azimuth feature matrix—is as follows: Behavioral trajectory information is extracted from multimodal feature data. This data originates from underground personnel positioning beacons, i.e., UWB positioning tags deployed on miners' safety helmets, which send location data every 30 seconds; and from mobile device monitoring terminals, i.e., the positioning module carried by the inspection robot, which records trajectory points every minute, including the real-time position coordinates of the workers and the inspection robot, using the underground coordinate system as a reference, with the X-axis along the roadway direction and the Y-axis perpendicular to the roadway direction, along with the corresponding timestamp, accurate to the second. Each behavioral trajectory is divided into continuous line segments according to the time interval. The time interval is set according to the trajectory movement speed: the walking speed of personnel is approximately 0.8 meters per second, and the line segments are divided at 1-minute intervals, corresponding to a movement distance of approximately 48 meters; the movement speed of the inspection robot is approximately 0.3 meters per second, and the line segments are divided at 5-minute intervals, corresponding to a movement distance of approximately 90 meters. Each line segment is represented by the coordinates of its starting point and ending point. The starting point is the position coordinates at a certain moment, such as the coordinates of a person at time t1: X1=100 meters, Y1=20 meters. The ending point is the coordinates at a later time, such as the coordinates of t1+1 minutes: X2=148 meters, Y2=20 meters. The same applies to the trajectory line segments of equipment. For example, the coordinates of a robot at time t2: X3=50 meters, Y3=15 meters, and the coordinates of t2+5 minutes: X4=140 meters, Y3=15 meters.

[0109] Calculate the relative orientation between different line segments: The direction angle is based on the underground due north direction and the same direction as the Y-axis. The extension direction is determined by the coordinate difference between the end point and the starting point of the line segment. For example, the personnel line segment extends from (100, 20) to (148, 20) along the positive X-axis, and the angle with due north is 90 degrees. The angle between line segments is the acute angle formed by the extension directions of the two line segments. For example, the angle between the 90-degree personnel line segment and the 90-degree robot line segment is 0 degrees, and the angle between the 0-degree robot line segment and another 0-degree line segment extending along the positive Y-axis is 90 degrees.

[0110] The positional arrangement is calculated by determining the midpoint coordinates of each line segment. For example, the midpoint of the personnel line segment is (124, 20), and the midpoint of the robot line segment is (95, 15). The position is determined by comparing the midpoint coordinates. The line segment with the larger Y-coordinate is in front, and the line segment with the larger X-coordinate is on the right. Therefore, the personnel line segment is to the right and in front of the robot line segment. These positional relationships are organized into a positional feature matrix according to the line segment pairs. The rows and columns of the matrix correspond to personnel line segment 1, personnel line segment 2, robot line segment 1, etc. Each element records the direction angle difference of the corresponding line segment pair. For example, the direction angle difference between personnel line segment 1 and robot line segment 1 is 0 degrees, the included angle is 0 degrees, and the positional arrangement description is: personnel line segment 1 is to the right and in front of robot line segment 1.

[0111] In this embodiment, when applied in a specific application, the specific implementation process of step 601, screening potential intersecting line segment pairs, is as follows: The orientation feature matrix generated in step 600 is retrieved, and all trajectory line segment pairs involved in the matrix are selected, such as personnel line segment 1 and robot line segment 1, personnel line segment 1 and personnel line segment 2, etc.; based on the principle of cross-sectional experiments, the intersection probability between line segments is determined. For each pair of line segments (line segment a, line segment b), the two endpoints a1 (Xa1, Ya1), a2 (Xa2, Ya2) of line segment a and the two endpoints b1 (Xb1, Ya2) of line segment b are taken. Given lines (Yb1, b2, Xb2, Yb2), determine whether a1 and a2 are located on opposite sides of line segment b. Along the extension direction of line segment b from b1 to b2, compare the relative positions of a1, a2, and line segment b. If the Y-coordinate of a1 is less than the Y-coordinate of line segment b at the corresponding X-axis, it is on the left; if it is greater, it is on the right. If a1 is on the left, then a2 must be on the right, or vice versa. Simultaneously, determine whether b1 and b2 are located on opposite sides of line segment a, using the same method. If both conditions are met, the line segment is considered a potential intersecting pair.

[0112] In this embodiment, when applied in a specific application, the specific implementation process of step 602 above, which determines the intersection coordinates and intersection type, is as follows: For the potential intersecting line segment pairs selected in step 601, obtain the starting point and ending point coordinates of each line segment (e.g., for line segment m, the starting point coordinates are m1 (50, 30) and the ending point coordinates are m2 (150, 30); for line segment n, the starting point coordinates are n1 (100, 10) and the ending point coordinates are n2 (100, 50)).

[0113] The direction is determined by the coordinate parameters of the line segments. Line segment m, from (50, 30) to (150, 30), is a horizontal line (Y=30), and line segment n, from (100, 10) to (100, 50), is a vertical line (X=100). The intersection point of the two lines is (100, 30). We check if this coordinate is within the coordinate range of the two line segments—the X range of line segment m is 50-150, and Y=30; the X range of line segment n is 100, and the Y range is 10-50. Therefore, the intersection point (100, 30) is valid. The intersection type is then determined. If the intersection point (100, 30) is not an endpoint of the two line segments and there is no overlap, it is determined to be an intersection; if the intersection point is the endpoint (150, 30) of line segment m and is within the Y range (10-50) of n, it is determined to be an endpoint intersection; if the overlap between line segment m (50, 30)-(150, 30) and line segment p (80, 30)-(120, 30) is 40 meters (accounting for 40% of the length of m (100 meters) and 100% of the length of p (40 meters), it is determined to be an overlapping intersection because the overlap exceeds 50% of the length of p.

[0114] In this embodiment, when applied in a specific application, step 603 above analyzes and generates an abnormal interaction feature description vector in the following way: analyzes the intersection distribution pattern, divides the underground into a 10×10 meter polygonal grid area, such as dividing the tunnel face into G1-G5 and the transport roadway into Y1-Y4, counts the number of intersections in each grid, such as the G3 grid with X=100-110 meters and Y=20-30 meters having 12 intersections, with a total of 20 intersections, accounting for 60%, and is determined to be a concentrated distribution of intersections.

[0115] Analyze the time series characteristics, extract the timestamp of each intersection point, and count the number of intersections per unit time. For example, if there are 5 intersections in the G3 grid within 10 minutes, an average of 2 intersections per hour during normal working hours, or intersections concentrated in the non-working hours of 2:00-5:00 AM, totaling 8 intersections, it is determined to be an anomaly in time characteristics. Identify abnormal intersection behavior patterns and compare them with historical normal patterns. For example, the Y2 grid in the transport tunnel has 3 intersections per hour during the working hours, and most of them are intersections between equipment and personnel. Currently, the high-risk tunneling face of the G3 grid has 5 intersections in 10 minutes, and these are intersections between personnel and the trajectory of equipment that has not stopped, which is determined to be an anomaly. Generate an abnormal interaction feature description vector, including grid area number G3, occurrence time 2:00-5:00, intersection type identifier 3 representing intersection, and 2.7 intersections per hour, etc.

[0116] In a preferred embodiment of the present invention, step 7 includes:

[0117] Step 700: Establish a multi-dimensional analytical model based on the CRIME theoretical framework;

[0118] Step 701: Extract key node data from the abnormal interaction feature description vector through a multi-dimensional analytical model, and generate a spatial segmentation model based on the key node data using a Voronoi graph construction algorithm to obtain Voronoi units and their spatial distribution.

[0119] Step 702: Identify the topological characteristics of potential collusion networks by analyzing the spatial proximity relationships of the Voronoi units;

[0120] Step 703: Define system security boundary conditions based on the topological features, and emit detection rays based on the ray normal boundary;

[0121] Step 704: Calculate the number of intersections between the ray and the boundary, determine whether the behavior point has crossed the boundary based on parity, and generate a boundary crossing behavior identifier.

[0122] Step 705: Construct a behavior path network based on topological features and boundary crossing behavior identifiers, and calculate the path deviation.

[0123] Step 706: Comprehensively analyze topological features, boundary crossing indicators, and path deviation to generate a multi-dimensional analysis report of abnormal behavior, so as to realize the detection of abnormal behavior of large-scale intelligent agents.

[0124] In this embodiment of the invention, a multi-dimensional analytical model is constructed based on CRIME theory, breaking through the limitations of single-level analysis and enabling the analysis of the deep logic of abnormal behavior, thus solving the problem that traditional systems struggle to identify the motivations behind behavior. Key node data is extracted, and a spatial segmentation model is generated using Voronoi diagrams to accurately characterize the spatial distribution and proximity relationships of nodes, resolving the problem of ambiguous node associations in traditional spatial analysis. Analyzing the spatial proximity relationships of Voronoi units can identify potential collusion network topologies, overcoming the limitation of traditional methods in detecting multi-person collaborative violations. Safety boundaries are defined based on topological features and transformed into detectable geometric boundaries using ray tracing, determining spatial constraint standards and resolving the problem of ambiguous safety boundaries. Boundary crossings are determined by the parity of ray intersections, improving the accuracy of boundary crossing detection and making it applicable to complex underground spaces. A behavioral path network is constructed and deviation is calculated to quantify the difference from normal patterns, addressing the deficiency of traditional methods in ignoring overall path anomalies. A multi-dimensional report is generated by integrating topological features, boundary crossing indicators, and path deviation, enabling comprehensive detection of abnormal behavior in large-scale intelligent agents. This report not only includes the result of whether an anomaly exists but also provides decision-making information such as why it is abnormal, which nodes are involved, and risk propagation paths, solving the problem that traditional systems only output binary results and lack decision support.

[0125] In this embodiment, when applied in a specific application, the specific implementation process of step 700 above, establishing a multi-dimensional analytical model, is as follows: The model dimensions are determined. Specifically, combining the motivation, opportunity, means, capability, and result in the CRIME theory, dimensions are selected and optimized based on the specific characteristics of the downhole working environment: The capability dimension is excluded because all downhole personnel have passed pre-job qualification review, and this dimension is not a key factor for real-time anomaly detection; the means and result dimensions are merged into a collaborative interaction dimension, given that downhole abnormal behavior often manifests as multi-person collaborative operation, this dimension is needed to link interactive behavior with subsequent risk impacts; a new environmental association dimension is added because downhole environmental risks (such as excessive gas levels or abnormal roof pressure) directly trigger abnormal behavior, this dimension is needed to bind the association between hazardous spatial areas and abnormal behavior.

[0126] Ultimately, three core analytical dimensions were identified: behavioral motivation (to answer why the anomaly occurred), collaborative interaction (to answer how the anomaly was implemented), and environmental correlation (to answer where the anomaly occurred). The behavioral motivation dimension aims to quantify personnel's tendency to violate regulations, distinguishing between proactive violations and passive errors in abnormal behavior; the collaborative interaction dimension aims to analyze the correlation strength between intelligent agent nodes (including personnel and equipment) to identify single-person violations or multi-person collusion; and the environmental correlation dimension aims to assess the spatial correlation between abnormal behavior and dangerous areas to determine the risk level (low / medium / high).

[0127] Secondly, specific implementations were carried out for the three dimensions mentioned above, with each implementation process following the logic of data collection, rule formulation, and quantitative scoring. For the behavioral motivation dimension, the implementation focused on quantifying personnel's tendency to violate regulations: In the data collection phase, basic data of the target personnel for the past three months was retrieved from the underground personnel management system. This basic data included violation records (covering behaviors such as not wearing a safety helmet, unauthorized absence from duty, and improper operation of equipment, with each record including the time, location, and type of violation), safety training records (covering whether they participated in safety training and their assessment grades for the past three months, with assessment grades divided into excellent, qualified, and unqualified), and reward and punishment records (covering whether they participated in safety training and their assessment grades for the past three months). (Warnings or fines for safety issues, or commendations for safety performance) are issued. Supplementary data on the recent work status of target personnel is obtained through interviews with team leaders (such as whether there is fatigue, emotional fluctuations, etc.). In the rule-making and enforcement stage, the violation level is first classified into minor, moderate, and severe violations. Minor violations refer to behaviors that do not directly cause safety accidents, such as not wearing a safety helmet or not wearing standard work clothes. Moderate violations refer to behaviors such as unauthorized entry into unauthorized areas or brief unauthorized operation of equipment (operation time ≤ 5 minutes). Severe violations refer to behaviors that may cause safety accidents, such as unauthorized operation of high-risk equipment such as coal mining machines and conveyors or intentional obstruction of sensors.

[0128] Next, the weighted number of violations is calculated, with 1 point for a minor violation, 2 points for a moderate violation, and 3 points for a severe violation. If the target personnel have no violation records in the past 3 months, the base score is 1 point. If there are violation records, weighted scores are added, with a maximum weighted score of 5 points. The score is then adjusted based on training and rewards / penalties. If the personnel have participated in safety training in the past 3 months and achieved excellent results, 0.5 points are deducted; if they fail the assessment, 0.5 points are added. If they are warned or fined for safety issues, 0.5 points are added; if they are commended for safety performance, 0.5 points are deducted. Finally, the motivation level is determined based on the adjusted score: 1-2 points correspond to low violation tendency (probably passive error, such as accidentally pressing an equipment button), 3-4 points correspond to medium violation tendency (possibly active violation, such as simplifying safety procedures to save time), and 5 points correspond to high violation tendency (probably intentional violation, such as stealing equipment parts).

[0129] For the collaborative interaction dimension, its implementation focuses on analyzing the correlation strength between intelligent agent nodes: In the data acquisition stage, the collected data includes personnel interaction data and equipment interaction data. Personnel interaction data is obtained from the underground wireless communication system and personnel positioning beacon system, specifically including voice communication records between the target personnel and other personnel (including call time, call duration, and call recipients) and trajectory overlap data (by extracting the movement trajectories of the target personnel and other personnel, calculating the overlap duration when the time difference in the same area is ≤5 minutes). Equipment interaction data is obtained from the equipment operation monitoring terminal, specifically including equipment operation records of different personnel (statistically checking whether different personnel operate the same equipment within the same time period of ≤10 minutes, such as operating the control panel of the transport machine sequentially) and data transmission records between sensors and equipment (checking for abnormal data sharing, such as a terminal carried by a certain personnel being highly associated with a terminal of an unauthorized device). (Frequency of interaction); In the rule-making and enforcement phase, firstly, statistical units and basic thresholds are set, with 1 hour as the statistical unit. The frequency of interaction within each statistical unit is calculated. This frequency of interaction is the sum of the number of voice communications, the number of trajectory overlaps, and the number of times devices are operated jointly. Next, the interaction frequency levels are divided: 0-2 interactions per hour is low frequency (no obvious signs of collaboration, possibly normal work communication); 3-5 interactions per hour is medium frequency (potential collaboration exists, further observation is needed); 6 interactions per hour or more is high frequency (probably collaborative operation, such as multiple people working together to obstruct monitoring equipment). Finally, the type of collaboration is determined based on the interaction frequency. If the target only interacts with 1 person at a medium or high frequency, it is judged as two-person collaboration; if the target interacts with 2 or more people at a medium or high frequency, it is judged as multiple-person collusion; if all of the target's interactions are low frequency, it is judged as single-person behavior.

[0130] Regarding the environmental correlation dimension, its implementation focuses on assessing the correlation between abnormal behavior and hazardous areas: In the data collection phase, hazardous areas are first delineated, including dynamic and static hazardous areas. Dynamic hazardous areas are selected from real-time data from environmental sensors, with selection criteria including methane concentration ≥ 0.5% (exceeding the safety threshold of 0.08%, requiring real-time monitoring), roof pressure ≤ 80 MPa (insufficient support strength, posing a risk of collapse), and dust concentration ≥ 10 mg / m³. 3 (It can impair visibility and easily cause respiratory diseases.) Static danger zones are defined according to the underground engineering design drawings, including fixed danger zones such as equipment maintenance areas during non-maintenance periods and a 5-meter radius around high-voltage electrical boxes.

[0131] Simultaneously, location data is extracted from personnel positioning beacons and equipment monitoring terminals. This location data represents the real-time coordinates of the target personnel or equipment, updated every 30 seconds with accuracy down to the meter. In the rule-making and enforcement phase, the real-time distance is first calculated. Using the boundary of the danger zone as a benchmark, the shortest straight-line distance between the target personnel or equipment and the boundary of the danger zone is calculated by comparing the target coordinates with the boundary coordinates of the danger zone. Then, distance levels and basic scores are assigned: 0-5 meters is considered high-risk (already entered or adjacent to the danger zone, extremely high risk), scoring 3 points; 5-10 meters is considered medium-risk (close to the danger zone, risk of accidental entry), scoring 2 points; and above 10 meters is considered low-risk (far from the danger zone, lower risk), scoring 1 point. The score is then adjusted based on the dwell time: if the target stays within the 0-5 meter range for ≥5 minutes, 0.5 points are added; if the stay is ≥10 minutes, 1 point is added; if the target only briefly passes through the 0-5 meter range (dwell time ≤1 minute), 0.5 points are deducted.

[0132] Finally, the models are integrated and validated to form a closed-loop analysis system. In the multi-dimensional data fusion stage, a mapping relationship between dimensions, data, and scores is established. Quantitative scores for behavioral motivation, collaborative interaction, and environmental association are integrated into a unified analysis result, ensuring that data from each dimension are interconnected and mutually corroborative. In the model validation and optimization stage, 10 confirmed underground anomaly cases from the past month (including 5 cases of single-person violations and 5 cases of multi-person collusion) are selected. The case data is input into the model to calculate and analyze the results, which are then compared with the actual anomalies. If the model's judgment does not match the actual situation (e.g., the model judges it as single-person behavior, but it is actually two-person collaboration), adjustments are made. Adjust the judgment thresholds for corresponding dimensions (e.g., reduce the lower limit of the medium frequency of collaborative interaction from 3 times to 2 times). If the environmental correlation score does not match the actual risk, recalibrate the distance calculation method (e.g., consider the roadway obstruction factor and use the actual path distance instead of the straight-line distance). In the model output stage, set a unified output standard. The output results must include motivation judgment (e.g., target person A has a high tendency to violate regulations and is suspected of actively colluding), collaboration type (e.g., 3 people colluding, the core person is A, and the auxiliary persons are B and C), and environmental risk (e.g., the target has entered a high-risk gas area, stayed for 12 minutes, and the risk level is extremely high). Ensure that the output results can provide a definite and comprehensive decision-making basis for the handling of abnormal behavior underground.

[0133] In this embodiment, when applied in a specific application, the specific implementation process of step 701 is as follows: using the multi-dimensional analytical model established in step 700, key node data is extracted from the abnormal interaction feature description vector related data; the initiator is the miner who first triggers the abnormal warning, that is, the first miner (worker number K012) identified by the positioning beacon data; the participants are miners who interact with the initiator ≥ 5 times / hour during the abnormal period, and worker numbers K008 and K035 are selected through voice communication records and equipment collaborative operation logs; the associated equipment is the coal mining machine (number M003) and the positioning beacon (number B107) in the area, and their identification information is obtained from the equipment operation monitoring terminal.

[0134] Record the underground spatial coordinates of each node, based on the underground roadway axis coordinate system. For example, the X-axis extends along the main roadway, and the Y-axis is perpendicular to the main roadway. The coordinates of K012 are obtained through UWB positioning beacons as X=200 meters and Y=50 meters, the coordinates of K008 are X=205 meters and Y=52 meters, the coordinates of K035 are X=198 meters and Y=48 meters, the coordinates of M003 are obtained through the device's built-in GPS module as X=180 meters and Y=45 meters, and the coordinates of B107 are X=202 meters and Y=51 meters.

[0135] A Voronoi diagram construction algorithm is used, with these nodes as generators, to calculate the perpendicular bisector of each node to all other nodes. For example, the perpendicular bisector of K012 and K008 is the perpendicular bisector of the line segment connecting the two points, and the direction of the bisector is determined by the difference in coordinates between the two points. The polygonal region formed by the intersection of all bisectors is the Voronoi cell corresponding to each node. The boundary coordinates of the Voronoi cell of K012 are (199, 49), (203, 51), and (201, 53), with an area of ​​about 12 square meters, covering 30% of the area of ​​the G2 grid (X=195-205 meters, Y=45-55 meters) of the tunnel face. The boundary coordinates, area size, and distribution position of all Voronoi cells in the underground grid are statistically analyzed to form a spatial distribution record.

[0136] In this embodiment, when applied in a specific application, the specific implementation process of step 702 above, identifying the topological characteristics of a potential collusion network, is as follows: For the Voronoi cells obtained in step 701, analyze the spatial proximity relationship between cells; determine shared boundaries: by comparing the boundary line coordinates of each cell, the Voronoi cells of K012 and K008 have overlapping boundary line segments (1.2 meters in length), and are determined to be adjacent; the boundary line segments of cells of K012 and K035 overlap by 0.8 meters, and are determined to be adjacent; the cells of M003 and B107 have no overlapping boundaries, and are determined to be non-adjacent. Calculate the straight-line distance between cell centers: take the geometric center coordinates of each cell, the straight-line distance between the center of cell K012 (200, 50) and the center of cell K008 (205, 52) is approximately 5.39 meters, which is greater than 5 meters, and is determined to be non-near neighbors; the distance between the centers of cells K012 and K035 is approximately 2.83 meters, which is less than 5 meters, and is determined to be near neighbors. Identify the topological characteristics of potential collusion networks: Units K012, K008, and K035 are adjacent, and through video surveillance and communication records, it was verified that the three individuals had a continuous 12-minute interaction record (10:05-10:17), thus identifying this group of nodes as the core node group; track the interaction frequency between adjacent units, K012 and K035 interact 7 times per hour, thus identifying them as the main association path; with the core node group as the center, all units with direct or indirect proximity relationships (including part of the area of ​​M003) are delineated as the collaboration range, forming a topological characteristic record.

[0137] In this embodiment, when applied in a specific application, the specific implementation process of step 703 above, defining the system security boundary conditions and the detection ray, is as follows: Based on the topological features identified in step 702, the system security boundary conditions are defined; the geometric center of the core node group is (201, 50), and the security boundary is a circular area with a radius of 20 meters centered at this point (covering X=181-221 meters and Y=30-70 meters); the security boundary of the associated path (K012-K035) is a rectangular area of ​​3 meters on each side of the path, with the path starting point (198, 48) and ending point (200, 50), and the rectangle is 2.83 meters long and 6 meters wide.

[0138] The ray method is used to emit detection rays to the boundary. The starting point of the ray is the real-time position coordinates of each key node (such as the 10:10 coordinates of K012, X=201 meters, Y=51 meters). The emission direction is clockwise 0 degrees (due north), 30 degrees, 60 degrees...330 degrees, for a total of 12 rays. The length of each ray is set to 25 meters (5 meters beyond the maximum distance of the safety boundary) to ensure that the rays can cover the boundary range, so that the safety boundary is transformed from an abstract concept into a detectable geometric boundary, which can adapt to the complex underground roadway space.

[0139] In this embodiment, when applied in a specific application, the specific implementation process of generating the boundary crossing behavior identifier in step 704 is as follows: For each detection ray emitted in step 703, count the number of intersections with the safety boundary; the 0-degree ray emitted by K012 at 10:10 (along the positive Y-axis) starts from (201, 51) and intersects with the safety boundary (circle) of the core node group at (201, 70), recording 1 intersection point (odd number); the 30-degree ray intersects with the boundary at two points (entry and exit), recording 2 intersection points (even number).

[0140] The behavior point is determined to be outside the boundary based on parity: if the number of intersections of the 0-degree ray is odd, it means that the ray has passed from inside the boundary to outside the boundary, and K012 is determined to be outside the safe boundary at 10:10 (out of bounds); if the number of intersections of the 30-degree ray is even, it helps to verify the out-of-bounds result; for nodes that have crossed the boundary, an out-of-bounds behavior identifier is generated, and the identifier includes the node number K012, the out-of-bounds time 10:10:23, and the out-of-bounds boundary type core node group safe boundary.

[0141] In this embodiment, when applied in a specific application, the specific implementation process of step 705, which involves constructing a behavior path network and calculating the path deviation, is as follows: Based on the topological features of step 702 and the boundary crossing behavior identifier of step 704, a behavior path network is constructed; historical location coordinates of K012 are collected at 30-second intervals using a positioning beacon: 10:00 (200, 50), 10:00:30 (201, 50), 10:01 (202, 51)...10:10 (201, 51), and connected in chronological order to form the actual behavior path; the centerline of the associated path (the straight line from 198, 48 to 200, 50) is set as the preset safe path, which is determined according to the route specified in the downhole operation procedure.

[0142] The path deviation is calculated as follows: distance deviation is the sum of the vertical distances between each point on the actual path and the safe path (e.g., the sum of the distances between 10 points is 8 meters) divided by the total path length (10 meters), resulting in 0.8 meters / meter; angle deviation is the sum of the angles between each segment of the actual path and the safe path (e.g., the sum of the angles between 5 segments is 75 degrees) divided by the number of segments (5), resulting in 15 degrees / segment; the preset thresholds are 0.5 meters / meter for distance deviation and 10 degrees / segment for angle deviation, which are judged as significant deviations.

[0143] In this embodiment, when applied in a specific application, the specific implementation process of generating a multi-dimensional analysis report of abnormal behavior in step 706 is as follows: comprehensively analyze the topological characteristics of step 702, the boundary crossing behavior identifier of step 704, and the path deviation of step 705; set an anomaly judgment threshold, and the situation where the core node group size exceeds 5, the number of boundary crossings exceeds 3 times per hour, and the path deviation exceeds the threshold is judged as a high anomaly level; the situation where two of the conditions are met is judged as a medium anomaly level; the situation where one condition is met is judged as a low anomaly level; the current core node group size is 3, the number of boundary crossings is 4 times per hour, and the path deviation is 0.8 meters / meter and 15 degrees / segment, which meet two anomaly conditions and is judged as a medium anomaly level.

[0144] A multi-dimensional analysis report is generated based on the anomaly level. The report includes a list of anomaly nodes (K012, K008, K035), a collusion network topology diagram, a boundary crossing location distribution diagram (marking 4 boundary crossing points in the G2 grid), path deviation curves, and anomalies in the risk level assessment, enabling the detection of abnormal behavior of large-scale intelligent agents.

[0145] like Figure 2 As shown, embodiments of the present invention also provide an abnormal behavior detection system for large-scale intelligent agents, comprising:

[0146] The data acquisition module is used to collect multi-source heterogeneous raw data from distributed intelligent agent nodes, and to preprocess the collected raw data to obtain a preprocessed standardized dataset.

[0147] The extraction module is used to extract the internal regions that intersect with the cropping window from the preprocessed standardized dataset to obtain a set of candidate abnormal regions in the spatial dimension; extract temporal behavior features from the set of candidate abnormal regions, and calculate the reconstruction error using an autoencoder to capture abnormal patterns in the data distribution and obtain abnormal regions in the image.

[0148] The fusion module is used to construct a planar subdivision structure based on the abnormal regions in the image, spatially locate the abnormal regions in the image through point query operations, determine the specific location and region affiliation of the abnormal regions in the polygon grid, and obtain the localized region features; the localized region features are weighted and the correlation weights between features are calculated to obtain weighted image features, and the weighted image features are fused with temporal behavior features in a fine-grained manner to form unified multimodal feature data;

[0149] The identification module is used to analyze abnormal crossover patterns in the behavioral trajectories of multimodal feature data and identify abnormal interaction feature description vectors in the behavioral trajectories. It performs multi-dimensional analysis on the abnormal interaction feature description vectors to achieve abnormal behavior detection of large-scale intelligent agents. The multi-dimensional analysis includes motivation analysis, operation method identification, and collusion network construction.

[0150] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0151] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0152] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0153] The multi-source heterogeneous data (including personnel positioning, video images, operation records, etc.) collected by the method and system described in this application are used solely for detecting abnormal behaviors in underground safety production. Collection and use comply with relevant laws and regulations and require the authorization and consent of the data subject. User privacy information (such as personnel identity, real-time location, and behavioral trajectory) involved in the patented technical solution must be protected through encrypted storage, access control, data anonymization, and other technical means to prevent information leakage, tampering, or misuse. The applicant and inventors of this application shall not be liable for any privacy information security issues caused by illegal data acquisition by third parties, improper authorization by users, or force majeure. This application is merely a presentation of a technical solution; in practical applications, privacy protection mechanisms need to be improved in conjunction with specific scenarios to ensure compliance with industry privacy standards and data security requirements.

[0154] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting abnormal behavior of a large-scale agent, characterized by, The method comprises: Collecting multi-source heterogeneous raw data from distributed agent nodes, and performing data preprocessing on the collected raw data to obtain a standardized data set after preprocessing; Extracting an internal region intersecting with a clipping window from the standardized data set after preprocessing to obtain a candidate abnormal region set in the spatial dimension; Extracting a time series behavior feature from the candidate abnormal region set, and calculating a reconstruction error of the time series behavior feature through a self-encoder to capture an abnormal pattern in the data distribution, and obtaining an image abnormal region; According to the image abnormal region, a planar subdivision structure is constructed, the image abnormal region is spatially positioned through a point query operation, the specific position and region attribution of the abnormal region in the polygon grid are determined, and a positioned region feature is obtained; The positioned region feature is weighted and processed, the correlation weight between the features is calculated, the weighted image feature is obtained, and the weighted image feature and the time series behavior feature are fused in fine granularity to form a unified multi-modal feature data; An abnormal cross-mode of the behavior trajectory of the multi-modal feature data is analyzed, and an abnormal interaction feature description vector in the behavior trajectory is identified; Multi-dimensional analysis is performed on the abnormal interaction feature description vector to realize abnormal behavior detection of large-scale agents, and the multi-dimensional analysis includes motive analysis, operation means identification and conspiracy network construction, including: establishing a multi-dimensional analysis model based on a CRIME theory framework; extracting key node data in the abnormal interaction feature description vector through the multi-dimensional analysis model, and generating a spatial partitioning model by a Voronoi diagram construction algorithm according to the key node data to obtain a Voronoi cell and its spatial distribution; by analyzing the spatial proximity relationship of the Voronoi cell, the topological structure features of the potential conspiracy network are identified; the system security boundary condition is defined based on the topological structure features, and the detection rays are emitted based on the ray method boundary; by calculating the intersection number of the rays and the boundary, it is judged whether the behavior point is out of boundary according to the parity, and an out-of-bound behavior identifier is generated; According to the topological structure features and the out-of-bound behavior identifier, a behavior path network is constructed, and the path deviation degree is calculated; the topological features, the out-of-bound identifier and the path deviation degree are comprehensively analyzed to generate a multi-dimensional analysis report of abnormal behavior, so as to realize abnormal behavior detection of large-scale agents. 2.The method of claim 1, wherein, Extracting an internal region intersecting with a clipping window from the standardized data set after preprocessing to obtain a candidate abnormal region set in the spatial dimension, comprising: Defining a polygon boundary of the clipping window to determine the geometric constraint condition of the clipping region; According to the geometric constraint condition of the clipping region, a sequence of vertices of the polygon to be clipped is obtained, and an output vertex list is initialized, each boundary edge of the clipping window is traversed in order, and the current vertex list is taken as input to perform visibility judgment to obtain the position relationship of the vertex relative to the current clipping edge; According to the position relationship of the vertex relative to the current clipping edge, a preset vertex processing rule is generated, and a new intermediate vertex sequence is generated according to the vertex processing rule; The generated intermediate vertex sequence is taken as the input vertex list of the next clipping edge, and all clipping edges are iteratively processed to obtain a final vertex sequence to form a clipped polygon region, and a candidate abnormal region set in the spatial dimension is generated. 3.The method of claim 2, wherein, Temporal behavior features are extracted from the candidate abnormal region set, and the temporal behavior features are used to calculate reconstruction errors by a self-encoder to capture abnormal patterns in the data distribution, and an image abnormal region is obtained, including: The candidate abnormal region set is processed to form a region sequence data in time sequence order, and a long short-term memory network is used to model the region sequence data in time sequence to extract an implicit temporal behavior feature vector; The temporal behavior feature vector is input into the self-encoder for encoding and decoding operations to reconstruct the corresponding output features, and the reconstruction error between the original temporal behavior features and the reconstructed features is calculated according to the output features; According to the reconstruction error value, an abnormal pattern is identified, and regions with errors exceeding a threshold are marked as image abnormal regions to obtain the image abnormal region.

4. The method of claim 3, wherein, According to the image abnormal region, a planar subdivision structure is constructed, and the image abnormal region is spatially positioned by a point query operation to determine the specific position and region attribution of the abnormal region in the polygon grid, and a positioned region feature is obtained, including: An image space and a coordinate point of the image abnormal region are obtained; The image space is divided into non-overlapping trapezoidal cells to construct a planar subdivision structure; The coordinate point of the image abnormal region is taken as a query point, and a point positioning query operation is performed in the planar subdivision structure to determine the specific trapezoidal cell and the corresponding polygon grid region of each query point by traversing the trapezoidal graph; According to the point positioning query result, the specific spatial position and region attribution relationship of the abnormal region in the polygon grid are determined, and the region feature is extracted based on the spatial positioning result to generate a positioned region feature containing spatial position information.

5. The method of claim 4, wherein, The positioned region feature is weighted and processed to calculate the correlation weight between the features to obtain a weighted image feature, and the weighted image feature and the temporal behavior feature are fused in fine granularity to form a unified multi-modal feature data, including: The positioned region feature is projected into a query vector, a key vector and a value vector, and the correlation score between the query vector and the key vector is calculated to generate an attention weight matrix; The value vector is weighted and summed by the attention weight matrix to obtain a weighted image feature representation; The weighted image feature and the temporal behavior feature are aligned and spliced in the feature dimension, and the spliced feature is nonlinearly transformed and dimensionally mapped by a multilayer perceptron to obtain a transformed feature; The transformed feature is standardized by a feature normalization technique to obtain multi-modal feature data.

6. The method of claim 5, wherein, An abnormal intersection pattern of the behavior trajectory of the multi-modal feature data is analyzed to identify an abnormal interaction feature description vector in the behavior trajectory, including: Line segment representations of the behavior trajectory are extracted from the multi-modal feature data, the relative orientation relationship between different behavior trajectory line segments is calculated, and an orientation feature matrix is generated; According to the orientation feature matrix, the intersection possibility between line segments is judged based on the cross-standing experiment principle, and potential intersecting line segment pairs are screened. Precise intersection parameter calculation is performed on potential intersecting line segment pairs to determine intersection point coordinates and intersection types; According to the intersection point coordinates and the intersection types, the intersection point distribution mode and the time sequence characteristics are analyzed, and the abnormal intersection behavior mode is identified to generate an abnormal interaction feature description vector.

7. A large-scale agent-oriented abnormal behavior detection system, the system implements the method as claimed in any one of claims 1 to 6, characterized in that, Comprise: The acquisition module is used for collecting multi-source heterogeneous original data from the distributed intelligent agent nodes, and performing data preprocessing on the collected original data to obtain a standardized data set after preprocessing; The extraction module is used for extracting an internal region intersecting with a clipping window from the standardized data set after preprocessing to obtain a candidate abnormal region set in a spatial dimension; time sequence behavior features are extracted from the candidate abnormal region set, and the time sequence behavior features are calculated by a self-encoder to obtain a reconstruction error, capture an abnormal mode in a data distribution, and obtain an image abnormal region; The fusion module is used for constructing a planar subdivision structure according to the image abnormal region, performing spatial positioning on the image abnormal region through a point query operation, determining a specific position and region attribution of the abnormal region in a polygon grid, obtaining a region feature after positioning, performing weighted processing on the region feature after positioning, calculating a correlation weight between features, obtaining a weighted image feature, and performing fine-grained fusion on the weighted image feature and the time sequence behavior feature to form a unified multi-modal feature data; The identification module is used for analyzing an abnormal intersection mode of a behavior trajectory of the multi-modal feature data, and identifying an abnormal interaction feature description vector in the behavior trajectory. The abnormal interaction feature description vector is analyzed in multiple dimensions to realize abnormal behavior detection of a large-scale intelligent agent, and the multiple-dimensional analysis includes motive analysis, operation means identification, and conspiracy network construction.

8. A computing device, comprising: Comprise: One or more processors; A storage device is configured to store one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program which is executed by the processor to implement the method of any one of claims 1 to 6.

Images