Intelligent illegal behavior identification method and system based on multi-algorithm fusion

Abstract

The invention discloses an intelligent illegal behavior identification method and system based on multi-algorithm fusion, and relates to the technical field of image analysis. The method comprises the steps of collecting a region image sequence, identifying real-time personnel density information, adjusting the scale of a laser curtain wall region based on the personnel density information, forming a self-adaptive monitoring boundary, when a laser sensor detects an intrusion event, recording the intrusion moment and intrusion characteristics, calling an intrusion region image set, and obtaining the intrusion region image sequence; and analyzing the intrusion area image set by adopting an ensemble learning-based violation behavior recognition network group to obtain an interactive violation behavior recognition result, fusing the interactive violation behavior recognition result with the intrusion characteristics, introducing personnel density information to carry out environment adaptive compensation calculation, and outputting a final violation behavior recognition result. According to the invention, through fusion of image analysis, laser triggering and environment perception, high-robustness and high-accuracy intelligent identification of illegal behaviors in scenes such as subway security check is realized.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, specifically to a method and system for intelligent identification of violations based on multi-algorithm fusion. Background Technology

[0002] Subway security checks are a crucial link in ensuring public safety on rail transit, aiming to intercept various dangerous items. With urban development and continuously increasing subway passenger flow, long queues at security checkpoints are becoming increasingly common. Some passengers, seeking convenience, attempt to circumvent normal security procedures, with "passing bags" being particularly prevalent. This refers to passengers not sending their bags through the X-ray security scanner, but instead passing them directly from the side or top to a companion on the other side of the security area. This allows the bags to evade scanning, potentially allowing prohibited items concealed within to be brought into the subway, posing a serious security risk.

[0003] Currently, monitoring of subway security checkpoints primarily relies on a combination of manual duty and video surveillance review. Security personnel must simultaneously interpret the screens of the security scanners and maintain order on-site, making them highly susceptible to oversights due to fatigue or limited visibility during peak hours. Existing intelligent monitoring technologies based on video analytics mostly focus on detecting general scenarios such as overall crowd density, abnormal gatherings, left-behind items, or violent movements. For violations like "passing bags," which have specific patterns, relatively concealed actions, and appearances similar to normal item handling, the accuracy is often insufficient, resulting in high false alarm and false negative rates. Specifically, existing technologies struggle to accurately distinguish the subtle differences between passengers placing and retrieving bags normally from the scanner and illegally passing them. They also cannot reliably track the bag's path and determine whether it bypasses the core scanning area of ​​the security equipment in complex, crowded, and obstructed environments. The lack of efficient and robust technical solutions specifically designed for such violations leaves significant blind spots in subway security checks. Summary of the Invention

[0004] This invention addresses the technical problems of insufficient accuracy in identifying specific violations in subway security check areas, poor robustness in complex and crowded environments, and lack of effective dynamic adaptive adjustment capabilities in existing technologies. It provides a method and system for intelligent identification of violations based on multi-algorithm fusion.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, the present invention provides an intelligent identification method for violations based on multi-algorithm fusion, comprising: The system acquires a sequence of regional images of the target area collected by an image sensor, performs personnel density identification, and obtains personnel density information. Based on the personnel density information, a laser curtain wall area scale decision is made to obtain the laser curtain wall area scale, and the laser sensor is controlled to adjust the laser curtain wall to form the laser curtain wall area. When the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and intrusion characteristics are obtained, and the image set of the intrusion area is retrieved; Interactive violation behavior identification is performed on the intrusion area image set to obtain interactive violation behavior identification results. Combined with the intrusion features, the final violation behavior identification result is obtained, wherein compensation processing is performed based on personnel density information and intrusion features.

[0006] Secondly, this invention provides an intelligent identification system for violations based on multi-algorithm fusion, comprising: The image acquisition and processing module is used to acquire regional image sequences of the target area collected by the image sensor, perform personnel density recognition, and obtain personnel density information. The laser curtain wall control module is used to make laser curtain wall area scale decisions based on the personnel density information, obtain the laser curtain wall area scale, control the laser sensor to adjust the laser curtain wall, and form the laser curtain wall area. The intrusion detection module is used to acquire the intrusion time and intrusion characteristics and retrieve the image set of the intrusion area when the laser sensor detects an intrusion in the laser curtain wall area; The violation behavior identification module is used to identify interactive violation behaviors in the intrusion area image set, obtain interactive violation behavior identification results, and combine them with the intrusion features to obtain the final violation behavior identification result, wherein compensation processing is performed based on personnel density information and intrusion features.

[0007] The beneficial effects of this invention are: Compared to existing technologies, this invention acquires real-time personnel density through image analysis and dynamically adjusts the monitoring scale of the laser curtain wall accordingly. This allows the system to adapt to changes in passenger flow, effectively reducing false alarms caused by environmental interference while ensuring detection accuracy. Laser sensors precisely capture the temporal and spatial characteristics of physical intrusions, providing objective and reliable evidence for subsequent analysis. After an event is triggered, the system employs a multi-algorithm fusion model to perform deep analysis on associated images. By integrating the judgment results of multiple recognition networks, the accuracy of identifying specific concealed violations such as "passing bags" is improved. Finally, by introducing personnel density and intrusion characteristics for dual compensation and correction of the recognition results, the system's decision thresholds under different levels of congestion and behavioral patterns are optimized. This achieves highly accurate and adaptive intelligent identification of violations in complex subway security check scenarios, improving the automation level and security control efficiency of the security check process. Attached Figure Description

[0008] Figure 1A flowchart illustrating the intelligent identification method for violations based on multi-algorithm fusion provided by this invention; Figure 2 This is a schematic diagram of the structure of the intelligent identification system for violations based on multi-algorithm fusion provided by the present invention.

[0009] In the attached diagram, the components represented by each number are as follows: Image acquisition and processing module 11, laser curtain wall control module 12, intrusion detection module 13, and violation recognition module 14. Detailed Implementation

[0010] Example 1, as Figure 1 As shown, this embodiment of the invention provides an intelligent identification method for violations based on multi-algorithm fusion, including: S10: Acquire the regional image sequence of the target area collected by the image sensor, perform personnel density recognition, and obtain personnel density information; Specifically, the system acquires a sequence of regional images of the target area collected by an image sensor, performs personnel density recognition, and obtains personnel density information, including: Take a sequence of regional images of the target area acquired by the image sensor, wherein the target area includes a first region and a second region, and the first region and the second region are separated by a dividing line; The image sequence of the region is input into the personnel density recognition plugin, and personnel density information is output.

[0011] First, a sequence of regional images of the target area is continuously acquired by an image sensor. An image sensor is a device that converts optical images into electronic signals, such as a surveillance camera deployed in a subway station. This image sensor is fixedly installed at a specific position above the security checkpoint and continuously acquires a stream of surveillance video of the target area at a preset constant frame rate, thus forming a continuous and ordered sequence of regional images in the time dimension. Specifically, the target area is the complete surveillance field of view covering the subway security checkpoint and its adjacent passages. This target area is clearly divided into two adjacent sub-areas by a physical or virtual dividing line, namely the first area and the second area. In a real-world scenario, the dividing line can be represented as a physical barrier used to separate passenger flow at the subway security checkpoint, clearly defining the pre-security checkpoint passage and the post-security checkpoint passage.

[0012] Therefore, each frame in the acquired regional image sequence fully contains real-time scene information of the channel to be inspected and the channels that have been inspected, and can synchronously reflect the dynamic distribution and behavior of people in the channels.

[0013] Furthermore, crowd density analysis is performed based on the acquired regional image sequences. Specifically, the acquired regional image sequences, including the first and second regions, are input in real time into a pre-trained crowd density recognition plugin. This plugin maps a complex functional relationship from image features to crowd density, and can analyze each input image frame, automatically identify and count the number or distribution characteristics of individuals appearing in the first and second regions, ultimately outputting a quantitative or graded crowd density information. This crowd density information can comprehensively reflect the real-time congestion level of the pre-security checkpoint and post-security checkpoint lanes, or the overall congestion level.

[0014] Specifically, the personnel density recognition plugin is trained and configured within the control system using the following steps: Based on historical monitoring data of the target area, collect a set of sample area images; The density of people in each sample region image is labeled to obtain a set of sample person density information; A personnel density recognition plugin was built based on a convolutional neural network. The personnel density recognition plugin is trained and tested under supervision using the sample region image set and the sample personnel density information set. After the test converges, it is embedded into the control system.

[0015] First, based on historical monitoring data of the target area, namely the subway security checkpoint, a large number of scene images containing different time periods and passenger flow conditions are systematically collected to form a diverse sample area image set. This sample area image set needs to cover various crowd density scenes from sparse to crowded to ensure that the model trained subsequently has broad adaptability.

[0016] Secondly, data annotation is performed. Specifically, for each image in the sample area image set, professionals or with the help of verified auxiliary annotation tools, based on predefined unified standards, accurately quantify or objectively classify the density of people presented in the image.

[0017] For example, a quantitative annotation method could be to label each visible human target in the image one by one and count their total number to obtain the accurate number of people in that frame. Alternatively, an optional hierarchical annotation method could be to divide the population density into three levels: "low density," "medium density," and "high density," based on a preset threshold range. For instance, an image with fewer than 10 people could be defined as low density, 10 to 30 people as medium density, and more than 30 people as high density. Through any one or a combination of these annotation methods, a set of sample population density information with accurate and realistic labels can be ultimately formed, corresponding one-to-one with the sample area image set.

[0018] Furthermore, a convolutional neural network was chosen as the basic architecture to build a people density recognition plugin. A convolutional neural network is a deep learning model specifically designed for processing image data. It automatically extracts spatial hierarchical features from images through multi-layer convolution and pooling operations, making it suitable for identifying and locating human targets in complex scenes.

[0019] Furthermore, using a set of sample region images and a set of sample personnel density information, the constructed convolutional neural network model undergoes end-to-end supervised training. During this process, the model continuously adjusts its internal parameters to learn the mapping relationship from the input image to the corresponding personnel density label. After training, the model's performance is evaluated using an independent test dataset until it converges. The convergence condition is pre-set based on the accuracy and stability requirements of the actual application scenario. For example, it can be set so that the model's average absolute error fluctuation on the test set is less than 2% over 20 consecutive training cycles, and the final average absolute error value is less than 2 people. Finally, the trained model is solidified as a personnel density recognition plugin and formally embedded into the software framework of the control system through an application programming interface (API) or dynamic link library, thereby realizing online and automatic personnel density analysis of real-time video streams.

[0020] For example, since there are highly complex and nonlinear visual correlations between the distribution, pose, occlusion and density assessment of people in an image, and convolutional neural networks have advantages in automatically extracting deep spatial features of images and handling such visual pattern recognition problems, convolutional neural networks are chosen as the infrastructure to build a people density recognition plugin.

[0021] Specifically, the main model of this personnel density recognition plugin consists of a feature extraction backbone network, a multi-scale feature fusion module, and a density regression output layer. The input layer receives a single-frame region image that has undergone size normalization and pixel standardization preprocessing. The feature extraction backbone network employs a residual network structure, such as ResNet34, pre-trained on the ImageNet dataset to leverage its powerful general feature extraction capabilities. Subsequent network layer parameters are fine-tuned during training to adapt to specific tasks. The multi-scale feature fusion module uses a spatial pyramid pooling structure to fuse feature maps output from different stages of the backbone network, enhancing the model's ability to perceive personnel targets at different scales. The output layer uses a fully connected layer and a linear activation function to ultimately map the fused high-dimensional features into a continuous real value representing personnel density, which can be an exact count or a normalized density level.

[0022] During training, key hyperparameters included an initial learning rate of 0.0001, a total training epoch count of 100, and 16 image samples per batch. The learning rate was set to ensure the stability of pre-trained weight fine-tuning, the number of training epochs ensured sufficient learning of the model on the task dataset, and the batch size was chosen to balance computational resource constraints with gradient update stability. The specific training data consisted of the aforementioned set of sample region images and their corresponding set of personnel density information, randomly divided into training, validation, and test sets in a 6:2:2 ratio.

[0023] Furthermore, image samples from the training set are used as input, with their corresponding personnel density labels serving as supervision signals. The model's weight parameters are iteratively optimized using a backpropagation algorithm in conjunction with a stochastic gradient descent optimizer. A smooth L1 loss function is employed to measure the deviation between the model's predicted density values ​​and the true labeled values. This loss function is relatively insensitive to outliers, contributing to training stability. The training process is monitored using a validation set. Training is terminated when the validation set loss function value does not show a decreasing trend for 10 consecutive training epochs, and the model's mean absolute error on the validation set reaches a preset convergence criterion, such as less than 2 people, thus obtaining a converged personnel density recognition model.

[0024] Finally, the collected regional image sequence is input into the configured personnel density recognition plugin, and the personnel density information is output.

[0025] S20: Based on the personnel density information, make a decision on the laser curtain wall area scale, obtain the laser curtain wall area scale, control the laser sensor to adjust the laser curtain wall, and form the laser curtain wall area; Specifically, based on the personnel density information, a laser curtain wall area scale decision is made to obtain the laser curtain wall area scale, and the laser sensor is controlled to adjust the laser curtain wall to form the laser curtain wall area, including: Obtain the average personnel density and the preset laser curtain wall area scale; Based on the personnel density information and average personnel density, the preset laser curtain wall area scale is adjusted by decision calculation to obtain the laser curtain wall area scale; According to the laser curtain wall area scale, the laser sensor is controlled to adjust the laser curtain wall to form a laser curtain wall area of ​​the laser curtain wall area scale. The laser curtain wall area includes a first sub-laser curtain wall area and a second sub-laser curtain wall area that fall into the first area and the second area, which are divided by the interval line.

[0026] First, two key baseline parameters are obtained. The first is the average personnel density value obtained from historical data analysis or empirical configuration, which characterizes the congestion level of the target area under normal or typical conditions. The second is the preset laser curtain area size. The laser curtain area refers to an invisible detection area constructed by laser sensor transmitting and receiving devices in the physical space on both sides of the security checkpoint interval line. It represents the sensitive space range for the system to actively detect intrusion behavior. The preset laser curtain area size is the initial, unadjusted reference value of the monitoring physical boundary width of this invisible detection area.

[0027] Secondly, the current real-time identification of personnel density information is compared and analyzed with the aforementioned average personnel density, and the preset laser curtain wall area scale is dynamically adjusted to output a laser curtain wall area scale that adapts to the current flow of people.

[0028] For example, when the real-time personnel density is significantly higher than the average density, it indicates that the passageway is more congested, and people have limited space to move around. The probability of unintentionally approaching the spacing lines during normal passage increases. To avoid generating too many invalid alarms, the decision-making process might favor appropriately reducing the size of the laser screen area, i.e., narrowing the width of the monitoring sensitive zone, so that the system only triggers when an object is very close to the spacing line. This allows for a more relaxed and humane monitoring strategy in congested environments. Conversely, when the real-time personnel density is lower than the average density, the passageway is relatively spacious, and people have more room to move around. In this case, the decision-making process might favor increasing the size of the laser screen area, i.e., widening the width of the monitoring sensitive zone, so that any movement of objects near the spacing lines is closely monitored, enabling more stringent detection of potential unauthorized baggage handling.

[0029] Specifically, based on the personnel density information and average personnel density, the preset laser curtain wall area scale is adjusted through decision calculation to obtain the laser curtain wall area scale, including: The ratio of the personnel density information to the average personnel density is calculated and used as a density adjustment coefficient; The density adjustment coefficient is used to make decision calculations and adjustments to the preset laser curtain wall area scale to obtain the laser curtain wall area scale.

[0030] First, the real-time personnel density information is used as the numerator, and the preset average personnel density is used as the denominator. The ratio between the two is then calculated, and this ratio is defined as the density adjustment coefficient.

[0031] For example, if the real-time crowd density is 45 people and the average crowd density is 30 people, then the calculated density adjustment coefficient is 1.5. This density adjustment coefficient directly and quantitatively reflects the deviation of the current crowding level from the normal level. When the density adjustment coefficient is greater than 1, it indicates that the current crowd flow is higher than the average level and the environment is more crowded; when the density adjustment coefficient is less than 1, it indicates that the current crowd flow is sparse and lower than the average level.

[0032] Secondly, the calculated density adjustment coefficient is used to make scale adjustment decisions. Specifically, the density adjustment coefficient is used to make decision calculations and adjustments to the preset laser curtain wall area scale. Optionally, the preset laser curtain wall area scale is directly multiplied by the reciprocal of the density adjustment coefficient to output the final laser curtain wall area scale to be executed.

[0033] For example, when the density adjustment coefficient is greater than 1, indicating that the current personnel density is higher than the average level, its reciprocal is less than 1, so that the laser curtain wall area scale obtained after multiplication is smaller than the preset laser curtain wall area scale, thereby narrowing the monitoring range in crowded environments and reducing the probability of false triggering.

[0034] Finally, based on the calculated dimensions of the laser curtain wall area, the laser sensors are controlled to perform physical adjustments to the laser curtain wall. Specifically, the laser sensors are non-contact distance and position detection devices based on optical principles, deployed in the target area, directly above the security checkpoint intervals in the subway, to generate an invisible, linear array of detection beams within a defined spatial range to form an optical detection curtain wall. Laser sensors typically consist of pairs of cooperating laser emitter modules and laser receiver modules. Control commands can precisely adjust the beam emission angle of the laser emitter module or the effective sensing range and direction of the laser receiver module, thereby accurately constructing a laser curtain wall area in physical space whose width, position, and calculated dimensions match the desired laser curtain wall area dimensions.

[0035] Specifically, the final laser curtain wall area corresponds completely to the target area defined by the image in terms of spatial layout, and is also divided into two parts by physical interval lines: one part of the beam array falls into the first area, that is, the pre-security inspection channel, forming the first sub-laser curtain wall area; the other part of the beam array falls into the second area, that is, the post-security inspection channel, forming the second sub-laser curtain wall area.

[0036] Therefore, an invisible, variable-width laser detection barrier is formed on both sides of the security checkpoint interval line. When an object, such as a package being passed or a person's arm, intrudes into and blocks any sub-laser barrier area, the laser receiver detects the interruption of the optical path, thus determining that an intrusion has occurred.

[0037] S30: When the laser sensor detects an intrusion into the laser curtain wall area, acquire the intrusion time and intrusion characteristics, and retrieve the image set of the intrusion area; Specifically, when the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and intrusion characteristics are acquired, and an image set of the intrusion area is retrieved, including: When the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and the intruded sub-laser curtain wall area are obtained, wherein the intruded sub-laser curtain wall area is the first sub-laser curtain wall area and the second sub-laser curtain wall area; Continue monitoring to obtain the interactive intrusion time of other sub-laser curtain wall areas, calculate the time interval with the intrusion time as the intrusion feature, wherein if no other sub-laser curtain wall area intrusion is detected within a preset time range, it is determined that no laser curtain wall area intrusion has occurred, and monitoring continues. Based on the intrusion time and intrusion characteristics, retrieve the image set of the intrusion area.

[0038] First, when the laser sensor detects for the first time that the beam of any sub-laser curtain wall area is blocked, an intrusion event is immediately determined. The exact time of this event must be recorded as the intrusion time, and the specific sub-laser curtain wall area that was triggered must also be recorded. That is, it is clear whether the intrusion occurred in the first sub-laser curtain wall area or the second sub-laser curtain wall area, and it is recorded as the intruded sub-laser curtain wall area.

[0039] Because typical unauthorized baggage passing physically involves continuous interaction between individuals on both sides of the security checkpoint line—that is, the actions of someone passing an item on one side and someone receiving it on the other side are closely linked in time and space—this pattern is expected to cause the first and second sub-laser screen areas to be triggered sequentially within a short period. Based on this understanding, after the laser sensor first detects an intrusion into a sub-laser screen area on one side and records the moment of intrusion, continuous monitoring within a preset time range will be automatically initiated.

[0040] Specifically, the preset time range refers to a finite time interval calculated from the moment of the first intrusion. The specific value of this interval is preset based on the statistical analysis results of the maximum reasonable interval between actions on both sides in typical packet delivery behavior patterns, for example, it can be set to 3 seconds.

[0041] Within a preset time range, continuous monitoring is conducted to detect whether subsequent intrusion events occur in the opposite sub-laser curtain wall area. If an intrusion is detected in the opposite sub-laser curtain wall area within the preset time range, the exact moment of this subsequent intrusion is recorded and defined as the interactive intrusion moment. The absolute time difference between the initial intrusion moment and the interactive intrusion moment is calculated, and this difference is defined as the intrusion characteristic of this composite intrusion event. This intrusion characteristic parameter quantifies the correlation between two independent physical triggering events from a temporal dimension, providing a basis for determining whether subsequent actions constitute a two-way interaction with spatial cross-regional reach.

[0042] Conversely, if no intrusion event is detected in the opposite sub-laser wall area within the preset time range, it is determined that there is only one isolated unilateral trigger. This situation typically corresponds to non-interactive scenarios such as pedestrians unintentionally approaching the fence, personal belongings accidentally swinging, or small items falling, and the behavioral pattern does not conform to the typical characteristics of unauthorized baggage handling. Therefore, this unilateral trigger is determined not to constitute a valid intrusion threat to the laser wall area, and the system resumes continuous synchronous monitoring of both sub-laser wall areas.

[0043] Finally, once a valid intrusion event and its corresponding intrusion characteristics are confirmed, the intrusion area image set is retrieved from the continuously recorded video stream of the image sensor based on the recorded intrusion time and the calculated intrusion characteristics.

[0044] Specifically, based on the intrusion time and intrusion characteristics, a set of images of the intrusion area is retrieved, including: Retrieve the region image corresponding to the intrusion time, and the region image of the interaction intrusion time within the intrusion feature after the intrusion time, to obtain an intrusion region image set.

[0045] Based on the recorded initial intrusion time, the system locates and extracts one or more frames of the region corresponding to that intrusion time from the continuously stored video stream data of the image sensor. Furthermore, since the intrusion features characterize the time interval between the initial intrusion and subsequent interactive intrusions, the system will continue to retrieve image sequences from the video stream within a time interval from the initial intrusion time to the interactive intrusion time, based on this time interval parameter. This sequence contains one or more key images recording the interactive intrusion time.

[0046] Finally, by retrieving images from the two aforementioned time points and the processes in between, the resulting intrusion area image set completely covers the entire suspicious behavior time period, from the initial trigger on one side to the interaction trigger on the other side. This intrusion area image set can provide a data foundation for subsequent image-based violation identification.

[0047] S40: Perform interactive violation behavior recognition on the intrusion area image set to obtain interactive violation behavior recognition results. Combine the intrusion features to obtain the final violation behavior recognition results, wherein compensation processing is performed based on personnel density information and intrusion features.

[0048] Specifically, interactive violation behavior identification is performed on the intrusion area image set to obtain interactive violation behavior identification results. Combined with the intrusion features, the final violation behavior identification result is obtained, including: Obtain a network cluster for identifying violations based on ensemble learning, wherein the network cluster includes multiple networks for identifying violations, and the training data of each network is not completely the same. The input features in supervised training are a set of images of the intrusion area of ​​the sample, and the supervision labels are the results of identifying the violation of the sample. The intrusion area image set is input into the violation behavior recognition network group, and multiple violation behavior recognition results are output, wherein the violation behavior recognition results include violation or no violation; Calculate the percentage of violations within the multiple violation identification results to obtain the violation rate, which is used as the interactive violation identification result; Based on the results of the interaction violation identification and the intrusion characteristics, the final violation identification result is obtained.

[0049] First, a pre-built violation detection network ensemble is invoked. This ensemble is built based on ensemble learning and contains multiple violation detection networks with identical or similar structures. Each violation detection network is trained independently using slightly different training datasets, for example, by sampling different subsets from the overall sample using bootstrapping, thus introducing model diversity. During the training phase, the input to each violation detection network is a set of images of sample intrusion areas, and its supervision label is the violation detection result of the corresponding image sequence as determined by human judgment, such as labeled "violation" or "non-violation".

[0050] For example, in subway security check scenarios, the "passing bag" behavior manifests in image sequences as the rapid transfer of objects across physical barriers and hand interactions. This visual pattern is complex, easily obscured, and highly similar to normal baggage handling, posing a challenge for stable recognition using a single model. Ensemble learning, by combining the predictions of multiple base learners, effectively improves the model's generalization ability and decision robustness. Therefore, we chose to construct a violation behavior recognition network based on ensemble learning.

[0051] Specifically, this violation detection network cluster consists of multiple parallel violation detection networks, each employing a 3D convolutional neural network as its basic architecture to process intrusion region image sets with a temporal dimension. The input layer receives preprocessed and temporally sampled image sequences. The main body of the network contains multiple 3D convolutional layers and 3D pooling layers to jointly extract motion and appearance features from video clips from a spatiotemporal perspective. Fully connected layers are located at the end of the network to integrate the extracted spatiotemporal features. The output layer uses the Softmax activation function to map the features to probability distributions for "violation" and "non-violation" categories. Each independent violation detection network maintains a consistent architecture, but differences in their model parameters are achieved through differentiated training processes.

[0052] During training, key hyperparameters were uniformly set as follows: initial learning rate of 0.0005, 120 training iterations, and batch size of 8 image sequences. The learning rate was set to consider the stability of 3D convolutional network training, the number of iterations ensured the model fully learned spatiotemporal patterns, and the batch size accommodated the high memory requirements of video data. The specific training data came from a large set of images of intrusion areas captured from historical surveillance events. Each image set was accompanied by manually determined violation identification results as supervisory labels. Multiple overlapping but not identical subsets were randomly selected with replacement from the total training dataset using a bootstrap sampling method.

[0053] Furthermore, each sampling subset is used independently to train a violation detection network. During the training of a single network, the sample image sequence from the sampling subset is used as input, and the corresponding violation detection results are used as supervision signals. Backpropagation and the Adam optimizer are employed to iteratively optimize the network parameters. Cross-entropy loss is used as the loss function to measure the difference between the network's predicted probability distribution and the true labels. After all networks have been trained independently, they collectively constitute a violation detection network swarm. This swarm can learn the spatiotemporal features of packet delivery behavior from multiple slightly different data perspectives. Its collective decision-making mechanism effectively reduces the risk of overfitting to a single noisy pattern or specific scene, thus exhibiting stronger generalization performance and more stable recognition accuracy when facing new intrusion event images.

[0054] Furthermore, the set of images of the intrusion area to be determined is simultaneously input into each of the violation recognition networks in the violation recognition network group. Each violation recognition network independently performs forward inference and outputs a binary preliminary recognition result, that is, whether the behavior shown in the current image sequence is "violation" or "not violation". Next, multiple violation recognition results are aggregated, the number of results judged as "violation" is counted, and the proportion of this number to the total number of networks is calculated to obtain a quantified violation rate. This violation rate reflects the collective decision-making tendency based on visual information and is defined as the interactive violation recognition result.

[0055] Finally, based on the obtained interactive violation identification results and intrusion characteristics, the final violation identification results are obtained.

[0056] Specifically, based on the interaction violation identification results and intrusion characteristics, the final violation identification result is obtained through processing, including: Obtain the baseline intrusion characteristics of the violation; Calculate the similarity between the intrusion feature and the baseline intrusion feature to obtain the intrusion violation rate; The fusion violation rate is calculated based on the intrusion violation rate and the violation rate within the interaction violation identification results; Based on the personnel density information and average personnel density, the fusion violation rate is compensated and calculated to obtain the compensated violation rate, which serves as the final violation identification result.

[0057] First, the baseline intrusion characteristics of the violation are obtained. These baseline intrusion characteristics are quantified time interval values ​​representing the expected time span from the initial triggering of the laser screen on one side of the security checkpoint to the interactive triggering on the other side, corresponding to a typical, verified violation of package passing behavior in historical data. These baseline intrusion characteristics are obtained by statistically analyzing laser trigger time interval data from a large sample of confirmed violation events and calculating their central tendency, such as the arithmetic mean or median.

[0058] Secondly, the actual intrusion characteristics obtained from current monitoring—namely, the actual time interval between the initial trigger and the interaction trigger—are compared with the aforementioned baseline intrusion characteristics. The similarity between the two is calculated to quantify the consistency between the current event's temporal pattern and typical violation temporal patterns. For example, a Gaussian function-based similarity calculation method can be used, where the baseline intrusion characteristics serve as the center of a Gaussian distribution. The closer the actual time interval is to this central baseline value, the higher the calculated similarity value. This similarity value is directly defined as the violation probability based on temporal characteristics, called the intrusion violation rate. This intrusion violation rate reflects the degree of agreement between the current event's spatiotemporal behavior pattern and typical violation patterns.

[0059] Then, the obtained intrusion violation rate is combined with the violation rate within the interaction violation recognition results obtained from the violation behavior recognition network group using a predetermined fusion algorithm. For example, the fusion algorithm can be a weighted average, that is, the visual violation rate and the temporal violation rate are assigned different weights and then summed. The formula for calculating the fused violation rate can be expressed as: Fusion Violation Rate = α × Visual Violation Rate + (1-α) × Intrusion Violation Rate, where α is the weight coefficient of the visual violation rate, which is pre-set based on the historical accuracy and confidence of the visual recognition model on the independent test set; for example, α can be set to 0.7. Through this calculation, a fused violation rate that simultaneously considers visual evidence and spatiotemporal behavioral evidence can be obtained.

[0060] Finally, considering the potential impact of personnel density on the reliability of behavior recognition, it is necessary to further use the real-time acquired personnel density information and the preset average personnel density to calculate the compensation for the above-mentioned fusion violation rate.

[0061] For example, when the real-time crowd density is much higher than the average density, overcrowding can lead to decreased image clarity and occlusion of actions, reducing the confidence level of image-based recognition. The system can use a compensation function to appropriately lower the fusion violation rate to avoid false alarms caused by degraded image quality. Conversely, when crowds are sparse, the confidence level of the fusion result can be maintained or enhanced. The value obtained after this environmental adaptive compensation calculation is called the compensation violation rate. Specifically, the compensation violation rate = fusion violation rate × (average crowd density / real-time crowd density).

[0062] When the real-time crowd density equals the average crowd density, the ratio is 1, the compensated violation rate equals the fusion violation rate, and the system does not perform environmental compensation adjustment. When the real-time crowd density is higher than the average crowd density, the ratio is less than 1, the compensated violation rate is lower than the fusion violation rate, and the system, based on the judgment of environmental congestion, appropriately lowers the final violation probability to suppress the tendency of false alarms that may be caused by image quality degradation and mutual occlusion. When the real-time crowd density is lower than the average crowd density, the ratio is greater than 1, the compensated violation rate is higher than the fusion violation rate, and the system, based on the judgment of a lenient environment, maintains or enhances the confidence of the fusion result, thereby improving the detection strictness and early warning sensitivity of potential violations.

[0063] The compensation violation rate integrates three types of information: visual analysis, spatiotemporal behavior patterns, and environmental congestion, and is output as the final, more robust violation identification result.

[0064] In summary, the embodiments of this application have at least the following technical effects: Compared to existing technologies, this invention first uses image sensors to perceive the density of people in the security check area in real time, and then dynamically decides and adjusts the monitoring range of the laser curtain wall based on this density information. This allows the physical detection boundary to adapt to changes in pedestrian flow, avoiding false triggers in crowded areas and increasing sensitivity in less crowded areas, thus achieving intelligent and flexible monitoring strategies. Secondly, by using the laser curtain wall as a high-precision, low-latency primary triggering mechanism, it can accurately capture intrusion events involving packages crossing preset physical boundaries and their precise timing characteristics, providing a reliable event starting point and key parameters for subsequent analysis.

[0065] Furthermore, after laser triggering, the system selectively retrieves local image sequences associated with the event and employs a multi-algorithm fusion network based on ensemble learning to perform deep behavioral analysis on the images. By integrating the judgment results of multiple models, the robustness and accuracy of identifying specific violations such as "passing packages" are improved, effectively overcoming the limitations of a single algorithm in complex scenarios. Finally, the system innovatively incorporates personnel density information and intrusion characteristics into the final decision-making process, performing dual compensation correction on the preliminary identification results based on environmental and behavioral patterns. This further reduces misjudgments and missed judgments caused by crowded scenes and high behavioral similarity, ultimately achieving efficient, accurate, and environmentally adaptive intelligent identification of illegal package-passing behavior in subway security check areas.

[0066] Example 2, as Figure 2 As shown, based on the same inventive concept as the intelligent identification method for violations based on multi-algorithm fusion provided in Embodiment 1, this embodiment of the invention also provides an intelligent identification system for violations based on multi-algorithm fusion, comprising: Image acquisition and processing module 11 is used to acquire regional image sequences of the target area acquired by the image sensor, perform personnel density recognition, and obtain personnel density information; The laser curtain wall control module 12 is used to make laser curtain wall area scale decisions based on the personnel density information, obtain the laser curtain wall area scale, control the laser sensor to adjust the laser curtain wall, and form the laser curtain wall area. The intrusion detection module 13 is used to acquire the intrusion time and intrusion characteristics and retrieve the image set of the intrusion area when the laser sensor determines that there is an intrusion in the laser curtain wall area; The violation behavior identification module 14 is used to perform interactive violation behavior identification on the intrusion area image set, obtain interactive violation behavior identification results, and combine the intrusion features to obtain the final violation behavior identification result, wherein compensation processing is performed based on personnel density information and intrusion features.

[0067] The image acquisition and processing module 11 is specifically used for: The system acquires a sequence of regional images of the target area captured by an image sensor, performs personnel density identification, and obtains personnel density information, including: Take a sequence of regional images of the target area acquired by the image sensor, wherein the target area includes a first region and a second region, and the first region and the second region are separated by a dividing line; The image sequence of the region is input into the personnel density recognition plugin, and personnel density information is output.

[0068] Specifically, the personnel density recognition plugin is trained and configured within the control system using the following steps: Based on historical monitoring data of the target area, collect a set of sample area images; The density of people in each sample region image is labeled to obtain a set of sample person density information; A personnel density recognition plugin was built based on a convolutional neural network. The personnel density recognition plugin is trained and tested under supervision using the sample region image set and the sample personnel density information set. After the test converges, it is embedded into the control system.

[0069] Specifically, the laser curtain wall control module 12 is used for: Based on the personnel density information, a laser curtain wall area scale decision is made to obtain the laser curtain wall area scale. The laser sensors are then controlled to adjust the laser curtain wall, forming the laser curtain wall area, including: Obtain the average personnel density and the preset laser curtain wall area scale; Based on the personnel density information and average personnel density, the preset laser curtain wall area scale is adjusted by decision calculation to obtain the laser curtain wall area scale; According to the laser curtain wall area scale, the laser sensor is controlled to adjust the laser curtain wall to form a laser curtain wall area of ​​the laser curtain wall area scale. The laser curtain wall area includes a first sub-laser curtain wall area and a second sub-laser curtain wall area that fall into the first area and the second area, which are divided by the interval line.

[0070] Specifically, based on the personnel density information and average personnel density, the preset laser curtain wall area scale is adjusted through decision calculation to obtain the laser curtain wall area scale, including: The ratio of the personnel density information to the average personnel density is calculated and used as a density adjustment coefficient; The density adjustment coefficient is used to make decision calculations and adjustments to the preset laser curtain wall area scale to obtain the laser curtain wall area scale.

[0071] Specifically, the intrusion detection module 13 is used for: When the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and intrusion characteristics are acquired, and an image set of the intrusion area is retrieved, including: When the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and the intruded sub-laser curtain wall area are obtained, wherein the intruded sub-laser curtain wall area is the first sub-laser curtain wall area and the second sub-laser curtain wall area; Continue monitoring to obtain the interactive intrusion time of other sub-laser curtain wall areas, calculate the time interval with the intrusion time as the intrusion feature, wherein if no other sub-laser curtain wall area intrusion is detected within a preset time range, it is determined that no laser curtain wall area intrusion has occurred, and monitoring continues. Based on the intrusion time and intrusion characteristics, retrieve the image set of the intrusion area.

[0072] Specifically, based on the intrusion time and intrusion characteristics, a set of images of the intrusion area is retrieved, including: Retrieve the region image corresponding to the intrusion time, and the region image of the interaction intrusion time within the intrusion feature after the intrusion time, to obtain an intrusion region image set.

[0073] The violation identification module 14 is specifically used for: Interactive violation behavior identification is performed on the intrusion area image set to obtain interactive violation behavior identification results. Combined with the intrusion features, the final violation behavior identification result is obtained, including: Obtain a network cluster for identifying violations based on ensemble learning, wherein the network cluster includes multiple networks for identifying violations, and the training data of each network is not completely the same. The input features in supervised training are a set of images of the intrusion area of ​​the sample, and the supervision labels are the results of identifying the violation of the sample. The intrusion area image set is input into the violation behavior recognition network group, and multiple violation behavior recognition results are output, wherein the violation behavior recognition results include violation or no violation; Calculate the percentage of violations within the multiple violation identification results to obtain the violation rate, which is used as the interactive violation identification result; Based on the results of the interaction violation identification and the intrusion characteristics, the final violation identification result is obtained.

[0074] Specifically, based on the interaction violation identification results and intrusion characteristics, the final violation identification result is obtained through processing, including: Obtain the baseline intrusion characteristics of the violation; Calculate the similarity between the intrusion feature and the baseline intrusion feature to obtain the intrusion violation rate; The fusion violation rate is calculated based on the intrusion violation rate and the violation rate within the interaction violation identification results; Based on the personnel density information and average personnel density, the fusion violation rate is compensated and calculated to obtain the compensated violation rate, which serves as the final violation identification result.

[0075] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for intelligent identification of violations based on multi-algorithm fusion, characterized in that, The method includes: The system acquires a sequence of regional images of the target area collected by an image sensor, performs personnel density identification, and obtains personnel density information. Based on the personnel density information, a laser curtain wall area scale decision is made to obtain the laser curtain wall area scale, and the laser sensor is controlled to adjust the laser curtain wall to form the laser curtain wall area. When the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and intrusion characteristics are obtained, and the image set of the intrusion area is retrieved; Interactive violation behavior identification is performed on the intrusion area image set to obtain interactive violation behavior identification results. Combined with the intrusion features, the final violation behavior identification result is obtained, wherein compensation processing is performed based on personnel density information and intrusion features.

2. The intelligent identification method for violations based on multi-algorithm fusion according to claim 1, characterized in that, The system acquires a sequence of regional images of the target area captured by an image sensor, performs personnel density identification, and obtains personnel density information, including: Take a sequence of regional images of the target area acquired by the image sensor, wherein the target area includes a first region and a second region, and the first region and the second region are separated by a dividing line; The image sequence of the region is input into the personnel density recognition plugin, and personnel density information is output.

3. The intelligent identification method for violations based on multi-algorithm fusion according to claim 2, characterized in that, The personnel density recognition plugin is trained and configured within the control system using the following steps: Based on historical monitoring data of the target area, collect a set of sample area images; The density of people in each sample region image is labeled to obtain a set of sample person density information; A personnel density recognition plugin was built based on a convolutional neural network. The personnel density recognition plugin is trained and tested under supervision using the sample region image set and the sample personnel density information set. After the test converges, it is embedded into the control system.

4. The intelligent identification method for violations based on multi-algorithm fusion according to claim 1, characterized in that, Based on the personnel density information, a laser curtain wall area scale decision is made to obtain the laser curtain wall area scale. The laser sensors are then controlled to adjust the laser curtain wall, forming the laser curtain wall area, including: Obtain the average personnel density and the preset laser curtain wall area scale; Based on the personnel density information and average personnel density, the preset laser curtain wall area scale is adjusted by decision calculation to obtain the laser curtain wall area scale; According to the laser curtain wall area scale, the laser sensor is controlled to adjust the laser curtain wall to form a laser curtain wall area of ​​the laser curtain wall area scale. The laser curtain wall area includes a first sub-laser curtain wall area and a second sub-laser curtain wall area that fall into the first area and the second area, which are divided by the interval line.

5. The intelligent identification method for violations based on multi-algorithm fusion according to claim 4, characterized in that, Based on the personnel density information and average personnel density, the preset laser curtain wall area scale is adjusted through decision calculation to obtain the laser curtain wall area scale, including: The ratio of the personnel density information to the average personnel density is calculated and used as a density adjustment coefficient; The density adjustment coefficient is used to make decision calculations and adjustments to the preset laser curtain wall area scale to obtain the laser curtain wall area scale.

6. The intelligent identification method for violations based on multi-algorithm fusion according to claim 1, characterized in that, When the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and intrusion characteristics are acquired, and an image set of the intrusion area is retrieved, including: When the laser sensor detects an intrusion into the laser curtain wall area, the intrusion time and the intruded sub-laser curtain wall area are obtained, wherein the intruded sub-laser curtain wall area is the first sub-laser curtain wall area and the second sub-laser curtain wall area; Continue monitoring to obtain the interactive intrusion time of other sub-laser curtain wall areas, calculate the time interval with the intrusion time as the intrusion feature, wherein if no other sub-laser curtain wall area intrusion is detected within a preset time range, it is determined that no laser curtain wall area intrusion has occurred, and monitoring continues. Based on the intrusion time and intrusion characteristics, retrieve the image set of the intrusion area.

7. The intelligent identification method for violations based on multi-algorithm fusion according to claim 6, characterized in that, Based on the intrusion time and intrusion characteristics, retrieve the intrusion area image set, including: Retrieve the region image corresponding to the intrusion time, and the region image of the interaction intrusion time within the intrusion feature after the intrusion time, to obtain an intrusion region image set.

8. The intelligent identification method for violations based on multi-algorithm fusion according to claim 1, characterized in that, Interactive violation behavior identification is performed on the intrusion area image set to obtain interactive violation behavior identification results. Combined with the intrusion features, the final violation behavior identification result is obtained, including: Obtain a network cluster for identifying violations based on ensemble learning, wherein the network cluster includes multiple networks for identifying violations, and the training data of each network is not completely the same. The input features in supervised training are a set of images of the intrusion area of ​​the sample, and the supervision labels are the results of identifying the violation of the sample. The intrusion area image set is input into the violation behavior recognition network group, and multiple violation behavior recognition results are output, wherein the violation behavior recognition results include violation or no violation; Calculate the percentage of violations within the multiple violation identification results to obtain the violation rate, which is used as the interactive violation identification result; Based on the results of the interaction violation identification and the intrusion characteristics, the final violation identification result is obtained.

9. The intelligent identification method for violations based on multi-algorithm fusion according to claim 8, characterized in that, Based on the interaction violation identification results and intrusion characteristics, the final violation identification results are obtained through processing, including: Obtain the baseline intrusion characteristics of the violation; Calculate the similarity between the intrusion feature and the baseline intrusion feature to obtain the intrusion violation rate; The fusion violation rate is calculated based on the intrusion violation rate and the violation rate within the interaction violation identification results; Based on the personnel density information and average personnel density, the fusion violation rate is compensated and calculated to obtain the compensated violation rate, which serves as the final violation identification result.

10. A smart identification system for violations based on multi-algorithm fusion, characterized in that: The method for intelligent identification of violations based on multi-algorithm fusion as described in any one of claims 1-9 includes: The image acquisition and processing module is used to acquire regional image sequences of the target area collected by the image sensor, perform personnel density recognition, and obtain personnel density information. The laser curtain wall control module is used to make laser curtain wall area scale decisions based on the personnel density information, obtain the laser curtain wall area scale, control the laser sensor to adjust the laser curtain wall, and form the laser curtain wall area. The intrusion detection module is used to acquire the intrusion time and intrusion characteristics and retrieve the image set of the intrusion area when the laser sensor detects an intrusion in the laser curtain wall area; The violation behavior identification module is used to identify interactive violation behaviors in the intrusion area image set, obtain interactive violation behavior identification results, and combine them with the intrusion features to obtain the final violation behavior identification result, wherein compensation processing is performed based on personnel density information and intrusion features.

Images