River embankment personnel intrusion intelligent recognition method and device based on target detection

Abstract

The application provides a kind of river section embankment personnel invasion intelligent identification method and device based on target detection, it is related to computer vision field.The method comprises: obtaining river embankment detection picture data;Adopt stem sub-network to extract the multi-scale semantic and texture features of river embankment detection picture data;Multi-scale semantic and texture features are aligned and fused using neck sub-network to obtain deep features;Neck sub-network is composed of top-down and bottom-up feature pyramid fusion module combined with channel attention and spatial attention convolution attention mechanism;Based on deep features, the target class is identified by detection head sub-network, and the target positioning is predicted;Detection head sub-network includes the classification module of KAN neural network.The application is used in the process of intelligent identification of river section embankment personnel invasion based on target detection, and solves the technical problem of low recognition accuracy of existing target detection technology in complex outdoor environment.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method and device for intelligent identification of personnel intrusion into river embankments based on object detection. Background Technology

[0002] The safety of river embankments is a crucial aspect of water conservancy protection and public management. With rapid urbanization, riverbanks have become common spots for residents' leisure activities such as fishing and swimming, leading to frequent incidents of illegal intrusion into embankment areas. This threatens public safety and poses risks to the operation and maintenance of water conservancy facilities. Traditional manual inspections and video surveillance methods are limited by high labor costs, slow response times, and low accuracy, making it difficult to achieve real-time intelligent identification of intrusions in complex environments. Existing object detection methods are mostly based on convolutional neural networks (CNNs), which extract image features through hierarchical convolutional structures and have achieved good results in static object detection. However, traditional CNNs suffer from limited feature representation capabilities and insufficient cross-scale object detection performance, especially in complex outdoor scenes like river embankments with drastic lighting changes, where models are prone to false positives and false negatives. To overcome these shortcomings, the YOLO (You Only Look Once) series of algorithms, as a single-stage detection model, is widely used due to its end-to-end detection structure and high inference efficiency. While inheriting the real-time detection advantages of previous versions, YOLOv11 effectively improves the detection performance of targets at different scales through an improved feature pyramid structure and adaptive anchor box mechanism, making it more suitable for real-time intrusion identification tasks in dynamic monitoring and complex backgrounds. However, under complex background interference, feature confusion can easily occur, affecting prediction accuracy. Therefore, there is an urgent need for an intelligent identification method for personnel intrusion into river embankments to solve the problem of low identification accuracy of existing target detection technologies in complex outdoor environments. Summary of the Invention

[0003] This application provides a method and device for intelligent identification of personnel intrusion into river embankments based on target detection, which solves the technical problem of low identification accuracy of existing target detection technologies in complex outdoor environments.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, a method for intelligent identification of personnel intrusion into river embankments based on target detection is provided, comprising: acquiring embankment detection image data; extracting multi-scale semantic and texture features from the embankment detection image data using a backbone sub-network; the backbone sub-network includes multiple convolutional layers, a cross-stage partial connection feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connection feature extraction module that integrates position-sensitive attention; aligning and fusing multi-scale semantic and texture features using a neck sub-network to obtain deep features; the neck sub-network is composed of a top-down and bottom-up feature pyramid fusion module combined with a convolutional attention mechanism of channel attention and spatial attention; based on the deep features, identifying target categories and predicting target location through a detection head sub-network; the detection head network includes a classification module of a KAN neural network.

[0006] In conjunction with the first aspect mentioned above, one possible implementation involves using a backbone subnetwork to extract multi-scale semantic and texture features from riverbank detection image data. This includes: sequentially performing convolution, normalization, and activation operations on the input riverbank detection image data to obtain basic feature representations; performing dual-path feature extraction on the basic features based on a cross-stage splitting structure, and fusing the outputs of the two paths through convolution to obtain multi-level semantic and texture features; extracting enhanced features of the multi-level semantic and texture features through a fast spatial pyramid pooling structure, and aggregating the enhanced features at multiple scales using max pooling operators of different scales to output a multi-scale semantic and texture feature set.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, the basic feature representation... Satisfy the following formula:

[0008]

[0009] in, This represents a two-dimensional convolution operation. For batch normalization operations, activation function SiLU is used, where I is the input image data of the river embankment detection. The convolution kernel parameter matrix, For bias terms;

[0010] In conjunction with the first aspect mentioned above, one possible implementation involves multi-level semantic and texture features. Satisfy the following formula:

[0011]

[0012] in, Indicates by indivual Subpaths formed by cascading -BN-SiLU Concat() is an aggregation operation. and This represents the basic feature representation of the input in two channels. express Convolutional networks;

[0013] In conjunction with the first aspect mentioned above, one possible implementation involves a feature set within the multi-scale semantic and texture feature set. Satisfy the following formula:

[0014]

[0015]

[0016] Wherein, U represents multi-level semantic and textural features. This represents a max-pooling operator with kernel k, where k is 5, 9, or 13. Indicates aggregation operation, for A convolutional network.

[0017] In conjunction with the first aspect mentioned above, one possible implementation involves using a neck sub-network to align and fuse multi-scale semantic and texture features to obtain deep features. This includes: performing high-level semantic upsampling on multi-scale semantic and texture features, convolving and aligning them with adjacent low-level features, and then adding them to obtain supplementary semantic features; performing low-level semantic downsampling on multi-scale semantic and texture features, and aggregating them level by level to high-level features to obtain enhanced target boundaries and small target features; fusing the supplementary semantic features and enhanced target boundaries and small target features through multiple layers to obtain multi-scale pyramid features; and applying a convolutional attention mechanism of channel attention and spatial attention to the multi-scale pyramid features to obtain deep features.

[0018] In conjunction with the first aspect mentioned above, in one possible implementation, deep features are obtained by using a convolutional attention mechanism combining channel attention and spatial attention to represent multi-scale pyramid features. This includes: obtaining deep features by using a convolutional attention mechanism combining channel attention and spatial attention to represent pyramid features at each scale, specifically: inputting pyramid features at each scale into a channel attention mechanism to extract channel attention features; multiplying the pyramid features at each scale with the channel attention features to obtain preliminary spatial attention features; using the preliminary spatial attention features with a spatial attention mechanism to obtain spatial attention features; and multiplying the preliminary spatial attention features with the spatial attention features to obtain attention-enhanced features.

[0019] In conjunction with the first aspect mentioned above, in one possible implementation, the pyramid features at each scale are input into the channel attention mechanism to extract channel attention features, including: operating the pyramid features at each scale through global average pooling and max pooling respectively to obtain global average pooling features and max pooling features; the global average pooling features and max pooling features are nonlinearly mapped through a shared two-layer perceptron and then added together, and activated by the Sigmoid function to obtain channel attention features.

[0020] In conjunction with the first aspect mentioned above, one possible implementation involves global average pooling features. Satisfy the following formula:

[0021]

[0022] in, This represents global average pooling, where F represents the pyramid feature at a certain scale, and C represents the number of channels. Represents the set of real numbers;

[0023] In conjunction with the first aspect mentioned above, in one possible implementation, the max pooling feature satisfies... The following formula:

[0024]

[0025] in, This represents global max pooling, where F represents the pyramid feature at a certain scale, and C represents the number of channels. Represents the set of real numbers;

[0026] In conjunction with the first aspect mentioned above, in one possible implementation, channel attention features Satisfy the following formula:

[0027]

[0028] in, For the Sigmoid function, It is a two-layer perceptron. This is a global average pooling feature. For max pooling features, C represents the number of channels. It represents the set of real numbers.

[0029] In conjunction with the first aspect mentioned above, in one possible implementation, the initial spatial attention features are processed through a spatial attention mechanism to obtain spatial attention features, including: performing operations on the initial spatial features using global average pooling and max pooling respectively to obtain spatial global average pooling features and spatial max pooling features; concatenating the spatial global average pooling features and spatial max pooling features, and then... Convolutional operations are performed, and the sigmoid function is used for activation to obtain spatial attention features; among them, spatial attention features... Satisfy the following formula:

[0030]

[0031]

[0032] in, For the Sigmoid function, for convolution, Let G be the feature space size, and G be the concatenated feature size resulting from the global average pooling feature and the max pooling feature. This represents a spatially global average pooling feature. For spatial max pooling features, This indicates a splicing operation. It represents the set of real numbers.

[0033] In conjunction with the first aspect mentioned above, one possible implementation involves identifying target categories and predicting target localization based on deep features through a detection head sub-network. This includes: obtaining classification branch features and regression branch features through two convolutional layers based on deep features; introducing spline function expansion and linear fusion with the channel dimension into the classification branch features through the KAN classification module to obtain higher-order nonlinear features; using the higher-order nonlinear features, employing the Sigmoid activation function to output the grid category probability at each grid position; for the regression branch, using an anchorless distributed regression method to predict the discrete probability distributions of the bounding box in the left, right, top, and bottom directions, respectively, and generating continuous margin values ​​through expectation calculation, and then combining the grid center coordinates to reconstruct the target bounding box.

[0034] Secondly, a smart identification device for personnel intrusion into river embankments based on target detection is provided, comprising: a communication unit and a processing unit; the communication unit is used to acquire river embankment detection image data; the processing unit is used to extract multi-scale semantic and texture features from the river embankment detection image data using a backbone sub-network; the backbone sub-network includes multiple convolutional layers, a cross-stage partial connection feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connection feature extraction module that integrates position-sensitive attention; a neck sub-network is used to align and fuse multi-scale semantic and texture features to obtain deep features; the neck sub-network is composed of a top-down and bottom-up feature pyramid fusion module combined with a convolutional attention mechanism of channel attention and spatial attention; based on the deep features, a detection head sub-network is used to identify the target category and predict the target location; the detection head network includes a classification module of a KAN neural network.

[0035] Thirdly, this application provides an electronic device, including: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the methods described in the first aspect and any possible implementation thereof. This electronic device may be an electronic device or a chip within an electronic device.

[0036] This application provides an intelligent identification method and device for personnel intrusion into river embankments based on target detection. By acquiring river embankment detection image data, multi-scale semantic and texture modeling of image features is performed in the backbone sub-network to fully distinguish the differences in personnel, embankment slope, water surface reflection, vegetation, and background clutter within the river embankment scene. The backbone sub-network consists of multiple convolutional layers, pyramid pooling layers, and lightweight cross-layer connection feature extraction modules that incorporate position-sensitive attention. This allows the model to maintain high inference speed while enhancing the semantic correlation and texture representation capabilities between deep and shallow features. Furthermore, a neck sub-network is employed to align and fuse the aforementioned multi-scale semantic and texture features. This neck sub-network combines top-down and bottom-up feature pyramid fusion structures and incorporates a convolutional attention mechanism with channel and spatial attention. This enables the establishment of more accurate contextual relationships between features at different scales, not only improving the saliency representation of small-scale pedestrian targets but also filtering irrelevant background information such as water surface highlights and tree shadow changes through the attention mechanism, thereby obtaining a more stable and adaptable deep feature representation for the specific environment of the embankment. Building upon this foundation, the detection head sub-network identifies target categories and predicts target locations based on extracted deep features. A KAN neural network is employed to construct the classification module, enabling the detection head to maintain a lightweight design while possessing stronger nonlinear expression capabilities and decision boundary modeling abilities. This effectively improves the accuracy of personnel intrusion classification in highly interfering environments, solving the technical problem of low recognition accuracy in complex outdoor environments inherent in existing target detection technologies.

[0037] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0038] Figure 1 A system architecture diagram of an intelligent identification system for personnel intrusion into river embankments based on target detection, provided in an embodiment of this application;

[0039] Figure 2 A flowchart illustrating an intelligent identification method for personnel intrusion into river embankments based on target detection, provided in an embodiment of this application;

[0040] Figure 3 A flowchart illustrating another intelligent identification method for personnel intrusion into river embankments based on target detection, provided in an embodiment of this application;

[0041] Figure 4 A flowchart illustrating another intelligent identification method for personnel intrusion into river embankments based on target detection, provided in an embodiment of this application;

[0042] Figure 5 A flowchart illustrating another intelligent identification method for personnel intrusion into river embankments based on target detection, provided in an embodiment of this application;

[0043] Figure 6 A flowchart illustrating another intelligent identification method for personnel intrusion into river embankments based on target detection, provided in an embodiment of this application;

[0044] Figure 7 A schematic diagram of the structure of an intelligent identification device for personnel intrusion into a river embankment based on target detection, provided in an embodiment of this application;

[0045] Figure 8 This is a schematic diagram illustrating the intelligent identification results of personnel intrusion into a river embankment based on target detection, as provided in an embodiment of this application. Detailed Implementation

[0046] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0047] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0048] The intelligent identification method for personnel intrusion into river embankments based on target detection provided in this application embodiment can be applied to, for example... Figure 1 The system shown is an intelligent identification system for personnel intrusion into river embankments based on target detection. Figure 1 As shown, the system includes: an image acquisition device 101 and an electronic device 102.

[0049] Among them, the image acquisition device 101 is used to acquire river embankment inspection image data;

[0050] Electronic device 102 is used to extract multi-scale semantic and texture features from the riverbank detection image data using a backbone sub-network. The backbone sub-network includes multiple convolutional layers, a cross-stage partial connection feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connection feature extraction module that integrates position-sensitive attention. A neck sub-network is used to align and fuse the multi-scale semantic and texture features to obtain deep features. The neck sub-network is composed of a top-down and bottom-up feature pyramid fusion module combined with a convolutional attention mechanism that integrates channel attention and spatial attention. Based on the deep features, a detection head sub-network is used to identify the target category and predict the target location. The detection head network includes a classification module of a KAN neural network.

[0051] To address the low recognition accuracy of existing target detection technologies in complex outdoor environments, this application provides an intelligent identification method for personnel intrusion into river embankments based on target detection. The method includes: acquiring river embankment detection image data; extracting multi-scale semantic and texture features from the river embankment detection image data using a backbone sub-network; the backbone sub-network comprising multiple convolutional layers, a cross-stage partial connection feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connection feature extraction module incorporating position-sensitive attention; aligning and fusing the multi-scale semantic and texture features using a neck sub-network to obtain deep features; the neck sub-network is composed of a top-down and bottom-up feature pyramid fusion module combined with a convolutional attention mechanism incorporating channel attention and spatial attention; based on the deep features, identifying target categories and predicting target location using a detection head sub-network; the detection head network includes a classification module of a KAN neural network.

[0052] Figure 2This is a flowchart illustrating the intelligent identification method for personnel intrusion into river embankments based on target detection provided in this application embodiment. Figure 2 As shown, the method includes:

[0053] S201. Obtain riverbank inspection image data.

[0054] The riverbank detection image data refers to a sequence of images collected in real time by monitoring cameras, drones, or fixed acquisition terminals deployed along the riverbank. These images mainly include the riverbank environment background, embankment structure, and targets such as people, vehicles, and equipment. This data serves as the input for subsequent target detection models and is the fundamental information source for identifying personnel intrusion.

[0055] In one possible implementation, the latest image frames are periodically retrieved from the monitoring camera, and the images are automatically scaled, denoised, and format-converted according to a preset pixel size to generate standardized image data that meets the model's input requirements. If a drone inspection method is used, the captured images of the river embankment can be transmitted back in real time via a wireless network, and the images can be cached and synchronized on a local server.

[0056] It should be noted that the acquisition frequency, resolution, and lighting conditions of riverbank detection images can affect the effectiveness of feature extraction. Therefore, in actual deployment, the image acquisition strategy can be dynamically adjusted according to the width of the river section, the density of camera deployment, and the expected detection accuracy. For example, to improve nighttime detection performance, infrared cameras or low-light enhanced cameras can be combined to obtain clear images.

[0057] As an example, in an embodiment of this application, during the inspection deployment of a certain river section, the system can automatically acquire a static image from a 4K monitoring camera every 200ms, scale the image to an input size of 640×640, and then input it into the model for processing, thereby realizing continuous monitoring of personnel in the river embankment area.

[0058] S202. Use a backbone subnetwork to extract multi-scale semantic and texture features from riverbank detection image data.

[0059] The backbone subnetwork includes multiple convolutional layers, a cross-stage partial connectivity feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connectivity feature extraction module that incorporates position-sensitive attention.

[0060] In one possible implementation, the backbone subnetwork performs convolution, normalization, and activation operations on the input image to obtain basic feature representations. Then, it utilizes the two bottleneck branches of the CSP structure to extract semantic information and detailed texture features respectively, and achieves more efficient feature representation through cross-stage feature fusion. In further processing, the backbone network aggregates features from different receptive fields using a fast spatial pyramid pooling structure to enhance the model's ability to perceive large, small, and distant targets. Simultaneously, a lightweight cross-layer module incorporating position-sensitive attention strengthens local region features, enabling the model to more accurately perceive human contours and boundaries in complex backgrounds.

[0061] It should be noted that the backbone subnetwork does not alter the semantic structure of the input image during multi-scale feature extraction. Instead, by designing appropriate convolutional kernel size, stride, and pooling strategies, the model can simultaneously learn shallow texture and deep semantic features. Furthermore, by introducing position-sensitive attention, the model's ability to recognize people in complex riverbank backgrounds can be effectively improved; for example, it can distinguish people from obstructing targets such as debris and vegetation on the embankment.

[0062] S203. Align and fuse multi-scale semantic and texture features of the neck sub-network to obtain deep features.

[0063] The neck subnetwork is composed of a convolutional attention mechanism that combines top-down and bottom-up feature pyramid fusion modules with channel attention and spatial attention.

[0064] In one possible implementation, a neck sub-network is used to upsample the multi-scale features output by the backbone, aligning high-level semantic features with low-level detail features before summing them to generate a semantic supplementary branch. Simultaneously, a bottom-up path is used to downsample low-level high-resolution features layer by layer and fuse them with high-level features to enhance the perception of small targets, boundary information, and human contour structures. Building upon this, a channel attention mechanism is further used to highlight important features from different channels, and a spatial attention mechanism is used to reinforce key spatial locations, thereby generating the final deep feature map for detection.

[0065] It should be noted that the bidirectional fusion structure of the neck subnetwork ensures that the model maintains high person detection accuracy across different scenarios and distances. In the riverbank detection scenario, since people may appear at a distance and their pixel distribution in the image is small, the bottom enhancement path of the PAN can effectively solve the problem of detecting small targets. In addition, the features enhanced by channel and spatial attention can significantly improve the model's ability to perceive person boundaries, actions, and contours.

[0066] S204. Based on deep features, the target category is identified and the target location is predicted by detecting the head sub-network.

[0067] The detection head sub-network includes a classification module of a KAN (Kolmogorov-Arnold Network) neural network. In this application, the detection head also includes two convolutional layers to generate features for classification and regression branches. The classification branch uses a KAN neural network module, incorporating spline function expansion and channel-dimensional linear fusion to generate higher-order nonlinear feature representations, thereby improving classification accuracy.

[0068] In one possible implementation, the detection head performs two convolutional operations on the deep features, resulting in classification and regression branch features. In the classification branch, the KAN module maps the features to a higher-order space, and the Sigmoid function is used to activate and obtain the target class probability at each grid location. In the regression branch, an anchor-free distributed regression method is used to predict the discrete probability distribution of the bounding box in the four directions. The continuous margin values ​​are then calculated using expectation, and combined with the grid center coordinates to generate the final target bounding box, achieving accurate location of the person.

[0069] It should be noted that the KAN module effectively addresses the limited expressive power of traditional convolutional classification modules in complex contexts, making it easier for the model to distinguish people from non-target areas such as embankment stones, weeds, or equipment in river embankment scenes. Furthermore, the anchorless distributed regression approach avoids biases caused by improper anchor box settings, allowing the model to more flexibly adapt to changes in person size from different camera perspectives. The KAN classification module replaces the traditional multilayer perceptron (MLP). KAN utilizes the learnable properties of piecewise spline basis functions to construct nonlinear mappings, thereby improving the model's classification extrapolation ability and data utilization efficiency. It exhibits stronger stability and generalization in handling long-tailed distributions, class similarity, and sample scarcity, especially in fine-grained behavior classification such as loitering, fishing, and swimming, achieving more accurate class distinctions and reducing false positives and false negatives, demonstrating greater flexibility and adaptability in complex behavior recognition tasks.

[0070] This application embodiment continuously acquires riverbank detection image data, enabling the detection device to monitor the dynamics, spatial distribution, and behavioral postures of people near the riverbank in real time. This provides reliable visual input for identifying abnormal approach, unauthorized entry, or lingering behavior. The backbone subnetwork performs multi-scale semantic and texture feature extraction on the detection images, allowing the device to simultaneously capture human contours, movement patterns, and detailed features against complex backgrounds. This enhances the ability to recognize human activities at different distances, under different lighting conditions, and against different backgrounds. A neck subnetwork aligns and fuses multi-scale features, constructing deep features with high semantic consistency and strong regional representation capabilities. This enables the device to more stably distinguish people from similar backgrounds such as riverbank structures, water bodies, and vegetation, reducing false positives and false negatives. Based on deep features, the device uses a detection head sub-network to identify personnel categories and predict their locations. This enables the device to accurately determine whether personnel have entered restricted riverbank areas or are near dangerous slopes. It can also output specific coordinate boxes with high precision, enabling real-time alarms and automatic risk assessment. This improves the accuracy, speed, and adaptability to complex environments for personnel intrusion detection on riverbanks. The device can maintain stable performance under conditions of day / night changes, noise interference, and background overlap, solving the technical problem of low recognition accuracy in complex outdoor environments in existing target detection technologies.

[0071] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 3 As shown, the above-mentioned S202, which uses a backbone sub-network to extract multi-scale semantic and texture features from riverbank detection image data, can be specifically implemented through the following S301, S302, and S303, which are explained in detail below:

[0072] S301. Perform convolution, normalization, and activation operations sequentially on the input riverbank detection image data to obtain basic feature representations.

[0073] The input river embankment detection image data refers to single or multiple frames of images taken by monitoring cameras, drones, or fixed acquisition modules along the river embankment. The image size is preprocessed and adjusted to the standard input resolution required by the neural network. The image content includes information such as personnel, embankment structure, water surface, and background vegetation.

[0074] In one possible implementation, the device performs several two-dimensional convolution operations with learnable weights on the input image to extract local texture and edge features. The processing unit then normalizes the convolution output to stabilize the feature distribution and accelerate convergence. Finally, the processing unit applies the SiLU or ReLU activation function to the normalized features to introduce nonlinearity, thereby generating a basic feature representation for subsequent layer processing.

[0075] It should be noted that the basic feature representation is affected by the setting of convolution kernel size, stride, padding method and normalization parameters. By reasonably configuring the convolution layer parameters and normalization hyperparameters, the numerical stability of different batches of input in the feature space can be ensured, and the activation function selection takes into account both feature sparsity and gradient propagation.

[0076] As an example, in an embodiment of this application, the basic feature representation Satisfy the following formula:

[0077]

[0078] in, This represents a two-dimensional convolution operation. For batch normalization operations, activation function SiLU is used, where I is the input image data of the river embankment detection. The convolution kernel parameter matrix, This is a bias term.

[0079] As an example, in this embodiment of the application, a 3×3 convolutional layer with a stride of 1 is first applied to an RGB image with a resolution of 640×640 to extract low-level textures. Then, three convolutional-BN-SiLU units are applied sequentially to generate initial feature maps of 80×80, 40×40 and 20×20. The processing unit uses these initial feature maps as basic feature representations and passes them to the cross-stage splitting module.

[0080] It should be noted that this step improves the feature signal-to-noise ratio of the input image by sequentially performing convolution, normalization, and activation operations. As a result, the device obtains a basic feature representation that retains local details and has a stable distribution, further improving the efficiency and accuracy of subsequent dual-path feature extraction and multi-scale fusion, thus laying a reliable foundation for the recognition of human activities in complex backgrounds.

[0081] S302. Based on the cross-stage splitting structure, dual-path feature extraction is performed on the basic features, and the outputs of the two paths are fused by convolution to obtain multi-level semantic and texture features.

[0082] Among them, the cross-stage split structure refers to dividing the basic features into two or more sub-channel branches along the channel direction, and extracting semantic and texture features respectively using bottleneck blocks with different depths or different convolution configurations in each branch. The split structure realizes information reuse and gradient flow through cross-stage connections.

[0083] In one possible implementation, the device divides the basic features into semantic and texture branches in the channel dimension. In the semantic branch, more convolutional layers are stacked to extract high-level semantic representations, while in the texture branch, a shallower convolutional sequence is used to preserve edge and detail information. Then, convolution is applied to the outputs of the two branches for channel alignment and concatenation. The convolutional layers are used to fuse the concatenated results to generate a fused feature containing multi-level semantic and texture information.

[0084] It should be noted that the channel allocation ratio, bottleneck layer number, and residual connection method of dual-path extraction affect the balance of feature expression. The most suitable branch configuration for small target detection and background suppression in the river embankment scene can be determined by hyperparameter tuning or automatic search to avoid excessive emphasis on a certain type of feature by a single path.

[0085] As an example, in an embodiment of this application, multi-layer texture features Satisfy the following formula:

[0086]

[0087] in, Indicates by indivual Subpaths formed by cascading -BN-SiLU Concat() is an aggregation operation. and This represents the basic feature representation of the input in two channels. express A convolutional network.

[0088] As an example, in this embodiment of the application, the basic features are divided into two paths in a 1:1 ratio. The semantic branch uses two sets of stacked bottleneck blocks of 3×3-Conv-BN-SiLU, and the texture branch uses a set of lightweight blocks of 3×3-Conv-BN-SiLU. The two outputs are aligned through 1×1 convolution and then spliced ​​together, and then fused through 3×3 convolution to generate multi-level features for the next stage.

[0089] It should be noted that this step achieves parallel extraction of semantic and texture features through a cross-stage splitting structure and fuses the two outputs through convolution, thereby generating multi-level features that have both rich semantic expression and retain fine-grained texture information, further improving the ability to detect distant and small-sized human targets and the ability to distinguish false targets in complex backgrounds.

[0090] S303. Enhanced features of multi-level semantic and texture features are extracted through a fast spatial pyramid pooling structure, and the enhanced features are aggregated at multiple scales through max pooling operators of different scales to output a multi-scale semantic and texture feature set.

[0091] Among them, the fast spatial pyramid pooling structure (SPPF) refers to a module built on multiple pooling operators of different sizes in parallel or in series. This module extracts multi-scale contextual information through pooling operations of different receptive fields and splices or fuses the results to expand the spatial receptive range of features and enhance the ability to recognize targets of different sizes.

[0092] In one possible implementation, the device inputs multi-level features from dual-path fusion into the SPPF module, in which several max pooling operators with different kernel sizes are applied sequentially or in parallel. The pooling results are concatenated with the original features and then subjected to convolutional dimensionality reduction. The multi-scale max pooling operator is applied again to the dimensionality-reduced features, and the results are aggregated layer by layer to output a feature set containing multi-scale semantic and texture information.

[0093] It should be noted that the choice of pooling kernel size, whether to use serial or parallel pooling, and the dimensionality reduction strategy after splicing in SPPF will affect the receptive field expansion and detail preservation of the features. According to the size distribution of people and the complexity of the background in the river embankment scene, the appropriate pooling kernel group and fusion order should be selected to maximize the expression of multi-scale information while taking into account the computational efficiency.

[0094] As an example, in an embodiment of this application, a feature in the multi-scale semantic and texture feature set... Satisfy the following formula:

[0095]

[0096]

[0097] Where U represents a multi-layer texture feature. This represents a max-pooling operator with kernel k, where k takes values ​​of 5, 9, or 13. Indicates aggregation operation, for A convolutional network.

[0098] As an example, in this embodiment, the device applies three max-pooling operators with kernel sizes of 5, 9, and 13 to the input features sequentially, concatenates the outputs of the three pooling operations with the original features, then performs channel compression using 1×1 convolution to output enhanced features, and then applies max-pooling operators of different scales (3×3 and 5×5) to the enhanced features for multi-scale aggregation, finally outputting multi-scale semantic and texture feature maps. In this embodiment, the backbone network ultimately outputs four levels of features. Their spatial strides are as follows: The number of channels is .

[0099] It should be noted that this step extracts and aggregates contextual information at different scales through a fast spatial pyramid pooling structure, expands the effective receptive field of the features, and enhances the response capability to distant small targets and local details through multi-scale max pooling, thereby improving the accuracy of subsequent neck fusion and detection head classification and localization, as well as the robustness of identifying human activities in complex river embankment backgrounds.

[0100] The method in this application not only enhances the fine-grained feature representation and multi-scale characterization of human activities in river embankment images, but also provides high-quality input for the deep feature fusion of the subsequent neck sub-network and the classification and localization of the detection head, thereby improving the detection accuracy, recognition stability and real-time performance of human activities along the river embankment.

[0101] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 4 As shown, S203 above uses a neck sub-network to align and fuse multi-scale semantic and texture features to obtain deep features. This can be specifically achieved through S401 to S404, which are explained in detail below:

[0102] S401. Perform high-level semantic upsampling on multi-scale semantic and texture features, and then add them after convolutional alignment with adjacent low-level features to obtain supplementary semantic features.

[0103] High-level semantic upsampling refers to enlarging the spatial resolution of high-level feature maps in a network through bilinear interpolation, deconvolution, or other upsampling operations, so that their size is comparable to that of adjacent low-level feature maps. Figure 1 This is done to ensure spatial alignment and fusion, so that high-level semantic information can be effectively combined with low-level texture details.

[0104] In one possible implementation, the high-level feature map is upsampled to adjust its spatial size, and convolution is used to align the channels of adjacent low-level feature maps. Then, the upsampled high-level features are added element by element to the convolution-aligned low-level features to generate supplementary semantic features, which are used to enhance the semantic expression of the low-level features.

[0105] As an example, in this embodiment of the application, the high-level feature map is subjected to 4x bilinear interpolation, and after upsampling, it is aligned with the 80×80 low-level feature map through 1×1 convolution and added element by element to generate supplementary semantic features for subsequent multi-layer fusion processing.

[0106] This step effectively injects deep semantic information into the low-level feature map by upsampling high-level semantics and fusing it with low-level features, thereby enhancing the system's ability to recognize the overall shape and action patterns of people and improving the detection accuracy of people at a distance, in a blurred or partially occluded manner.

[0107] S402. Perform low-level semantic downsampling on multi-scale semantic and texture features, and aggregate them to high levels step by step to obtain enhanced target boundaries and small target features.

[0108] Among them, low-level semantic downsampling refers to reducing the size of low-level feature maps with high spatial resolution to the size of high-level feature maps through convolution stride, pooling or downsampling operations, so as to pass detailed information to the higher levels step by step, thereby strengthening the target boundary and small target features.

[0109] In one possible implementation, the device downsamples the low-level feature map, and then performs channel alignment and element-level addition fusion with the upper-level feature map step by step. This process is repeated until the information is passed to the highest level, thereby generating enhanced target boundaries and small target features.

[0110] It should be noted that the downsampling stride, convolution kernel size, and fusion method affect the degree of preservation of small target features. The system can adjust parameters for common behaviors in river embankment monitoring, such as long-distance personnel, lingering or fishing, to ensure that the boundaries of small targets are clear and the details are complete.

[0111] As an example, in this embodiment of the application, the 80×80 low-level feature map is downsampled to 40×40 by applying a 3×3 convolution with a stride of 2, and then aggregated into a 20×20 high-level feature map in sequence, while retaining key boundary information and outputting enhanced target boundary and small target features.

[0112] This step passes details from lower to higher layers, enabling the higher-level feature maps to acquire boundary and small target information, thereby improving the detection capability for small or partially occluded personnel targets in the riverbank area and enhancing the stable response to target contours in complex backgrounds.

[0113] S403. Supplementing semantic features and strengthening target boundaries and small target features through multi-layer fusion to obtain multi-scale pyramid features.

[0114] Among them, multi-layer fusion refers to integrating supplementary semantic features with enhanced target boundaries and small target features in the channel or spatial dimension, generating a unified representation through convolution or addition operations, and forming a multi-scale pyramid structure so that subsequent detection heads can perform multi-scale target recognition.

[0115] In one possible implementation, the device performs channel concatenation or element-wise addition of supplementary semantic features and enhanced target boundary and small target features at various scales. Then, it performs convolutional dimensionality reduction and activation processing on the fused features, and finally outputs a pyramid feature containing multi-scale semantic and texture information for target classification and localization by the detection head.

[0116] As an example, in this embodiment of the application, the supplementary semantic features and the enhanced small target features are added element-wise at each scale, and 3×3 convolution and SiLU activation are used to generate multi-scale pyramid feature sets of 20×20, 40×40 and 80×80, which provide input for subsequent attention mechanism enhancement and detection.

[0117] This step integrates features from different sources to achieve a unified representation of multi-scale information, enabling the system to possess both high-level semantic understanding capabilities and retain low-level texture and boundary details, thereby improving the detection accuracy and robustness of personnel targets at different scales on the river embankment.

[0118] S404. Deep features are obtained by using the convolutional attention mechanism of channel attention and spatial attention to transform the multi-scale pyramid features.

[0119] In one possible implementation, the device inputs the pyramid features at each scale into the channel attention mechanism to extract channel attention features; multiplies the pyramid features at each scale with the channel attention features to obtain preliminary spatial attention features; passes the preliminary spatial attention features through the spatial attention mechanism to obtain spatial attention features; and multiplies the preliminary spatial attention features with the spatial attention features to obtain attention enhancement features.

[0120] It should be noted that the pyramid features at each scale are input into the channel attention mechanism separately, and the extraction of channel attention features adopts the following process: the pyramid features at each scale are operated on by global average pooling and max pooling respectively to obtain global average pooling features and max pooling features; the global average pooling features and the max pooling features are nonlinearly mapped through a shared two-layer perceptron and then added together, and activated by the Sigmoid function to obtain the channel attention features.

[0121] As an example, in an embodiment of this application, the global average pooling feature... Satisfy the following formula:

[0122]

[0123] in, This represents global average pooling, where F represents the pyramid feature at a certain scale, and C represents the number of channels. Represents the set of real numbers;

[0124] As an example, in an embodiment of this application, the max pooling feature satisfies The following formula:

[0125]

[0126] in, This represents global max pooling, where F represents the pyramid feature at a certain scale, and C represents the number of channels. Represents the set of real numbers;

[0127] As an example, in an embodiment of this application, channel attention features Satisfy the following formula:

[0128]

[0129] in, For the Sigmoid function, It is a two-layer perceptron. This is a global average pooling feature. For max pooling features, It represents the set of real numbers.

[0130] It should also be noted that the process of obtaining spatial attention features from the preliminary spatial attention features through the spatial attention mechanism is as follows: the preliminary spatial features are operated on by global average pooling and max pooling respectively to obtain spatial global average pooling features and spatial max pooling features.

[0131] After concatenating the spatial global average pooling feature and the spatial max pooling feature, through... Convolutional operations are performed, and the sigmoid function is used for activation to obtain spatial attention features.

[0132] As an example, in an embodiment of this application, spatial attention features Satisfy the following formula:

[0133]

[0134]

[0135] in, For the Sigmoid function, for convolution, Let G be the feature space size, and G be the concatenated feature size resulting from the global average pooling feature and the max pooling feature. This represents a spatially global average pooling feature. For spatial max pooling features, This indicates a splicing operation. It represents the set of real numbers.

[0136] As an example, in an embodiment of this application, Figure 5 The overall implementation flowchart of the channel and spatial attention mechanism provided in this application is as follows: Figure 5As shown, for the input features, the channel attention features are first extracted through the channel attention enhancement part. Specifically, the input features are processed by max pooling and average pooling respectively. The max pooling and average pooling features are extracted separately using a shared-parameter MLP module. The max pooling and average pooling features are then added together and activated by the Sigmoid function to obtain the channel attention features. The channel attention features are multiplied by the input features and used as the input to the spatial attention enhancement part. The spatial attention enhancement part extracts spatial attention features by processing the input of this part through max pooling and average pooling, concatenating the processed inputs, and then extracting spatial attention features through convolution. Finally, the input of the spatial attention enhancement part is multiplied by the spatial attention features to obtain the attention enhancement features.

[0137] This step enhances multi-scale pyramid features through channel and spatial attention mechanisms, highlighting personnel targets and suppressing background interference such as water surfaces, vegetation, and shadows, thereby improving the target detection accuracy, recognition stability, and real-time performance of the device in complex lighting and occlusion environments.

[0138] This application's embodiments implement a complete process from multi-scale feature extraction, vertical feature interaction, feature fusion to attention enhancement in a river embankment detection scenario. This effectively improves the accuracy, real-time performance, and robustness of identifying different human activities such as loitering, fishing, and swimming, providing technical support for river embankment safety monitoring and intelligent early warning. By fully utilizing the synergistic effect of channel attention and spatial attention, the model can automatically focus on key areas and salient features of intrusion targets. It effectively suppresses interference from non-critical environmental factors (such as water surface reflection, vegetation swaying, and shadow changes), thereby improving the model's robustness and generalization under complex lighting and multi-background conditions. This effectively enhances feature representation capabilities, making the model more stable in feature recalibration and saliency recognition.

[0139] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 6 As shown, the above S204, based on deep features, identifies the target category and predicts target localization through the detection head sub-network. Specifically, it can be implemented through the following S601 to S604, which are explained in detail below:

[0140] S601. Based on deep features, classification branch features and regression branch features are obtained through two layers of convolution.

[0141] In one possible implementation, the device inputs the deep feature map into the first convolutional layer for channel dimensionality reduction and feature enhancement, performs nonlinear mapping through an activation function, and feeds the output into the second convolutional layer to further extract local spatial information. Finally, it outputs feature maps for classification and regression, respectively, providing basic features for subsequent target classification and localization.

[0142] As an example, in this embodiment of the application, features are first extracted from the deep feature map using 3×3 convolution, and then further processed by 3×3 convolution to generate 40×40 classification branch features and regression branch features, which are used for processing by the KAN classification module and the anchorless regression module.

[0143] This step generates classification and regression branch features through two convolutional layers, providing a structured feature representation for the network. This makes subsequent classification and bounding box prediction more accurate and improves the basic ability to detect targets of human activity on the river embankment.

[0144] S602. By introducing spline function expansion and channel dimension linear fusion into the classification branch features through the KAN classification module, high-order nonlinear features are obtained.

[0145] In one possible implementation, the device standardizes the classification features of each channel and clips them to the spline domain. Then, it constructs cubic B-spline basis functions on each channel using equally spaced nodes to expand the channel features. Finally, it linearly fuses the expanded vectors of each channel in the channel dimension to generate high-order nonlinear features, thereby enhancing the ability to discriminate fine-grained behavior categories.

[0146] As an example, in an embodiment of this application, for the first Classification feature tensors at various scales ,in Indicates the number of channels. These represent the height and width of the feature map at this scale, respectively. Represents the set of real numbers. At any position in the feature map. Take the channel vector: First, the features of each channel are standardized to eliminate the influence of scale differences, resulting in: in, and These represent the mean and standard deviation of the channel, respectively. Small constants to prevent division by zero. Standardized features are clipped to the interval [...]. This ensures that the input falls within the spline domain. Within the interval... Evenly distributed nodes ,in Let the number of segments be equal. Based on these nodes, construct a set of cubic B-spline basis functions: For each channel C, calculate the corresponding spline basis expansion: The basis expansion vectors of all channels are concatenated along the channel dimension to form the overall feature representation: , This indicates a channel dimension splicing operation.

[0147] This step enhances the expressive power of features through KAN nonlinear mapping, improves the ability to distinguish similar behavioral categories, and enables the system to stably identify the behavior of different people along the riverbank under complex backgrounds and long-tailed category distributions.

[0148] S603. Based on high-order nonlinear features, the Sigmoid activation function is used to output the grid category probability at each grid location.

[0149] In one possible implementation, high-order nonlinear features are input into a sigmoid activation function, with each channel corresponding to a behavior category. The probability value for each grid location is calculated, and a probability map with a size consistent with the input feature map space is output to represent the possible categories of human activities in each grid.

[0150] As an example, in an embodiment of this application, the first Logarithmic odds of class Defined as:

[0151]

[0152] in, For KAN classification weight parameters, This represents the class bias term. The corresponding predicted probability. for:

[0153]

[0154] This step transforms high-order nonlinear features into probability values, enabling the device to perform fine classification of personnel behavior on the riverbank. This provides a reliable probabilistic basis for real-time behavior recognition and alarm decision-making, improving the accuracy and interpretability of behavior recognition.

[0155] S604. For the regression branch, an anchorless distributed regression method is adopted to predict the discrete probability distribution of the bounding box in the four directions of left, right, top and bottom, respectively. Continuous margin values ​​are generated by expectation calculation, and then the target bounding box is restored by combining the grid center coordinates.

[0156] In one possible implementation, the regression branch features are used to predict the discrete probability distributions in the four directions of left, right, up, and down at each grid location. Then, the continuous margin values ​​are calculated by weighted averaging or expectation. Finally, the bounding box of the target is calculated by combining the grid center coordinates to generate the final positioning information.

[0157] As an example, in an embodiment of this application, the distance from the left boundary is used. For example, let the number of discrete buckets be... (In this embodiment, 16 is used), the predicted distribution is as follows: Then the left boundary distance is expected The calculation formula is:

[0158]

[0159] The same applies to the distances to the right, top, and bottom boundaries. Here, q represents the q-th discrete distance level, combined with the grid center coordinates. It can restore the bounding box: ,in, The expected distance to the left boundary. The expected distance to the upper boundary. The expected distance to the right boundary. This represents the expected distance to the lower boundary.

[0160] This step achieves flexible and accurate target localization through anchorless distributed regression, accommodating both large and small targets. It enhances the detection capability for people at long distances, with obstructions, or significant size differences in riverbank scenarios, while ensuring the stability and reliability of the regression process. This provides high-precision location support for real-time monitoring and early warning. Furthermore, this decoupled prediction structure effectively improves the accuracy and stability of target classification and localization, exhibiting higher robustness, especially in complex backgrounds and multi-scale target scenarios.

[0161] The embodiments of this application not only improve the detection accuracy and behavior recognition capability of personnel targets of different scales and postures in the river embankment detection scenario, but also take into account the processing of complex backgrounds, occlusions and small target categories, ensuring high robustness, high accuracy and interpretability in real-time monitoring, and providing solid technical support for river embankment safety management and intelligent early warning.

[0162] It should be noted that, in the embodiments of this application, during the model training phase, the BCEWithLogits loss function is used to optimize the classification branch of the detection head to balance numerical stability and gradient convergence efficiency. Specifically, the training samples are first scaled to a uniform size of 640×640 and then input into the network after combining data augmentation strategies such as random flipping, affine transformation, illumination perturbation, and Mosaic. The network forward propagation obtains the classification log odds (logits) and bounding box regression results on feature maps at each scale, where the classification branch directly outputs the unnormalized logits value. For each predicted location and class label, BCEWithLogitsLoss is used to combine the Sigmoid activation and binary cross-entropy calculation, and the calculation result... Satisfy the following formula:

[0163]

[0164] in, Let y ∈ {0,1} be the log-odds output for the classification branch, and y ∈ {0,1} be the true label. This is the Sigmoid activation function.

[0165] This loss effectively avoids the numerical instability problem caused by using the Sigmoid function alone during backpropagation. The classification loss and bounding box regression loss (IoU / DFL) are jointly optimized in a weighted manner. The entire network is trained end-to-end using the AdamW optimizer, with an initial learning rate set to... The weight decay coefficient is The model employs a cosine annealing learning rate scheduling mechanism combined with a warm-up strategy in the first few rounds to improve stability during the initial training phase. During training, the batch size is set to 32, with 200 epochs, and mixed precision training and exponential moving average (EMA) are enabled to further improve model convergence speed and generalization performance, thereby achieving stable identification of personnel intrusion and fine-grained behaviors in complex river embankment scenarios.

[0166] The above primarily describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as the intelligent identification device for personnel intrusion into river embankments based on target detection, includes at least one of the hardware structures and software modules corresponding to each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] This application embodiment can divide the intelligent identification device for personnel intrusion into river embankments based on target detection into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0168] When using integrated units, Figure 7 A possible structural schematic diagram of the intelligent identification device for personnel intrusion into river embankments based on target detection (referred to as intelligent identification device 70 for personnel intrusion into river embankments based on target detection) involved in the above embodiments is shown. The intelligent identification device 70 for personnel intrusion into river embankments based on target detection includes a processing unit 701 and a communication unit 702, and may also include a storage unit 703. Figure 7The schematic diagram shown can be used to illustrate the structure of the intelligent identification device for personnel intrusion into river embankments based on target detection involved in the above embodiments.

[0169] when Figure 7 The schematic diagram shown illustrates the structure of the intelligent identification device for intrusion into river embankments based on target detection involved in the above embodiments. The processing unit 701 is used to control and manage the operation of the intelligent identification device for intrusion into river embankments based on target detection. The communication unit 702 is used for the intelligent identification device for intrusion into river embankments based on target detection to communicate with other devices. The storage unit 703 is used to store the program code and data of the intelligent identification device for intrusion into river embankments based on target detection.

[0170] For example, communication unit 702 is used to acquire riverbank detection image data;

[0171] Processing unit 701 is used to extract multi-scale semantic and texture features from riverbank detection image data using a backbone sub-network. The backbone sub-network includes multiple convolutional layers, a cross-stage partial connection feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connection feature extraction module that integrates position-sensitive attention. A neck sub-network is used to align and fuse multi-scale semantic and texture features to obtain deep features. The neck sub-network is composed of a top-down and bottom-up feature pyramid fusion module combined with a convolutional attention mechanism that integrates channel attention and spatial attention. Based on the deep features, a detection head sub-network is used to identify target categories and predict target localization. The detection head sub-network includes a classification module of a KAN neural network.

[0172] In one possible implementation, the processing unit 701 is further configured to extract multi-scale semantic and texture features from the riverbank detection image data using a backbone sub-network, including: sequentially performing convolution, normalization, and activation operations on the input riverbank detection image data to obtain basic feature representations; performing dual-path feature extraction on the basic features based on a cross-stage splitting structure, and fusing the outputs of the two paths through convolution to obtain multi-level semantic and texture features; extracting enhanced features of the multi-level semantic and texture features through a fast spatial pyramid pooling structure, and aggregating the enhanced features at multiple scales through max pooling operators of different scales to output a multi-scale semantic and texture feature set.

[0173] In one possible implementation, the basic feature representation Satisfy the following formula:

[0174]

[0175] in, This represents a two-dimensional convolution operation. For batch normalization operations, activation function SiLU is used, where I is the input image data of the river embankment detection. The convolution kernel parameter matrix, For bias terms;

[0176] In one possible implementation, multi-level semantics and texture features Satisfy the following formula:

[0177]

[0178] in, Indicates by indivual Subpaths formed by cascading -BN-SiLU Concat() is an aggregation operation. and This represents the basic feature representation of the two-channel input. express Convolutional networks;

[0179] In one possible implementation, a feature is set within the multi-scale semantic and texture feature set. Satisfy the following formula:

[0180]

[0181]

[0182] Wherein, U represents multi-level semantic and texture features. This represents a max-pooling operator with kernel k, where k takes values ​​of 3, 5, or 9. Indicates aggregation operation, for A convolutional network.

[0183] In one possible implementation, a neck sub-network is used to align and fuse multi-scale semantics and texture features to obtain deep features. This includes: performing high-level semantic upsampling on multi-scale semantics and texture features, convolving and aligning them with adjacent low-level features, and then adding them to obtain supplementary semantic features; performing low-level semantic downsampling on multi-scale semantics and texture features, and aggregating them level by level to high-level features to obtain enhanced target boundaries and small target features; fusing the supplementary semantic features and enhanced target boundaries and small target features through multiple layers to obtain multi-scale pyramid features; and applying a convolutional attention mechanism of channel attention and spatial attention to the multi-scale pyramid features to obtain deep features.

[0184] In one possible implementation, the processing unit 701 is further configured to obtain deep features by passing the multi-scale pyramid features through a convolutional attention mechanism of channel attention and spatial attention, including: passing the pyramid features at each scale through a convolutional attention mechanism of channel attention and spatial attention to obtain deep features, including: inputting the pyramid features at each scale into a channel attention mechanism to extract channel attention features; multiplying the pyramid features at each scale and the channel attention features to obtain preliminary spatial attention features; passing the preliminary spatial attention features through a spatial attention mechanism to obtain spatial attention features; and multiplying the preliminary spatial attention features and the spatial attention features to obtain attention-enhanced features.

[0185] In one possible implementation, the processing unit 701 is further configured to input the pyramid features at each scale into the channel attention mechanism to extract channel attention features, including: performing operations on the pyramid features at each scale through global average pooling and max pooling respectively to obtain global average pooling features and max pooling features; performing nonlinear mapping on the global average pooling features and max pooling features through a shared two-layer perceptron and then adding them together, and then activating them through the Sigmoid function to obtain channel attention features.

[0186] In one possible implementation, the processing unit 701 is further configured to obtain spatial attention features from the preliminary spatial attention features through a spatial attention mechanism, including: performing operations on the preliminary spatial features using global average pooling and max pooling respectively to obtain spatial global average pooling features and spatial max pooling features; concatenating the spatial global average pooling features and spatial max pooling features, and then... Convolutional operations are performed, and the sigmoid function is used for activation to obtain spatial attention features; among them, spatial attention features... Satisfy the following formula:

[0187]

[0188]

[0189] in, For the Sigmoid function, for convolution, Let G be the feature space size, and G be the concatenated feature size resulting from the global average pooling feature and the max pooling feature. This represents a spatially global average pooling feature. For spatial max pooling features, This indicates a splicing operation. It represents the set of real numbers.

[0190] In one possible implementation, the processing unit 701 is further configured to identify target categories and predict target localization based on deep features through a detection head sub-network, including: obtaining classification branch features and regression branch features through two layers of convolution based on deep features; introducing spline function expansion and channel dimension linear fusion into the classification branch features through the KAN classification module to obtain higher-order nonlinear features; using the sigmoid activation function based on the higher-order nonlinear features to output the grid category probability at each grid position; for the regression branch, using an anchorless distributed regression method to predict the discrete probability distribution of the bounding box in the left, right, up, and down directions respectively, and generating continuous margin values ​​through expectation calculation, and then combining the grid center coordinates to reconstruct the target bounding box.

[0191] As an example, in an embodiment of this application, Figure 8 Examples of recognition results provided in embodiments of this application, such as Figure 8 As shown, the behavior of people in three river embankment images with different environments was identified. The identification result of the left image was swimming (probability of 0.90), the identification result of the upper right image was fishing (probability of 0.77), and the identification result of the lower right image was staying (probability of 0.81).

[0192] Figure 7 If the integrated units in the process are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0193] Figure 7 The units in the process can also be called modules; for example, a processing unit can be called a processing module.

[0194] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0195] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method for intelligent identification of personnel intrusion into river embankments based on target detection, characterized in that, include: Obtain riverbank inspection image data; A backbone subnetwork is used to extract multi-scale semantic and texture features from the riverbank detection image data. The backbone subnetwork includes multiple convolutional layers, a cross-stage partial connection feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connection feature extraction module that integrates position-sensitive attention. The cross-stage partial connection feature extraction module with two bottleneck structures refers to the parallel extraction of semantic information and detailed texture features of basic features using two bottleneck branches constructed with a CSP structure, and the fusion of semantic information and detailed texture features through convolution to obtain multi-level semantic and texture features. The multi-scale semantic and texture features are aligned and fused using a neck sub-network to obtain deep features; The neck subnetwork is composed of a top-down and bottom-up feature pyramid fusion module combined with a convolutional attention mechanism that integrates channel attention and spatial attention. The neck subnetwork is used to align and fuse the multi-scale semantic and texture features to obtain deep features, including: Multi-scale semantic and texture features are upsampled at a high level and then convolved with adjacent low-level features and added to obtain supplementary semantic features. Multi-scale semantic and texture features are downsampled at a low level and aggregated to a high level to obtain enhanced target boundaries and small target features. The supplementary semantic features and the enhanced target boundary and small target features are fused through multiple layers to obtain multi-scale pyramid features; The multi-scale pyramid features are then processed through a convolutional attention mechanism combining channel attention and spatial attention to obtain deep features. Based on the deep features, the target category is identified and the target location is predicted by the detection head sub-network; the detection head sub-network includes a classification module of the KAN neural network; the detection head network uses an anchorless distributed regression method to predict the target location.

2. The intelligent identification method for personnel intrusion into river embankments based on target detection according to claim 1, characterized in that, The process of extracting multi-scale semantic and texture features from the riverbank detection image data using a backbone sub-network includes: The input riverbank detection image data is sequentially subjected to convolution, normalization, and activation operations to obtain basic feature representations; Based on the cross-stage splitting structure, dual-path feature extraction is performed on the basic features, and the outputs of the two paths are fused by convolution to obtain multi-level semantic and texture features. Enhanced features of the multi-level semantic and texture features are extracted by a fast spatial pyramid pooling structure, and the enhanced features are aggregated at multiple scales by max pooling operators of different scales to output a multi-scale semantic and texture feature set.

3. The intelligent identification method for personnel intrusion into river embankments based on target detection according to claim 2, characterized in that, The basic feature representation Satisfy the following formula: in, This represents a two-dimensional convolution operation. For batch normalization operations, activation function SiLU is used, where I is the input image data of the river embankment detection. The convolution kernel parameter matrix, For bias terms; The multi-level semantic and texture features Satisfy the following formula: in, Indicates by indivual Subpaths formed by cascading -BN-SiLU Concat() is an aggregation operation. and This represents the basic feature representation of the input in two channels. express Convolutional networks; The multi-scale semantic and texture feature set contains one feature Satisfy the following formula: Wherein, U represents multi-level semantic and textural features. This represents a max-pooling operator with kernel k, where k is 5, 9, or 13. Indicates aggregation operation, for A convolutional network.

4. The intelligent identification method for personnel intrusion into river embankments based on target detection according to claim 1, characterized in that, The process of obtaining deep features by applying a convolutional attention mechanism combining channel attention and spatial attention to the multi-scale pyramid features includes: The pyramid features at each scale are processed through convolutional attention mechanisms combining channel attention and spatial attention to obtain deep features, including: The pyramid features at each scale are input into the channel attention mechanism to extract channel attention features; Multiplying the pyramid features at each scale with the channel attention features yields the preliminary spatial attention features; The preliminary spatial attention features are used to obtain spatial attention features through a spatial attention mechanism; Multiplying the initial spatial attention features and the spatial attention features together yields the attention enhancement features.

5. The intelligent identification method for personnel intrusion into river embankments based on target detection according to claim 4, characterized in that, The step of inputting the pyramid features at each scale into the channel attention mechanism to extract channel attention features includes: The pyramid features at each scale are operated on using global average pooling and max pooling respectively to obtain global average pooling features and max pooling features. The global average pooling feature and the max pooling feature are nonlinearly mapped through a shared two-layer perceptron and then added together. After activation by the Sigmoid function, the channel attention feature is obtained.

6. The intelligent identification method for personnel intrusion into river embankments based on target detection according to claim 5, characterized in that, The global average pooling feature Satisfy the following formula: in, This represents global average pooling, where F represents the pyramid feature at a certain scale, and C represents the number of channels. Represents the set of real numbers; The maximum pooling feature satisfies The following formula: in, This represents global max pooling, where F represents the pyramid feature at a certain scale, and C represents the number of channels. Represents the set of real numbers; The channel attention features Satisfy the following formula: in, For the Sigmoid function, It is a two-layer perceptron. This is a global average pooling feature. For max pooling features, C represents the number of channels. It represents the set of real numbers.

7. The intelligent identification method for personnel intrusion into river embankments based on target detection according to claim 5, characterized in that, The step of obtaining spatial attention features from the preliminary spatial attention features through a spatial attention mechanism includes: The initial spatial features are processed by global average pooling and max pooling respectively to obtain spatial global average pooling features and spatial max pooling features; After concatenating the spatial global average pooling feature and the spatial max pooling feature, through... A convolution operation is performed, and the spatial attention feature is obtained by activating it with the Sigmoid function; wherein, the spatial attention feature Satisfy the following formula: in, For the Sigmoid function, for convolution, Let G be the feature space size, and G be the concatenated feature size resulting from the global average pooling feature and the max pooling feature. This represents a spatially global average pooling feature. For spatial max pooling features, This indicates a splicing operation. It represents the set of real numbers.

8. The intelligent identification method for personnel intrusion into river embankments based on target detection according to claim 1, characterized in that, The step of identifying target categories and predicting target localization based on the deep features through a detection head network includes: Based on deep features, classification branch features and regression branch features are obtained through two layers of convolution; By introducing spline function expansion and linear fusion of channel dimension into the classification branch features through the KAN classification module, higher-order nonlinear features are obtained. Based on the aforementioned high-order nonlinear features, the Sigmoid activation function is used to output the grid category probability at each grid position; For the regression branch, an anchorless distributed regression method is adopted to predict the discrete probability distributions of the bounding box in the left, right, top, and bottom directions respectively. Continuous margin values ​​are generated by expectation calculation, and then the target bounding box is restored by combining the grid center coordinates.

9. A smart identification device for personnel intrusion into river embankments based on target detection, characterized in that, The intelligent identification device for personnel intrusion into river embankments based on target detection includes: a communication unit and a processing unit; The communication unit is used to acquire river embankment detection image data; The processing unit employs a backbone subnetwork to extract multi-scale semantic and texture features from the riverbank detection image data. The backbone subnetwork includes multiple convolutional layers, a cross-stage partial connection feature extraction module with two bottleneck structures, a pyramid pooling layer, and a lightweight cross-layer connection feature extraction module that incorporates position-sensitive attention. The cross-stage partial connection feature extraction module with two bottleneck structures uses a CSP cross-stage partial connection structure composed of two bottleneck branches to extract semantic information and detailed texture features from basic features, respectively, and obtains multi-level semantic and texture features through cross-stage feature fusion. A neck subnetwork aligns and fuses the multi-scale semantic and texture features to obtain deep features. The neck subnetwork is composed of a top-down and bottom-up feature pyramid fusion module combined with a convolutional attention mechanism that incorporates channel attention and spatial attention. The neck sub-network aligns and fuses the multi-scale semantic and texture features to obtain deep features, including: upsampling the multi-scale semantic and texture features at higher levels and convolving and aligning them with adjacent lower-level features before adding them to obtain supplementary semantic features; downsampling the multi-scale semantic and texture features at lower levels and aggregating them to higher levels to obtain enhanced target boundaries and small target features; fusing the supplementary semantic features and the enhanced target boundaries and small target features through multiple layers to obtain multi-scale pyramid features; applying a convolutional attention mechanism of channel attention and spatial attention to the multi-scale pyramid features to obtain deep features; based on the deep features, the detection head sub-network identifies the target category and predicts the target location; the detection head sub-network includes a classification module of a KAN neural network; the detection head network uses an anchorless distributed regression method to predict the target location.

Images