A method and system for identifying cycling behavior in an urban integrated management scenario

Abstract

The application discloses a kind of urban comprehensive management scene under the riding behavior identification method and system, through multimodal data fusion, dynamic feature optimization (adopting SAM2 Hiera backbone network and DINOv2 ViT model to extract multiscale feature, and realize semantic enhancement by content guide attention module), intelligent target detection and tracking (based on natural language prompt word Grounding DINO detection and ByteTrack tracking), the high-precision identification of the helmet wearing behavior of rider is realized;While building data analysis closed loop, real-time generates the clustering and associated factor analysis of hotspot area of illegal behavior, drives the intelligent formulation of safety education strategy.The application significantly improves the robustness and real-time performance of the system in complex environments, supports large-scale deployment in cities, reduces operation and maintenance costs, provides precise and closed-loop technical solutions for cycling safety supervision, and effectively promotes the upgrade of urban traffic safety management from passive response to proactive prevention.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method and system for recognizing cycling behavior in urban integrated management scenarios based on a large CV model. Background Technology

[0002] With the acceleration of urbanization, cycling, as a convenient and low-carbon short-distance travel mode, is playing an increasingly important role in urban transportation systems. However, cycling safety issues are becoming increasingly prominent, with the widespread failure of cyclists to wear helmets contributing to a persistently high rate of head injuries in traffic accidents. Statistics show that approximately 35% of cyclist injuries and fatalities in urban road traffic accidents in my country are directly related to not wearing helmets, highlighting the urgent need for efficient and precise regulatory measures to improve safety levels.

[0003] Currently, urban cycling safety management mainly relies on traditional video surveillance systems and manual inspections. Existing solutions typically use fixed high-definition cameras to capture video streams, combined with basic image processing algorithms (such as background subtraction and moving object detection) to achieve behavior recognition. However, this type of method has significant drawbacks:

[0004] Insufficient recognition accuracy: Traditional methods (such as Haar cascade classifiers or SVM classifiers) are difficult to cope with the variable lighting, occlusion (such as trees and buildings), target scale changes and background interference in complex urban environments, resulting in a recognition accuracy of helmet wearing behavior generally below 60% and a high false alarm rate.

[0005] Weak real-time performance and coverage: manual inspection is inefficient and it is difficult to achieve large-scale, all-weather coverage; fixed camera systems cannot dynamically adapt to the needs of multiple scenarios and lack the ability to integrate and analyze multimodal data (such as infrared thermal imaging and environmental sensors).

[0006] Low level of intelligence: Existing systems rely heavily on predefined rules (such as "helmet color matching"), lack a deep understanding of semantic information, and cannot combine natural language prompts (such as "cyclist wearing a helmet") for adaptive recognition, resulting in a sharp drop in performance in dynamic scenarios (such as rainy days and nights).

[0007] Data analysis is lagging: historical data is only used for simple statistics, lacking in-depth analysis of hotspots, time periods, and related factors (such as weather and population density) of violations, making it difficult to generate targeted safety education strategies.

[0008] In recent years, large-scale computer vision (CV) models (such as SAM, DINOv2, and Grounding DINO) have made breakthroughs in object detection and feature extraction, providing new ideas for behavior recognition. However, existing research mostly focuses on general scenarios (such as pedestrian detection and vehicle recognition) and has not yet been systematically optimized for the specific behavior of "wearing a cycling helmet."

[0009] The failure to effectively integrate multimodal data (such as visible light and infrared images) leads to recognition failure in low light or inclement weather.

[0010] The lack of a content-guided feature fusion mechanism makes it difficult to dynamically highlight key semantic information (such as the helmet outline and the rider's head area).

[0011] Without establishing a closed loop between behavior recognition and urban management decision-making (such as real-time alarms and hotspot area cluster analysis), it is difficult to support the practical application of intelligent traffic management.

[0012] Therefore, there is an urgent need for a cycling behavior recognition technology based on large-scale CV models and oriented towards urban integrated management scenarios. Through multimodal data fusion, dynamic feature optimization, and intelligent analysis, it can achieve high-precision, real-time, and practical safety supervision, thereby filling the gaps in existing technologies and improving the efficiency of urban traffic safety governance. Summary of the Invention

[0013] This invention addresses the shortcomings of existing technologies by providing a method and system for recognizing cycling behavior in urban integrated management scenarios.

[0014] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0015] A method for recognizing cycling behavior in an urban integrated management scenario includes the following steps:

[0016] S1. Data Acquisition: Deploy high-definition cameras and infrared cameras on important urban roads to acquire video streams and simultaneously capture environmental sensor data;

[0017] S2. Data preprocessing: Image enhancement, noise removal, inter-frame deduplication, and keyframe extraction are performed on the video stream to achieve timestamp alignment of multi-source data;

[0018] S3. Multimodal feature extraction and fusion: The image segmentation model is called to extract multi-scale image features, the semantic understanding model is called to extract high-dimensional semantic features, and the multi-scale image features and the high-dimensional semantic features are fused to obtain fused features;

[0019] S4. Target Detection and Tracking: Based on the fused features and predefined natural language prompts, target objects in the video are detected, and the detected targets are tracked across frames.

[0020] S5. Behavior recognition: Based on the tracking results, specific region features of the target object are extracted from the fused features. By matching the region features with pre-stored behavior feature templates, the behavior of the target object is recognized, and an early warning is triggered when a specific behavior is recognized.

[0021] S6. Data Analysis: Perform statistical analysis on historical monitoring data, identify behavioral patterns and high-risk areas, and generate analysis reports.

[0022] Further, in step S3, the multi-scale image features are extracted through the Hiera backbone network in the SAM2 model; the high-dimensional semantic features are extracted through the DINOv2 model; and the multi-scale image features are weighted and fused by the content-guided attention module, guided by the high-dimensional semantic features, to obtain the fused features.

[0023] Furthermore, the content-guided attention module, guided by the high-dimensional semantic features, performs attention-weighted fusion of the multi-scale image features to obtain the fused features. The content-guided attention module includes three steps: feature alignment, channel compression and unification, and content-guided attention fusion. It guides the feature representation of SAM2 with the semantic features of DINOv2, dynamically adjusting the weights of the feature maps to highlight important semantic information.

[0024] Furthermore, in step S4, the fused features and natural language prompts are input into the GroundingDINO model for target detection;

[0025] Using the ByteTrack algorithm, inter-frame correlation and tracking of detected targets are performed based on the intersection-union ratio of the detection boxes and the similarity of appearance features.

[0026] Furthermore, in step S5,

[0027] Based on the tracking results, the rider's head region is located, and the corresponding head region features are cropped from the fused features;

[0028] Calculate the cosine similarity between the head region features and each template in the helmet feature template library;

[0029] When the maximum similarity is less than a preset threshold, it is determined as not wearing a helmet and an alert is triggered.

[0030] Furthermore, the helmet feature template library contains helmet features of different styles and colors, which are obtained through pre-training; the similarity threshold is set to 0.6.

[0031] Furthermore, in step S6, the DBSCAN clustering algorithm is used to identify high-risk areas based on the location and frequency of helmet-less behavior.

[0032] The Apriori association rule algorithm was used to explore the association between the behavior of not wearing a helmet and the time of occurrence and weather conditions.

[0033] Furthermore, the clustering parameters of the DBSCAN clustering algorithm are set as follows: neighborhood radius ε = 50 meters, minimum number of samples is 20; the association relationship between "not wearing a helmet, time period, and weather" is mined through association rule learning, and when the support of the association rule mined by the Apriori algorithm is ≥15%, it is determined to be a strong association pattern;

[0034] Furthermore, in step S3, the SAM2 Hiera backbone network outputs multi-scale features at four levels, including: 144×88×88, 288×44×44, 576×22×22, and 1152×11×11. The DINOv2 ViT model outputs high-dimensional semantic features of 1024×37×37, which are downsampled to 11×11 by bilinear interpolation. The number of channels is then adjusted to 1152 through depthwise separable convolution to match 1152×11×11.

[0035] This invention also discloses a cycling behavior recognition system for urban integrated management scenarios, used to implement the above-mentioned method, specifically including:

[0036] The data acquisition and preprocessing module is used to acquire and preprocess the surveillance video stream of the target area.

[0037] The multimodal feature extraction and fusion module is connected to the data acquisition and preprocessing module. It is used to call the large image segmentation model to extract multi-scale image features, call the large semantic understanding model to extract high-dimensional semantic features, and perform feature fusion.

[0038] The target detection and tracking module is connected to the multimodal feature extraction and fusion module and is used for target detection and cross-frame tracking based on fused features and natural language prompts;

[0039] The behavior recognition and early warning module is connected to the target detection and tracking module. It is used to extract target area features based on tracking results and perform behavior matching and recognition, and to trigger an early warning when a specific behavior is recognized.

[0040] The data analysis module, connected to the behavior recognition and early warning module, is used to perform statistical analysis and report generation on historical behavior data.

[0041] Compared with the prior art, the advantages of the present invention are as follows:

[0042] 1. By fusing multi-scale features of the SAM2 Hiera backbone network and the DINOv2 ViT model, and combining the content-guided attention mechanism (CGA), key semantic information (such as helmet outline and rider head area) is dynamically enhanced, significantly improving the accuracy of helmet wearing behavior recognition and significantly reducing the false alarm rate, making the system more reliable in complex urban environments (such as occlusion and changes in lighting).

[0043] 2. Integrating visible light and infrared multimodal data enables all-weather coverage, effectively addressing adverse weather conditions such as low light, rain, and fog, as well as background interference, ensuring continuous and stable operation of the system in dynamic urban scenarios, and avoiding the failure issues of traditional solutions in non-ideal environments.

[0044] 3. The data preprocessing module is deployed on edge computing nodes, supporting local real-time processing of video streams, achieving high throughput and low latency response, meeting the real-time requirements of large-scale monitoring of urban main roads, avoiding cloud transmission bottlenecks, and improving system scalability.

[0045] 4. The behavior recognition and data analysis modules form a closed loop: real-time triggering of audible and visual alarms and remote push notifications enable immediate intervention in violations; through hotspot area clustering and correlation factor analysis, high-risk scenarios are accurately located, and targeted safety education strategies are generated to promote a shift in safety management from passive response to proactive prevention.

[0046] 5. It is compatible with existing urban monitoring infrastructure, reducing the need for hardware upgrades; relying on pre-trained feature template libraries and automated analysis capabilities, it reduces the complexity of manual annotation and operation and maintenance, improves the ease of use and long-term sustainability of the system, and provides a cost-effective intelligent solution for urban management. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of an application environment for the cycling behavior recognition method in the context of urban integrated management, as described in this embodiment of the invention.

[0049] Figure 2 This is a flowchart of the multimodal feature extraction and fusion process in an embodiment of the present invention;

[0050] Figure 3 This is a flowchart of the target detection and tracking process in an embodiment of the present invention;

[0051] Figure 4This is a flowchart of the data analysis process in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] like Figure 1 As shown, the present invention provides a method for recognizing cycling behavior in an urban integrated management scenario based on a large CV model. The main process is as follows:

[0054] Step 1: Data Collection

[0055] High-definition cameras (1920×1080 resolution, 30fps) are deployed along key urban roads to capture video streams, and infrared cameras are used in conjunction to ensure effective data collection at night and in low-light conditions. Simultaneously, integrated environmental sensors capture auxiliary information such as weather conditions and light intensity. All devices support 4G / 5G / Wi-Fi 6 multi-mode transmission protocols to ensure highly reliable real-time uploading and aggregation of collected data.

[0056] Step 2: Data Preprocessing

[0057] Deployed on edge computing nodes, it performs lightweight preprocessing operations: image enhancement through algorithms to improve details in low-light areas; noise removal using a combination of Gaussian and median filtering; inter-frame deduplication and keyframe extraction of the video stream to reduce the pressure on subsequent processing; and simultaneous alignment of timestamps from multiple data sources to ensure spatiotemporal consistency.

[0058] Step 3: Multimodal Feature Extraction and Fusion

[0059] like Figure 2 As shown, the core technology layer enables accurate extraction and collaborative enhancement of multimodal features, specifically including multimodal feature extraction and multimodal fusion.

[0060] 3.1 Multimodal Feature Extraction

[0061] 1) Call the SAM2 Hiera backbone network to output multi-scale features at four levels (S1: 144×88×88, S2: 288×44×44, S3: 576×22×22, S4: 1152×11×11), extract multi-scale features from the image, and accurately capture local details and segmentation masks such as the cyclist's head and helmet.

[0062] 2) The DINOv2 ViT module is used. A pre-trained DINOv2 ViT-L / 14 model processes video frames, outputting high-dimensional semantic features of 1024×37×37. These features are downsampled to 11×11 via bilinear interpolation, and then the number of channels is adjusted to 1152 using depthwise separable convolution, resulting in semantic features V1 that match S4. This effectively captures target category and global association information. High-dimensional semantic features are extracted through self-supervised learning, and these features excel in capturing the global semantic information of the image.

[0063] 3.2 Multimodal Fusion

[0064] DINOv2 and SAM2 feature fusion is achieved through dense feature collaboration and a content-guided attention module (CGA). High-dimensional semantic features extracted by the self-supervised DINOv2 model are used to guide multi-scale feature fusion. This mechanism allows features at different scales to better complement each other during the fusion process, enabling the model to capture richer image information, effectively integrating everything from low-level texture details to high-level semantic concepts. The specific steps are as follows:

[0065] 1) Feature alignment:

[0066] The SAM2 Hiera module outputs four levels of features (S1: 144×88×88, S2: 288×44×44, S3: 576×22×22, S4: 1152×11×11).

[0067] The output features of the DINOv2 ViT module (1024×37×37) are downsampled to 11×11 by bilinear interpolation, and the number of channels is adjusted to 1152 by depthwise separable convolution to match S4, and is denoted as V1.

[0068] 2) Channel compression and unification:

[0069] The number of channels in S1~S4 and V1 is uniformly compressed to 64 using 1×1 convolution, as shown in the formula:

[0070] ,

[0071] 3) Content-guided attention fusion:

[0072] The CGA module uses the semantic features of DINOv2 as guidance and enhances the feature representation of SAM2 through an attention mechanism. This module dynamically adjusts the weights of the feature maps, highlighting important semantic information, and ultimately obtains fused features. This feature integrates the advantages of DINOv2 and SAM2 in key areas, providing a more discriminative feature representation for subsequent behavior recognition.

[0073] Step 4: Target Detection and Tracking

[0074] like Figure 3 As shown, a technical architecture of "Grounding DINO detection + ByteTrack tracking + feature matching re-identification" is adopted. This architecture ensures both detection and tracking accuracy and engineering feasibility. First, the Grounding DINO model is used based on the fused features. Text prompts are used to detect cyclists and bicycles. Then, the ByteTrac algorithm is used to achieve inter-frame target tracking through IOU and appearance features. The specific process is as follows:

[0075] 1) Object detection: fusing image features The Grounding DINO model is input with natural language prompts ("cyclist, bicycle, helmet") and outputs bounding boxes and category information for cyclists (confidence threshold ≥ 0.7) and bicycles, achieving precise target localization. Natural language descriptions are used to locate specific targets.

[0076] 2) Basic tracking: The ByteTrack algorithm is used for inter-frame target association: the detection boxes in the current frame are sorted by confidence. High-confidence boxes (≥0.6) are directly matched with historical trajectories, and low-confidence boxes (0.1-0.6) are included in the candidate set. The association cost is calculated by weighting the IOU (Intersection over Union) and appearance feature similarity to achieve cross-frame target matching. The association threshold is set to 0.5.

[0077] Step 5: Behavior Recognition

[0078] Based on the tracking results, the system performs real-time analysis of target behavior, with a core focus on identifying unsafe behaviors such as not wearing a helmet. Once a violation is detected, the system immediately triggers an on-site audible and visual alarm and a remote information push notification mechanism, simultaneously recording key data such as the time, location, and type of behavior, providing a complete basis for subsequent data analysis. The specific process is as follows:

[0079] 1) Target Region Extraction: Based on the tracking results, the cyclist's head region is located, and features are fused. By cropping the corresponding region features, the head features are obtained. .

[0080] 2) Feature matching calculation: Construct a helmet feature template library T (containing features of helmets of different styles and colors, obtained through pre-training), and calculate... Cosine similarity with each template in T .

[0081] Behavior judgment: Set the similarity threshold Sim_threshold to 0.6. When the maximum similarity is less than the set similarity threshold, it is judged as not wearing a helmet and the behavior confidence score is output.

[0082] 3) Data recording: Record relevant data, including time, location, behavior type, etc., for subsequent analysis.

[0083] Step Six: Data Analysis

[0084] like Figure 4 As shown, statistical analysis of historical data identifies hotspots and time periods for cycling violations. Multimodal analysis techniques are used to uncover patterns and regularities in these violations. Targeted safety education recommendations are then generated to provide decision-making support for relevant government departments.

[0085] Based on multimodal data mining and association rule learning, the system transforms monitoring data into security protection strategies, specifically including:

[0086] 1) Data Input

[0087] Historical monitoring data (including behavioral records: time, location, whether a helmet was worn; environmental data: weather, traffic flow; user data: cyclist photos, geographical location information of cycling routes);

[0088] 2) Core Analysis Algorithm

[0089] ① Hotspot area discovery: The DBSCAN clustering algorithm is used to identify high-risk areas (such as within 30 meters of a school gate or at intersections without traffic lights) based on the frequency and location of helmet-wearing behavior. Clustering parameters are set as follows: ε = 50 meters, minimum number of samples = 20.

[0090] ② Behavioral Pattern Analysis: Through association rule learning (Apriori algorithm), the association between "not wearing a helmet - time period - weather" is mined. The support formula is:

[0091]

[0092] Where X represents "not wearing a helmet", Y represents "7:30-8:30 school hours", and N represents the total number of samples. When the support is ≥15%, it is determined to be a strong correlation pattern.

[0093] ③ Personalized Education Suggestion Generation: Based on the cyclist's historical behavior (e.g., not wearing a helmet ≥ 3 times per week), combined with a pre-trained safety education knowledge base (including helmet safety cases and cycling regulations), customized suggestions are generated by fine-tuning the large language model through Prompt Tuning.

[0094] 3) Output format:

[0095] The government provides heat maps of high-risk areas and monthly statistics reports on violations (Excel / PDF format).

[0096] The following is a specific embodiment of the application of the method of the present invention, including the following steps:

[0097] Step 1: Data Acquisition and Preprocessing

[0098] 1) Install high-definition cameras on roads surrounding a middle school to collect video streams in real time.

[0099] 2) The video stream undergoes preliminary processing through edge computing devices, including resolution adjustment and frame rate conversion.

[0100] 3) The processed video frames are transmitted to the central server via the network to complete the timestamp alignment.

[0101] Step 2: Multimodal Feature Extraction and Fusion

[0102] 1) The central server loads the pre-trained SAM2 and DINOv2 models.

[0103] 2) Input the video frames into the SAM2 and DINOv2 models respectively, and extract image features and semantic features.

[0104] 3) The two features are fused through a content-guided attention module to form a multimodal feature representation. .

[0105] Step 3: Target Detection and Tracking

[0106] 1) Features after fusion The Grounding DINO model is invoked as input using cue words composed of specific natural language descriptions to detect cyclists and bicycles in the video.

[0107] 2) Input the detection results into the ByteTrack algorithm model to perform target tracking and ensure consistency across frames.

[0108] 3) Analyze the tracking results to determine whether the cyclist is wearing a helmet.

[0109] Step Four: Behavioral Analysis and Early Warning

[0110] 1) If the system detects that a person is not wearing a helmet, it can trigger an on-site alarm and send a warning message to the management personnel.

[0111] 2) The system records relevant data, including time, location, and behavior type, for subsequent analysis.

[0112] 3) Managers can view the warning details and take appropriate action via mobile devices.

[0113] Step 5: Data Analysis and Report Generation

[0114] 1) The system regularly performs statistical analysis on historical data to identify hotspots and time periods for illegal cycling near schools.

[0115] 2) Utilize multimodal analysis technology to uncover patterns and rules in minors' cycling behavior.

[0116] 3) Generate targeted safety education recommendations and push them to parents, schools, and government departments via web and mobile applications.

[0117] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0118] In another embodiment, a cycling behavior recognition system for urban integrated management scenarios is provided, which corresponds one-to-one with the cycling behavior recognition methods in the above embodiments. Specifically, it includes:

[0119] Data acquisition module: used to deploy high-definition cameras and infrared cameras on important urban roads to acquire video streams and simultaneously capture environmental sensor data;

[0120] Data preprocessing module: Deployed on edge computing nodes, it performs image enhancement, noise removal, inter-frame deduplication, and keyframe extraction, and realizes multi-source data timestamp alignment;

[0121] Multimodal feature extraction and fusion module: used to call the SAM2 Hiera backbone network and DINOv2 ViT model to extract multi-scale features and high-dimensional semantic features, and to achieve feature fusion through the content-guided attention module (CGA);

[0122] The target detection and tracking module is used to detect cyclists and bicycles based on fused features and natural language prompts, using the GroundingDINO model and the ByteTrack algorithm for target tracking.

[0123] Behavior recognition module: used to extract features of the rider's head area based on the tracking results, calculate the cosine similarity with the helmet feature template library, determine whether a helmet is being worn, and trigger on-site sound and light alarms and remote information push;

[0124] Data analysis module: Used to perform statistical analysis on historical monitoring data, identify hotspots and time periods for illegal cycling behavior, and generate targeted safety education suggestions.

[0125] Specific limitations regarding the cycling behavior recognition system can be found in the limitations of the cycling behavior recognition method described above, and will not be repeated here. Each module in the above system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0126] In another embodiment, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or function. The processor described in this embodiment can be used for the operation of a cycling behavior recognition method.

[0127] In another embodiment, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor; these instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0128] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the cycling behavior recognition method in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by a processor.

[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0130] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0131] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for recognizing cycling behavior in an urban integrated management scenario, characterized in that, Includes the following steps: S1. Data Acquisition: Deploy high-definition cameras and infrared cameras on important urban roads to acquire video streams and simultaneously capture environmental sensor data; S2. Data preprocessing: Image enhancement, noise removal, inter-frame deduplication, and keyframe extraction are performed on the video stream to achieve timestamp alignment of multi-source data; S3. Multimodal feature extraction and fusion: The image segmentation model is called to extract multi-scale image features, the semantic understanding model is called to extract high-dimensional semantic features, and the multi-scale image features and the high-dimensional semantic features are fused to obtain fused features; S4. Target Detection and Tracking: Based on the fused features and predefined natural language prompts, target objects in the video are detected, and the detected targets are tracked across frames. S5. Behavior recognition: Based on the tracking results, specific region features of the target object are extracted from the fused features. By matching the region features with pre-stored behavior feature templates, the behavior of the target object is recognized, and an early warning is triggered when a specific behavior is recognized. S6. Data Analysis: Perform statistical analysis on historical monitoring data, identify behavioral patterns and high-risk areas, and generate analysis reports.

2. The method according to claim 1, characterized in that, In step S3, the multi-scale image features are extracted using the Hiera backbone network in the SAM2 model; the high-dimensional semantic features are extracted using the DINOv2 model; and the multi-scale image features are weighted and fused using the content-guided attention module, guided by the high-dimensional semantic features, to obtain the fused features.

3. The method according to claim 2, characterized in that, The content-guided attention module, guided by the high-dimensional semantic features, performs attention-weighted fusion of the multi-scale image features to obtain the fused features. The content-guided attention module includes three steps: feature alignment, channel compression and unification, and content-guided attention fusion. It guides the feature representation of SAM2 with the semantic features of DINOv2, dynamically adjusting the weights of the feature maps to highlight important semantic information.

4. The method according to claim 2, characterized in that, In the multimodal feature extraction and fusion module, the SAM2Hiera backbone network outputs multi-scale features at four levels, including: 144×88×88, 288×44×44, 576×22×22, and 1152×11×11. The DINOv2 ViT model outputs high-dimensional semantic features of 1024×37×37, which are downsampled to 11×11 by bilinear interpolation. The number of channels is then adjusted to 1152 through depthwise separable convolution to match 1152×11×11.

5. The method according to claim 1, characterized in that, In step S4, the fused features and natural language prompts are input into the Grounding DINO model for target detection; Using the ByteTrack algorithm, inter-frame correlation and tracking of detected targets are performed based on the intersection-union ratio of the detection boxes and the similarity of appearance features.

6. The method according to claim 1, characterized in that, In step S5 Based on the tracking results, the rider's head region is located, and the corresponding head region features are cropped from the fused features; Calculate the cosine similarity between the head region features and each template in the helmet feature template library; When the maximum similarity is less than a preset threshold, it is determined as not wearing a helmet and an alert is triggered.

7. The method according to claim 6, characterized in that, The helmet feature template library contains features of helmets of different styles and colors, which are obtained through pre-training; the similarity threshold is set to 0.

6.

8. The method according to claim 1, characterized in that, In step S6 The DBSCAN clustering algorithm is used to identify high-risk areas based on the location and frequency of helmet-less behavior. The Apriori association rule algorithm was used to explore the association between the behavior of not wearing a helmet and the time of occurrence and weather conditions.

9. The method according to claim 1, characterized in that, The clustering parameters of the DBSCAN clustering algorithm are set as follows: neighborhood radius ε = 50 meters, minimum number of samples is 20; the association relationship between "not wearing a helmet, time period, and weather" is mined by association rule learning. When the support of the association rule mined by the Apriori algorithm is ≥15%, it is determined to be a strong association pattern.

10. A cycling behavior recognition system for urban integrated management scenarios, characterized in that, The method for implementing any one of claims 1 to 9 specifically includes: The data acquisition and preprocessing module is used to acquire and preprocess the surveillance video stream of the target area. The multimodal feature extraction and fusion module is connected to the data acquisition and preprocessing module. It is used to call the large image segmentation model to extract multi-scale image features, call the large semantic understanding model to extract high-dimensional semantic features, and perform feature fusion. The target detection and tracking module is connected to the multimodal feature extraction and fusion module and is used for target detection and cross-frame tracking based on fused features and natural language prompts; The behavior recognition and early warning module is connected to the target detection and tracking module. It is used to extract target area features based on tracking results and perform behavior matching and recognition, and to trigger an early warning when a specific behavior is recognized. The data analysis module, connected to the behavior recognition and early warning module, is used to perform statistical analysis and report generation on historical behavior data.

Images