Traffic behavior recognition method and device, computer device and storage medium

Abstract

The embodiment of the application belongs to the field of artificial intelligence, and relates to a traffic behavior recognition method, comprising the following steps: obtaining sensor data of each sensor in a user terminal; inputting the sensor data of each sensor into a residual network to perform feature extraction of a local receptive field, and obtaining shallow features and local features of each sensor; inputting the shallow features and the local features of each sensor into a time domain convolutional neural network to perform feature extraction of a global receptive field, and obtaining deep features of each sensor; inputting the deep features of each sensor into an attention network to obtain weighted features of each sensor; splicing the shallow features and the weighted features of each sensor to obtain a comprehensive feature vector of a user; and inputting the comprehensive feature vector into a classification layer to obtain a traffic behavior recognition result of the user. The application also provides a traffic behavior recognition device, a computer device and a storage medium. The application improves the accuracy and efficiency of traffic behavior recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a traffic behavior recognition method, device, computer equipment, and storage medium. Background Technology

[0002] With the popularization of smart devices, traffic behavior recognition through sensor data has become an important research direction in user behavior recognition. It aims to determine the user's traffic mode (such as walking, cycling, driving, etc.) by analyzing data from sensors in user devices (such as accelerometers, gyroscopes, etc.), and can be widely used in fields such as intelligent transportation, health monitoring, and car insurance claims.

[0003] Current traffic behavior recognition technologies primarily rely on deep learning models, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). However, CNNs often fail to fully utilize temporal information when processing time-series data, affecting recognition accuracy. Secondly, while RNNs can capture temporal dependencies, their complexity and high computational resource consumption result in low application efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a traffic behavior recognition method, device, computer equipment, and storage medium to improve the accuracy and efficiency of traffic behavior recognition.

[0005] To address the aforementioned technical problems, this application provides a traffic behavior recognition method, employing the following technical solution:

[0006] Acquire sensor data from each sensor in the user terminal;

[0007] The sensor data of each sensor is input into the residual network to extract features of the local receptive field, thereby obtaining the shallow features and local features of each sensor.

[0008] The shallow and local features of each sensor are input into a temporal convolutional neural network to extract features from the global receptive field, thereby obtaining the deep features of each sensor.

[0009] The deep features of each sensor are input into the attention network to obtain the weighted features of each sensor;

[0010] The shallow features and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector;

[0011] The comprehensive feature vector is input into the classification layer to obtain the user's traffic behavior recognition result.

[0012] To address the aforementioned technical problems, this application also provides a traffic behavior recognition device, which employs the following technical solution:

[0013] The data acquisition module is used to acquire sensor data from various sensors in the user terminal.

[0014] The residual processing module is used to input the sensor data of each sensor into the residual network to extract the features of the local receptive field, thereby obtaining the shallow features and local features of each sensor.

[0015] The temporal processing module is used to input the shallow and local features of each sensor into a temporal convolutional neural network to extract features from the global receptive field, thereby obtaining the deep features of each sensor.

[0016] An attention processing module is used to input the deep features of each sensor into an attention network to obtain the weighted features of each sensor;

[0017] The feature stitching module is used to stitch together the shallow features and weighted features of each sensor to obtain the user's comprehensive feature vector;

[0018] The behavior recognition module is used to input the comprehensive feature vector into the classification layer to obtain the user's traffic behavior recognition result.

[0019] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0020] Acquire sensor data from each sensor in the user terminal;

[0021] The sensor data of each sensor is input into the residual network to extract features of the local receptive field, thereby obtaining the shallow features and local features of each sensor.

[0022] The shallow and local features of each sensor are input into a temporal convolutional neural network to extract features from the global receptive field, thereby obtaining the deep features of each sensor.

[0023] The deep features of each sensor are input into the attention network to obtain the weighted features of each sensor;

[0024] The shallow features and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector;

[0025] The comprehensive feature vector is input into the classification layer to obtain the user's traffic behavior recognition result.

[0026] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0027] Acquire sensor data from each sensor in the user terminal;

[0028] The sensor data of each sensor is input into the residual network to extract features of the local receptive field, thereby obtaining the shallow features and local features of each sensor.

[0029] The shallow and local features of each sensor are input into a temporal convolutional neural network to extract features from the global receptive field, thereby obtaining the deep features of each sensor.

[0030] The deep features of each sensor are input into the attention network to obtain the weighted features of each sensor;

[0031] The shallow features and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector;

[0032] The comprehensive feature vector is input into the classification layer to obtain the user's traffic behavior recognition result.

[0033] Compared with existing technologies, the embodiments of this application have the following main advantages: Sensor data from each sensor in the user terminal is acquired and input into a residual network for feature extraction of the local receptive field, resulting in shallow and local features of each sensor. This preserves the fine-grained information of the sensor data and provides a foundation for global feature extraction. The shallow and local features of each sensor are then input into a temporal convolutional neural network for feature extraction of the global receptive field, resulting in deep features of each sensor. The temporal convolutional neural network enhances the capture of time-dependent features and, combined with shallow features, compensates for the lack of perception of local details. The deep features of each sensor are then input into an attention network. The attention network dynamically adjusts feature weights to ensure the model focuses on the most representative channel features, reducing computational overhead and improving computational efficiency, resulting in weighted features of each sensor. The shallow and weighted features of each sensor are concatenated to obtain a comprehensive feature vector of the user, effectively reflecting the user's traffic behavior. Inputting this comprehensive feature vector into a classification layer allows for accurate identification of the user's traffic behavior. Attached Figure Description

[0034] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0036] Figure 2 This is a flowchart of an embodiment of the traffic behavior recognition method according to this application;

[0037] Figure 3 This is a schematic diagram of a structure of one embodiment of the traffic behavior recognition device according to this application;

[0038] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0040] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0042] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0043] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0044] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, and desktop computer, etc.

[0045] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0046] It should be noted that the traffic behavior recognition method provided in this application embodiment is generally executed by a server, and correspondingly, the traffic behavior recognition device is generally installed in the server.

[0047] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0048] Continue to refer to Figure 2 A flowchart of an embodiment of the traffic behavior recognition method according to this application is shown. The traffic behavior recognition method includes the following steps:

[0049] Step S201: Obtain sensor data from each sensor in the user terminal.

[0050] In this embodiment, the traffic behavior recognition method operates on an electronic device (e.g., Figure 1 The server shown can communicate with the terminal device via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, Wi-Fi connections, Bluetooth connections, Wi-Fi connections, Zigbee connections, UWB (ultra-Width band) connections, and other currently known or future wireless connection methods.

[0051] Specifically, sensor data is acquired from multiple sensors in the user terminal (such as smartphones, wearable devices, etc.). Sensors can include accelerometers, gyroscopes, geomagnetic sensors, etc., which can capture the user's motion state and environmental information.

[0052] Step S202: Input the sensor data of each sensor into the residual network to extract the features of the local receptive field, and obtain the shallow features and local features of each sensor.

[0053] Specifically, sensor data from each sensor is input into a ResNet, which effectively extracts local receptive field features, thus preserving local details in the sensor data. In this process, shallow features are low-level features extracted by the early layers of the ResNet, while local features are high-level features extracted by the later layers.

[0054] Step S203: Input the shallow and local features of each sensor into the temporal convolutional neural network to extract the features of the global receptive field and obtain the deep features of each sensor.

[0055] Specifically, shallow and local features from each sensor are input into a temporal convolutional neural network (TCN). The TCN uses dilated convolutions to extract features from the global receptive field, enabling it to capture long-term dependent sequence information. Shallow features contain low-level information from the sensor data and play a crucial role in enhancing the TCN's ability to perceive key temporal features, particularly in shallow temporal relationships. Local features are the final output of the residual network, containing complex local patterns and high-level information after multi-layer convolution processing. Inputting local features into the TCN helps extract and enhance fine-grained features within the global receptive field.

[0056] Temporal convolutional neural networks use dilated convolution to extract features from the global receptive field, capturing sequence information over long time spans. Shallow features help the network enhance its ability to perceive important features by preserving key temporal information during convolution; local features provide higher-level local information, compensating for the limitations of dilated convolution in processing fine-grained information.

[0057] The temporal convolutional neural network ultimately outputs deep features that reflect the time-series patterns of user behavior.

[0058] Step S204: Input the deep features of each sensor into the attention network to obtain the weighted features of each sensor.

[0059] Specifically, the deep features of each sensor are input into an attention network. Each sensor's deep features have multiple channels. The attention network calculates the relationship between the channels through an attention mechanism and dynamically adjusts the weights of each channel, enabling the model to focus more on the feature channels that contribute most to traffic behavior recognition, thus obtaining the weighted features of each sensor.

[0060] Step S205: The shallow features and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector.

[0061] Specifically, for each type of sensor, the weighted features of the sensor are concatenated with the shallow features obtained through the residual network, usually at the channel dimension, to integrate the multi-level information of the sensor and form a high-dimensional comprehensive feature vector, making full use of the complementarity of the features at each layer.

[0062] Step S206: Input the comprehensive feature vector into the classification layer to obtain the user's traffic behavior recognition result.

[0063] Specifically, the comprehensive feature vector is input into the classification layer (such as a fully connected layer). The classification layer analyzes and learns the comprehensive feature vector and outputs the final traffic behavior recognition result to determine the user's current traffic mode, such as walking, cycling, or driving.

[0064] It should be noted that this application uses a traffic behavior recognition model to identify user traffic behavior. The traffic behavior recognition model includes the residual network, temporal convolutional neural network, attention network, and classification layer mentioned above.

[0065] In this embodiment, sensor data from each sensor in the user terminal is acquired and input into a residual network for feature extraction of the local receptive field, yielding shallow and local features of each sensor. This preserves the fine-grained information of the sensor data and provides a foundation for global feature extraction. The shallow and local features of each sensor are then input into a temporal convolutional neural network for feature extraction of the global receptive field, yielding deep features of each sensor. The temporal convolutional neural network enhances the capture of time-dependent features and, combined with shallow features, compensates for the lack of perception of local details. The deep features of each sensor are then input into an attention network. The attention network dynamically adjusts feature weights to ensure that the model focuses on the most representative channel features, reducing computational overhead and improving the model's computational efficiency, resulting in weighted features of each sensor. The shallow and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector, effectively reflecting the user's traffic behavior. Inputting the comprehensive feature vector into a classification layer allows for accurate identification of the user's traffic behavior.

[0066] Furthermore, after step S201 above, the method may further include: performing integrity detection on the sensor data of each sensor to obtain the missing values ​​of each sensor; and filling in the detected missing values ​​using Lagrange interpolation.

[0067] Specifically, after acquiring sensor data from each sensor, a data integrity check is performed to examine whether there are any missing or abnormal data, such as data loss due to signal loss, equipment malfunction, or external environmental influences. Through this check, missing values ​​in the sensor data are identified.

[0068] Missing values ​​detected are filled using Lagrange interpolation. Lagrange interpolation is a commonly used mathematical method to smoothly interpolate between known data points, thus filling in missing data. Sensor data has temporal continuity; through Lagrange interpolation, missing data can be recovered without significantly affecting the accuracy of the data, ensuring the integrity of subsequent feature extraction processes and the stability of the model.

[0069] After missing value imputation, the sensor data from each sensor is input into the residual network for feature extraction of the local receptive field.

[0070] In this embodiment, by detecting missing values ​​in the sensor data and filling in the missing values ​​using Lagrange interpolation, the continuity and integrity of the sensor data are restored, reducing errors caused by incomplete data. This ensures that the residual network and the temporal convolutional neural network can obtain more reliable input data during the feature extraction stage, thereby improving the accuracy of subsequent feature extraction.

[0071] Furthermore, step S203 may include: for each sensor, integrating the shallow features and local features of the sensor to obtain integrated features; inputting the integrated features into the first dilated convolutional layer in the temporal convolutional neural network for feature extraction to obtain a first feature; inputting the first feature into a nonlinear activation function to obtain a second feature, and performing a normalization operation on the second feature to obtain a third feature; using the third feature as a new integrated feature and inputting it into the next dilated convolutional layer in the temporal convolutional neural network for iteration, until the deep features of the sensor are generated based on the output of the last dilated convolutional layer.

[0072] Specifically, in a temporal convolutional neural network (TCN), shallow features and local features of each sensor are integrated, for example, by splicing or weighted combination, to make shallow features and local features complementary.

[0073] Temporal convolutional networks have multiple dilated convolutional layers. The integrated features are first input into the first dilated convolutional layer for dilation convolution. By expanding the receptive field, dilated convolution can capture dependencies over a longer time span and extract the first feature, which contains important temporal information from the integrated features and initially constructs the global features of the sensor data.

[0074] The first feature is processed by a nonlinear activation function (such as ReLU) to generate the second feature. The nonlinear activation function introduces a nonlinear factor, enhancing the model's expressive power. Subsequently, a normalization operation (such as Batch Normalization) is performed to adjust the second feature to a uniform numerical distribution, resulting in the third feature. The normalization process helps improve the model's training stability and convergence speed.

[0075] The third feature, as a new integrated feature, is input into the next dilated convolutional layer for iterative processing. Through this iterative process, the model extracts features layer by layer, with each layer further expanding its global receptive field based on the output of the previous layer. Finally, after multiple layers of dilated convolutional processing, deep features for each sensor are generated. These deep features integrate local and global information from sensor data, enabling a more comprehensive description of the user's traffic behavior.

[0076] In this embodiment, for each sensor, the shallow and local features of the sensor are integrated to obtain integrated features. The multi-layer dilated convolutional layers in the temporal convolutional neural network iterate over the integrated features, which can expand the receptive field, capture long-term dependent features, and gradually refine and enhance the expressive power of these features. Nonlinear activation and normalization operations optimize the nonlinear representation of features and data distribution, making the model more efficient and stable in subsequent convolutional processing and feature extraction. The final generated deep features contain both the global temporal relationship of sensor data and retain local fine-grained information. The rich information content ensures the accuracy of subsequent traffic behavior recognition.

[0077] Furthermore, step S204 may include: for each sensor's deep features, calculating the channel information of each channel based on the feature map of each channel in the deep features; calculating the channel information of each channel through an attention mechanism to obtain the attention weight of each channel; and adding the attention weight of each channel to the feature map of each channel to obtain weighted features.

[0078] Specifically, the deep features of each sensor typically consist of multiple channels, each representing the network's response to a specific aspect or class of features in the input data. The feature map in each channel is analyzed to extract the channel information. This channel information can be a numerical value, thus compressing the information from each feature map into a single channel information value.

[0079] Channel information is input into the attention network. The attention network employs an attention mechanism based on the relationships between feature channels, which calculates the importance of each channel and assigns corresponding weights based on this channel information. This process can be implemented using fully connected layers or more complex operations (such as nonlinear transformations), and the generated attention weights reflect the relative importance of each channel in the current recognition task.

[0080] The generated attention weights are applied to the feature maps of their respective channels; that is, all values ​​in each feature map are multiplied by the attention weight of that channel. This weighting process allows the model to focus more on feature channels that contribute more to traffic behavior recognition while suppressing less important channels, and the weighted feature maps are used as weighted features. These weighted features contain optimized deep features, providing more accurate input for subsequent processing.

[0081] In this embodiment, for each sensor's deep features, channel information for each channel is calculated based on the feature maps of each channel in the deep features, resulting in a global feature representation for each channel. Based on the channel information, the attention weight of each channel is dynamically adjusted through an attention mechanism, automatically identifying and highlighting feature channels important to the traffic behavior recognition task, while unnecessary or redundant features are suppressed, reducing computational redundancy and enabling the model to exhibit higher accuracy and computational efficiency when processing complex high-dimensional features. The attention weights of each channel are added to the feature maps of each channel to obtain weighted features, effectively integrating the deep information of the sensor data and ensuring the accuracy of subsequent model processing.

[0082] Furthermore, the steps described above for calculating the channel information of each channel based on the feature maps of each channel in the deep features may include: performing global average pooling or global max pooling on the feature maps of each channel in the deep features to obtain the pooling values ​​of each channel; and using the pooling values ​​of each channel as the channel information of each channel.

[0083] Specifically, the feature map of each channel in the deep feature set contains the channel's response values ​​at different spatial locations or time points. To simplify these high-dimensional feature maps and extract representative global information, global pooling can be used to average all values ​​in the feature map, generating a single numerical value representing the channel's average response; alternatively, global max pooling can be used to extract the maximum value in the feature map, representing the channel's strongest response. These two pooling methods can effectively compress the information in the feature map, allowing the model to obtain the channel's global features with lower computational complexity.

[0084] The pooling values ​​generated by global pooling are considered channel information for each channel, representing the global response of each channel in the deep features across the entire input data. Through pooling operations, the model can measure the relative importance of each channel, thereby more accurately allocating weights in the attention mechanism.

[0085] In this embodiment, the high-dimensional feature map of each channel is compressed into a single pooling value through global pooling operation, which simplifies the complex deep feature representation. The pooling value can reflect the global response of each channel, enabling the attention mechanism to allocate more reasonable weights based on channel information and highlight the importance of key feature channels. The global pooling operation has stability and robustness when extracting global information from the feature map, reducing the bias caused by local anomalies or noise, and providing data support for the subsequent processing of the attention mechanism.

[0086] Furthermore, step S205 may include: performing feature stitching on the shallow features and weighted features of each sensor along the channel dimension to obtain the stitched features of each sensor; and performing a fusion operation on the stitched features of each sensor to obtain the user's comprehensive feature vector.

[0087] Specifically, shallow features from each sensor are concatenated with weighted features processed by an attention mechanism along the channel dimension. Shallow features capture preliminary temporal information, while weighted features, adjusted by the attention mechanism, highlight key features for traffic behavior recognition. Concatenating these features along the channel dimension creates a richer feature representation, resulting in the concatenated features. This concatenation operation increases feature diversity and information content, ensuring the model can utilize feature information from different levels in subsequent processing.

[0088] The stitched features contain multi-dimensional information from various sensors. To further integrate this information and make it more efficient and useful in the classification stage, a fusion operation is performed on the stitched features, transforming the high-dimensional stitched features into a more compact and representative comprehensive feature vector. The comprehensive feature vector integrates information from different sensors and retains the most representative features through the fusion operation, ultimately providing input to the classification layer.

[0089] In this embodiment, the shallow features and weighted features of each sensor are concatenated along the channel dimension, integrating feature information at different levels, increasing the feature capacity of the model, and allowing it to capture more complex behavioral patterns. Further fusion operations transform the concatenated high-dimensional features into a more refined comprehensive feature vector, which not only preserves the diversity of multi-sensor data, but also undergoes appropriate processing and compression, making the comprehensive feature vector more representative and computationally efficient in the classification stage, thus ensuring the accuracy of traffic behavior recognition.

[0090] Furthermore, the steps described above for fusing the stitched features of each sensor to obtain the user's comprehensive feature vector may include: performing convolution processing on the stitched features of each sensor to obtain a fourth feature; performing pooling operation on the fourth feature to obtain a fifth feature; and inputting the fifth feature into a fully connected layer to obtain the user's comprehensive feature vector.

[0091] Specifically, the splicing features of each sensor are convolved to further integrate the information in the splicing features and extract a higher-level and more representative fourth feature.

[0092] The fourth feature typically has high dimensionality and requires further compression and simplification. Pooling operations (such as max pooling or average pooling) can further reduce the fourth feature to a more compact form, yielding the fifth feature. Pooling operations reduce the spatial dimensionality of the feature map, eliminate redundant information, and retain the most representative feature parts, effectively preventing overfitting and improving the computational efficiency of the model.

[0093] The fifth feature, after pooling, is flattened and input into the fully connected layer. The fully connected layer further weights and integrates the flattened fifth feature to generate the final comprehensive feature vector. This global integration of information from different sensors and feature dimensions ensures that the final comprehensive feature vector reflects the overall characteristics of all input data, providing high-quality input for subsequent classification layers.

[0094] In this embodiment, the spliced ​​features of each sensor are convolved to extract local correlations between features and generate a more representative high-level feature map, resulting in the fourth feature. The fourth feature is then compressed through pooling to generate a more compact fifth feature, reducing redundant information and improving computational efficiency. The pooled fifth feature is then input into a fully connected layer to further integrate information from all feature dimensions, generating a comprehensive feature vector. This comprehensive feature vector has strong expressive power and can accurately represent the user's traffic behavior characteristics, providing accurate data input for subsequent classification layers and thus improving the accuracy of traffic behavior recognition.

[0095] It should be emphasized that, to further ensure the privacy and security of the aforementioned sensor data, the sensor data can also be stored in a blockchain node.

[0096] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0097] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0098] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0099] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0100] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0101] Further reference Figure 3 As a response to the above Figure 2 The present application provides an embodiment of a traffic behavior recognition device to implement the method shown. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0102] like Figure 3As shown, the traffic behavior recognition device 300 described in this embodiment includes: a data acquisition module 301, a residual processing module 302, a temporal processing module 303, an attention processing module 304, a feature splicing module 305, and a behavior recognition module 306, wherein:

[0103] The data acquisition module 301 is used to acquire sensor data from each sensor in the user terminal.

[0104] The residual processing module 302 is used to input the sensor data of each sensor into the residual network to extract the features of the local receptive field, thereby obtaining the shallow features and local features of each sensor.

[0105] The temporal processing module 303 is used to input the shallow and local features of each sensor into the temporal convolutional neural network to extract features from the global receptive field and obtain the deep features of each sensor.

[0106] The attention processing module 304 is used to input the deep features of each sensor into the attention network to obtain the weighted features of each sensor.

[0107] The feature stitching module 305 is used to stitch together the shallow features and weighted features of each sensor to obtain the user's comprehensive feature vector.

[0108] The behavior recognition module 306 is used to input the comprehensive feature vector into the classification layer to obtain the user's traffic behavior recognition results.

[0109] In this embodiment, sensor data from each sensor in the user terminal is acquired and input into a residual network for feature extraction of the local receptive field, yielding shallow and local features of each sensor. This preserves the fine-grained information of the sensor data and provides a foundation for global feature extraction. The shallow and local features of each sensor are then input into a temporal convolutional neural network for feature extraction of the global receptive field, yielding deep features of each sensor. The temporal convolutional neural network enhances the capture of time-dependent features and, combined with shallow features, compensates for the lack of perception of local details. The deep features of each sensor are then input into an attention network. The attention network dynamically adjusts feature weights to ensure that the model focuses on the most representative channel features, reducing computational overhead and improving the model's computational efficiency, resulting in weighted features of each sensor. The shallow and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector, effectively reflecting the user's traffic behavior. Inputting the comprehensive feature vector into a classification layer allows for accurate identification of the user's traffic behavior.

[0110] In some optional implementations of this embodiment, the traffic behavior recognition device 300 may further include:

[0111] The integrity detection module is used to perform integrity detection on the sensor data of each sensor and obtain the missing values ​​of each sensor.

[0112] The missing value filling module is used to fill in the detected missing values ​​using Lagrange interpolation.

[0113] In some optional implementations of this embodiment, the time-domain processing module 303 may include:

[0114] The feature integration submodule is used to integrate the shallow and local features of each sensor to obtain integrated features.

[0115] The dilated convolution submodule is used to input the integrated features into the first dilated convolution layer in the temporal convolutional neural network for feature extraction, thereby obtaining the first feature.

[0116] The feature processing submodule is used to input the first feature into a nonlinear activation function to obtain the second feature, and to perform a normalization operation on the second feature to obtain the third feature.

[0117] The iterative convolution submodule is used to take the third feature as a new integrated feature and input it into the next dilated convolutional layer in the temporal convolutional neural network for iteration until the deep features of the sensor are generated based on the output of the last dilated convolutional layer.

[0118] In some optional implementations of this embodiment, the attention processing module 304 may include:

[0119] The information calculation submodule is used to calculate the channel information of each channel based on the feature map of each channel in the deep features of each sensor.

[0120] The weight calculation submodule is used to calculate the channel information of each channel through the attention mechanism to obtain the attention weight of each channel.

[0121] The weighting submodule is used to add the attention weights of each channel to the feature maps of each channel to obtain weighted features.

[0122] In some optional implementations of this embodiment, the information calculation submodule may include:

[0123] The pooling processing unit is used to perform global average pooling or global max pooling on the feature maps of each channel in the deep features to obtain the pooling values ​​of each channel.

[0124] The numerical determination unit is used to use the pooling values ​​of each channel as the channel information of each channel.

[0125] In some optional implementations of this embodiment, the feature splicing module 305 may include:

[0126] The feature stitching submodule is used to stitch together the shallow and weighted features of each sensor along the channel dimension to obtain the stitched features of each sensor.

[0127] The feature fusion submodule is used to fuse the stitched features from various sensors to obtain the user's comprehensive feature vector.

[0128] In some optional implementations of this embodiment, the feature fusion submodule may include:

[0129] The convolution processing unit is used to perform convolution processing on the stitched features of each sensor to obtain the fourth feature.

[0130] The pooling operation unit is used to perform pooling operations on the fourth feature to obtain the fifth feature.

[0131] The synthesis generation unit is used to input the fifth feature into the fully connected layer to obtain the user's comprehensive feature vector.

[0132] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0133] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only a computer device 4 with a memory 41, a processor 42, and a network interface 43 is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0134] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0135] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for traffic behavior recognition methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0136] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the traffic behavior recognition method.

[0137] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0138] The computer device provided in this embodiment can execute the traffic behavior recognition method described above. The traffic behavior recognition method here can be any of the traffic behavior recognition methods described in the various embodiments above.

[0139] In this embodiment, sensor data from each sensor in the user terminal is acquired and input into a residual network for feature extraction of the local receptive field, yielding shallow and local features of each sensor. This preserves the fine-grained information of the sensor data and provides a foundation for global feature extraction. The shallow and local features of each sensor are then input into a temporal convolutional neural network for feature extraction of the global receptive field, yielding deep features of each sensor. The temporal convolutional neural network enhances the capture of time-dependent features and, combined with shallow features, compensates for the lack of perception of local details. The deep features of each sensor are then input into an attention network. The attention network dynamically adjusts feature weights to ensure that the model focuses on the most representative channel features, reducing computational overhead and improving the model's computational efficiency, resulting in weighted features of each sensor. The shallow and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector, effectively reflecting the user's traffic behavior. Inputting the comprehensive feature vector into a classification layer allows for accurate identification of the user's traffic behavior.

[0140] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the traffic behavior recognition method described above.

[0141] In this embodiment, sensor data from each sensor in the user terminal is acquired and input into a residual network for feature extraction of the local receptive field, yielding shallow and local features of each sensor. This preserves the fine-grained information of the sensor data and provides a foundation for global feature extraction. The shallow and local features of each sensor are then input into a temporal convolutional neural network for feature extraction of the global receptive field, yielding deep features of each sensor. The temporal convolutional neural network enhances the capture of time-dependent features and, combined with shallow features, compensates for the lack of perception of local details. The deep features of each sensor are then input into an attention network. The attention network dynamically adjusts feature weights to ensure that the model focuses on the most representative channel features, reducing computational overhead and improving the model's computational efficiency, resulting in weighted features of each sensor. The shallow and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector, effectively reflecting the user's traffic behavior. Inputting the comprehensive feature vector into a classification layer allows for accurate identification of the user's traffic behavior.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0143] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A traffic behavior recognition method, characterized in that, Includes the following steps: Acquire sensor data from each sensor in the user terminal; The sensor data of each sensor is input into the residual network to extract features of the local receptive field, thereby obtaining the shallow features and local features of each sensor. The shallow features are low-level features extracted by the early layers of the residual network, and the local features are high-level features extracted by the later layers of the network. The shallow and local features of each sensor are input into a temporal convolutional neural network to extract features from the global receptive field, thereby obtaining the deep features of each sensor. The deep features of each sensor are input into the attention network to obtain the weighted features of each sensor; The shallow features and weighted features of each sensor are concatenated to obtain the user's comprehensive feature vector. Specifically, this includes: concatenating the shallow features and weighted features of each sensor along the channel dimension to obtain the concatenated features of each sensor; and performing a fusion operation on the concatenated features of each sensor to obtain the user's comprehensive feature vector. The comprehensive feature vector is input into the classification layer to obtain the user's traffic behavior recognition result.

2. The traffic behavior recognition method according to claim 1, characterized in that, Following the step of acquiring sensor data from each sensor in the user terminal, the method further includes: Integrity checks are performed on the sensor data of each sensor to obtain the missing values ​​of each sensor; Data is filled with the detected missing values ​​using Lagrange interpolation.

3. The traffic behavior recognition method according to claim 1, characterized in that, The step of inputting the shallow and local features of each sensor into a temporal convolutional neural network to extract features from the global receptive field and obtain the deep features of each sensor includes: For each sensor, the shallow features and local features of the sensor are integrated to obtain integrated features; The integrated features are input into the first dilated convolutional layer in the temporal convolutional neural network for feature extraction to obtain the first feature. The first feature is input into a nonlinear activation function to obtain the second feature, and the second feature is normalized to obtain the third feature; The third feature is used as a new integrated feature and input into the next dilated convolutional layer in the temporal convolutional neural network for iteration until the deep features of the sensor are generated based on the output of the last dilated convolutional layer.

4. The traffic behavior recognition method according to claim 1, characterized in that, The step of inputting the deep features of each sensor into the attention network to obtain the weighted features of each sensor includes: For each sensor's deep features, the channel information of each channel is calculated based on the feature map of each channel in the deep features; The attention weights of each channel are obtained by calculating the channel information of each channel through an attention mechanism. The attention weights of each channel are added to the feature maps of each channel to obtain weighted features.

5. The traffic behavior recognition method according to claim 4, characterized in that, The step of calculating the channel information of each channel based on the feature map of each channel in the deep features includes: Global average pooling or global max pooling is performed on the feature maps of each channel in the deep features to obtain the pooling values ​​of each channel; The pooling values ​​of each channel are used as the channel information of each channel.

6. The traffic behavior recognition method according to claim 1, characterized in that, The step of fusing the stitched features of the various sensors to obtain the user's comprehensive feature vector includes: The splicing features of each sensor are convolved to obtain the fourth feature; The fourth feature is pooled to obtain the fifth feature; The fifth feature is input into the fully connected layer to obtain the user's comprehensive feature vector.

7. A traffic behavior recognition device, characterized in that, include: The data acquisition module is used to acquire sensor data from various sensors in the user terminal. The residual processing module is used to input the sensor data of each sensor into the residual network to extract features of the local receptive field, thereby obtaining the shallow features and local features of each sensor. The shallow features are low-level features extracted by the early layers of the residual network, and the local features are high-level features extracted by the later layers of the network. The temporal processing module is used to input the shallow and local features of each sensor into a temporal convolutional neural network to extract features from the global receptive field, thereby obtaining the deep features of each sensor. An attention processing module is used to input the deep features of each sensor into an attention network to obtain the weighted features of each sensor; The feature stitching module is used to stitch together the shallow features and weighted features of each sensor to obtain the user's comprehensive feature vector. Specifically, it includes: stitching together the shallow features and weighted features of each sensor along the channel dimension to obtain the stitched features of each sensor; and performing a fusion operation on the stitched features of each sensor to obtain the user's comprehensive feature vector. The behavior recognition module is used to input the comprehensive feature vector into the classification layer to obtain the user's traffic behavior recognition result.

8. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the traffic behavior recognition method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the traffic behavior recognition method as described in any one of claims 1 to 6.

Images