Traffic behavior recognition method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of artificial intelligence, and relates to a traffic behavior recognition method, which comprises the following steps: acquiring sensor data of each sensor in a user terminal; inputting the sensor data of each sensor into a residual network to perform feature extraction of a local receptive field to obtain shallow features and local features of each sensor; inputting the shallow features and the local features of each sensor into a time domain convolutional neural network to perform feature extraction of a global receptive field to obtain deep features of each sensor; inputting the deep features of each sensor into an attention network to obtain weighted features of each sensor; splicing the shallow features and the weighted features of the sensors to obtain a comprehensive feature vector of the user; and inputting the comprehensive feature vector into a classification layer to obtain a traffic behavior recognition result of the user. The invention further provides a traffic behavior recognition device, computer equipment and a storage medium. According to the invention, the accuracy and efficiency of traffic behavior recognition are improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a traffic behavior recognition method, device, computer equipment and storage medium. Background Art

[0002] With the popularization of smart devices, traffic behavior recognition through sensor data has become an important research direction in user behavior recognition. It aims to judge the user's traffic mode (such as walking, cycling, driving, etc.) by analyzing the data of sensors (such as accelerometers, gyroscopes, etc.) in user devices. It can be widely used in smart transportation, health monitoring, and auto insurance claims.

[0003] Current traffic behavior recognition technology mainly relies on deep learning models, such as convolutional neural networks (CNN) and recurrent neural networks (RNN). However, when processing time series data, convolutional neural networks often cannot fully utilize time series information, which affects the accuracy of recognition. Secondly, although recurrent neural networks can capture time dependencies, their application efficiency is low due to the complexity of the model and the large consumption of computing resources. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a traffic behavior recognition method, apparatus, computer equipment and storage medium to improve the accuracy and efficiency of traffic behavior recognition.

[0005] In order to solve the above technical problems, the present application embodiment provides a traffic behavior recognition method, which adopts the following technical solutions:

[0006] Obtain sensor data of each sensor in the user terminal;

[0007] Inputting the sensor data of each sensor into the residual network to extract the features of the local receptive field, so as to obtain the shallow features and local features of each sensor;

[0008] Inputting shallow features and local features of each sensor into a time-domain convolutional neural network to extract features of the global receptive field, thereby obtaining deep features of each sensor;

[0009] Inputting the deep features of each sensor into the attention network to obtain the weighted features of each sensor;

[0010] The shallow features and weighted features of each sensor are combined to obtain a comprehensive feature vector of the user;

[0011] The comprehensive feature vector is input into the classification layer to obtain the traffic behavior recognition result of the user.

[0012] In order to solve the above technical problems, the embodiment of the present application further provides a traffic behavior recognition device, which adopts the following technical solution:

[0013] A data acquisition module, used to acquire sensor data of each sensor in the user terminal;

[0014] A residual processing module, used for inputting the sensor data of each sensor into a residual network to extract features of a local receptive field, and obtaining shallow features and local features of each sensor;

[0015] A time domain processing module, used for inputting shallow features and local features of each sensor into a time domain convolutional neural network to extract features of the global receptive field, so as to obtain deep features of each sensor;

[0016] An attention processing module, used for inputting the deep features of each sensor into an attention network to obtain weighted features of each sensor;

[0017] A feature splicing module, used to splice the shallow features and weighted features of each sensor to obtain a comprehensive feature vector of the user;

[0018] The behavior recognition module is used to input the comprehensive feature vector into the classification layer to obtain the traffic behavior recognition result of the user.

[0019] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0020] Obtain sensor data of each sensor in the user terminal;

[0021] Inputting the sensor data of each sensor into the residual network to extract the features of the local receptive field, so as to obtain the shallow features and local features of each sensor;

[0022] Inputting shallow features and local features of each sensor into a time-domain convolutional neural network to extract features of the global receptive field, thereby obtaining deep features of each sensor;

[0023] Inputting the deep features of each sensor into the attention network to obtain the weighted features of each sensor;

[0024] The shallow features and weighted features of each sensor are combined to obtain a comprehensive feature vector of the user;

[0025] The comprehensive feature vector is input into the classification layer to obtain the traffic behavior recognition result of the user.

[0026] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0027] Obtain sensor data of each sensor in the user terminal;

[0028] Inputting the sensor data of each sensor into the residual network to extract the features of the local receptive field, so as to obtain the shallow features and local features of each sensor;

[0029] Inputting shallow features and local features of each sensor into a time-domain convolutional neural network to extract features of the global receptive field, thereby obtaining deep features of each sensor;

[0030] Inputting the deep features of each sensor into the attention network to obtain the weighted features of each sensor;

[0031] The shallow features and weighted features of each sensor are combined to obtain a comprehensive feature vector of the user;

[0032] The comprehensive feature vector is input into the classification layer to obtain the traffic behavior recognition result of the user.

[0033] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: the sensor data of each sensor in the user terminal is obtained, and the data is input into the residual network for feature extraction of the local receptive field, so as to obtain the shallow features and local features of each sensor, which not only retains the fine-grained information of the sensor data, but also provides a basis for the extraction of global features; the shallow features and local features of each sensor are input into the time domain convolutional neural network for feature extraction of the global receptive field, so as to obtain the deep features of each sensor, and the time domain convolutional neural network strengthens the capture of time-dependent features, and at the same time, combines the shallow features to make up for the lack of perception of local details; the deep features of each sensor are input into the attention network, and the attention network ensures that the model focuses on the most representative channel features by dynamically adjusting the feature weights, thereby reducing the computational overhead and improving the computational efficiency of the model, and obtaining the weighted features of each sensor; the shallow features and weighted features of each sensor are spliced ​​to obtain the comprehensive feature vector of the user, which effectively reflects the user's traffic behavior, and the comprehensive feature vector is input into the classification layer, so as to accurately identify the user's traffic behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0036] Figure 2 is a flow chart of an embodiment of a traffic behavior recognition method according to the present application;

[0037] Figure 3 is a schematic structural diagram of an embodiment of a traffic behavior recognition device according to the present application;

[0038] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0041] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0042] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0043] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0044] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer and a desktop computer, etc.

[0045] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0046] It should be noted that the traffic behavior recognition method provided in the embodiment of the present application is generally executed by a server, and accordingly, the traffic behavior recognition device is generally set in the server.

[0047] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0048] Continue to refer Figure 2 , shows a flow chart of an embodiment of a traffic behavior recognition method according to the present application. The traffic behavior recognition method comprises the following steps:

[0049] Step S201: acquiring sensor data of each sensor in the user terminal.

[0050] In this embodiment, the traffic behavior recognition method is executed on the electronic device (eg Figure 1 The server shown in the figure) can communicate with the terminal device through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (Ultra Wi-Fi Band) connection, and other wireless connection methods currently known or to be developed in the future.

[0051] Specifically, sensor data is obtained from multiple sensors in a user terminal (such as a smart phone, a wearable device, etc.). The sensors may include accelerometers, gyroscopes, geomagnetic sensors, etc., which can capture the user's motion state and environmental information.

[0052] Step S202: input the sensor data of each sensor into the residual network to extract the features of the local receptive field, and obtain the shallow features and local features of each sensor.

[0053] Specifically, the sensor data of each sensor is input into the residual network (ResNet), which can effectively extract local receptive field features, that is, retain local detail information in the sensor data. In this process, shallow features are low-level features extracted by the early layers of the residual network, and local features are high-level features extracted by the later layers of the network.

[0054] Step S203, inputting shallow features and local features of each sensor into a time-domain convolutional neural network to extract features of the global receptive field, thereby obtaining deep features of each sensor.

[0055] Specifically, the shallow features and local features of each sensor are input into the time-domain convolutional neural network (TCN). The time-domain convolutional neural network uses dilated convolution to extract features of the global receptive field, which can capture long-term dependent sequence information. Among them, the shallow features contain low-level information of the sensor data. Shallow features help enhance the network's perception of key temporal features in the time-domain convolutional neural network, especially in shallower temporal relationships. The local features are the final output of the residual network, which contain local complex patterns and high-level information after multi-layer convolution processing. After the local features are input into the time-domain convolutional neural network, it helps to extract and strengthen the fine-grained features in the global receptive field.

[0056] The time-domain convolutional neural network uses dilated convolution to extract features of the global receptive field and capture sequence information over a long time span. Shallow features help the network enhance its perception of important features by retaining key temporal information during the convolution process; local features provide more advanced local information, which supplements the limitations of dilated convolution in processing fine-grained information.

[0057] The time-domain convolutional neural network finally outputs deep features, which reflect the time series patterns of user behaviors.

[0058] Step S204: input the deep features of each sensor into the attention network to obtain the weighted features of each sensor.

[0059] Specifically, the deep features of each sensor are input into the attention network. The deep features of each sensor have multiple channels. The attention network calculates the relationship between the channels through the attention mechanism and dynamically adjusts the weights of each channel, so that the model can pay more attention to the feature channels that contribute most to traffic behavior recognition, and obtain the weighted features of each sensor.

[0060] Step S205, concatenating the shallow features and weighted features of each sensor to obtain a comprehensive feature vector of the user.

[0061] Specifically, for each sensor, the weighted features of the sensor are concatenated with the shallow features obtained through the residual network, usually in the channel dimension, to integrate the multi-level information of the sensor to form a high-dimensional comprehensive feature vector, which fully utilizes the complementarity of the features of each layer.

[0062] Step S206, input the comprehensive feature vector into the classification layer to obtain the user's traffic behavior recognition result.

[0063] Specifically, the comprehensive feature vector is input into the classification layer (such as the fully connected layer). The classification layer outputs the final traffic behavior recognition result through analysis and learning of the comprehensive feature vector to determine the user's current traffic mode, such as walking, cycling, driving, etc.

[0064] It should be noted that the present application uses a traffic behavior recognition model to identify the user's traffic behavior. The traffic behavior recognition model includes the residual network, time domain convolutional neural network, attention network and classification layer mentioned above.

[0065] In this embodiment, the sensor data of each sensor in the user terminal is obtained and input into the residual network for feature extraction of the local receptive field to obtain the shallow features and local features of each sensor, which not only retains the fine-grained information of the sensor data, but also provides a basis for the extraction of global features; the shallow features and local features of each sensor are input into the time domain convolutional neural network for feature extraction of the global receptive field to obtain the deep features of each sensor. The time domain convolutional neural network strengthens the capture of time-dependent features and combines the shallow features to make up for the lack of perception of local details; the deep features of each sensor are input into the attention network. The attention network ensures that the model focuses on the most representative channel features by dynamically adjusting the feature weights, reduces the computational overhead and improves the computational efficiency of the model, and obtains the weighted features of each sensor; the shallow features and weighted features of each sensor are spliced ​​to obtain the user's comprehensive feature vector, which effectively reflects the user's traffic behavior. The comprehensive feature vector is input into the classification layer to accurately identify the user's traffic behavior.

[0066] Furthermore, after the above step S201, the method may further include: performing integrity detection on sensor data of each sensor to obtain missing values ​​of each sensor; and performing data filling on the detected missing values ​​by Lagrange interpolation.

[0067] Specifically, after acquiring the sensor data of each sensor, a data integrity check is performed to check whether the sensor data has missing or abnormal conditions, such as data missing due to signal loss, equipment failure, or external environmental influences. Through the detection, the missing values ​​in the sensor data are identified.

[0068] The detected missing values ​​are filled by Lagrangian interpolation. Lagrangian interpolation is a commonly used mathematical method for smooth interpolation between known data points to complete missing data. Sensor data has temporal continuity. Through Lagrangian interpolation, the missing part of the data can be restored without significantly affecting the authenticity of the data, ensuring the integrity of the subsequent feature extraction process and the stability of the model.

[0069] After completing the missing value filling, the sensor data of each sensor is input into the residual network to perform feature extraction of the local receptive field.

[0070] In this embodiment, by detecting missing values ​​in the sensor data and filling the missing values ​​through Lagrange interpolation, the continuity and integrity of the sensor data are restored, the errors caused by incomplete data are reduced, and it is ensured that the residual network and the time domain convolutional neural network can obtain more reliable input data in the feature extraction stage, thereby improving the accuracy of subsequent feature extraction.

[0071] Furthermore, the above step S203 may include: for each sensor, integrating the shallow features and local features of the sensor to obtain integrated features; inputting the integrated features into the first dilated convolution layer in the time domain convolutional neural network for feature extraction to obtain a first feature; inputting the first feature into a nonlinear activation function to obtain a second feature, and normalizing the second feature to obtain a third feature; using the third feature as a new integrated feature and inputting it into the next dilated convolution layer in the time domain convolutional neural network for iteration, until the deep features of the sensor are generated according to the output of the last dilated convolution layer.

[0072] Specifically, in a time-domain convolutional neural network (TCN), the shallow features and local features of each sensor are integrated, for example, by splicing or weighted combination, so that the shallow features are complementary to the local features.

[0073] The temporal convolutional network has multiple layers of dilated convolutional layers, and the integrated features are first input into the first dilated convolutional layer for dilated convolution. By expanding the receptive field, the dilated convolution can capture dependencies over a longer time span, extract the first feature, which contains important temporal information in the integrated features, and preliminarily construct the global features of the sensor data.

[0074] The first feature is processed by a nonlinear activation function (such as ReLU) to generate the second feature. The nonlinear activation function introduces nonlinear factors and enhances the expressiveness of the model. Subsequently, the second feature is adjusted to a uniform numerical distribution through a normalization operation (such as Batch Normalization) to obtain the third feature. The normalization process helps to improve the training stability and convergence speed of the model.

[0075] The third feature is used as a new integrated feature and is input into the next dilated convolution layer for iterative processing. Through this iterative process, the model extracts features layer by layer, and each layer further expands the global receptive field based on the output of the previous layer. Finally, after multiple layers of dilated convolution processing, the deep features of each sensor are generated. The deep features integrate the local and global information of the sensor data and can more comprehensively describe the user's traffic behavior.

[0076] In this embodiment, for each sensor, the shallow features and local features of the sensor are integrated to obtain integrated features; the multi-layer dilated convolutional layers in the time-domain convolutional neural network iterate the integrated features, which can expand the receptive field, capture long-term dependent features, and gradually refine and enhance the expression ability of these features; the nonlinear activation and normalization operations optimize the nonlinear representation and data distribution of the features, making the model more efficient and stable in subsequent layers of convolution processing and feature extraction; the deep features finally generated not only contain the global temporal relationship of the sensor data, but also retain the local fine-grained information, and the rich information content ensures the accuracy of subsequent traffic behavior recognition.

[0077] Furthermore, the above step S204 may include: for the deep features of each sensor, according to the feature map of each channel in the deep features, calculating the channel information of each channel; calculating the channel information of each channel through the attention mechanism to obtain the attention weight of each channel; adding the attention weight of each channel to the feature map of each channel to obtain the weighted feature.

[0078] Specifically, the deep features of each sensor are usually composed of multiple channels, each of which represents the network's response to a certain aspect or type of feature of the input data. The feature map in each channel is analyzed to extract the channel information of the channel. The channel information can be a numerical value, in which way the information of each feature map is compressed into a single channel information value.

[0079] The channel information is input into the attention network. The attention network has an attention mechanism based on the relationship between feature channels, which can calculate the importance of each channel and assign corresponding weights based on the channel information. This process can be achieved through a fully connected layer or more complex operations (such as nonlinear transformations). The generated attention weights reflect the relative importance of each channel in the current recognition task.

[0080] The generated attention weights are applied to the feature maps in the corresponding channels, that is, all values ​​of each feature map are multiplied by the attention weight of the channel. The weighting process allows the model to pay more attention to the feature channels that contribute more to traffic behavior recognition, while suppressing unimportant channels, and using the weighted feature maps as weighted features. The weighted features contain optimized deep features, providing more accurate input for subsequent processing.

[0081] In this embodiment, for the deep features of each sensor, the channel information of each channel is calculated according to the feature map of each channel in the deep features to obtain the global feature representation of each channel; based on the channel information, the attention weight of each channel is dynamically adjusted through the attention mechanism to automatically identify and highlight the feature channels that are important for the traffic behavior recognition task, and unnecessary or redundant features will be suppressed, reducing the redundancy of calculation, so that the model can show higher accuracy and computational efficiency when processing complex high-dimensional features; the attention weight of each channel is added to the feature map of each channel to obtain the weighted feature, which effectively integrates the deep information of the sensor data and ensures the accuracy of the subsequent processing of the model.

[0082] Furthermore, the step of calculating the channel information of each channel based on the feature map of each channel in the deep features may include: performing global average pooling or global maximum pooling on the feature map of each channel in the deep features to obtain the pooling value of each channel; and using the pooling value of each channel as the channel information of each channel.

[0083] Specifically, the feature map of each channel in the deep features contains the response values ​​of the channel at different spatial positions or time points. In order to simplify these high-dimensional feature maps and extract representative global information, a global pooling operation can be used to average all the values ​​of the feature map to generate a single value representing the average response of the channel; or a global maximum pooling operation can be used to extract the maximum value in the feature map to represent the strongest response of the channel. These two pooling methods can effectively compress the information of the feature map, allowing the model to obtain the global features of the channel with lower computational complexity.

[0084] The pooled values ​​generated by global pooling are regarded as channel information of each channel, which represents the global response of each channel in the deep feature on the entire input data. Through the pooling operation, the model can measure the relative importance of each channel, so as to more accurately allocate weights in the attention mechanism.

[0085] In this embodiment, the high-dimensional feature map of each channel is compressed into a single pooled value through a global pooling operation, which simplifies the complex deep feature expression. The pooled value can reflect the global response of each channel, so that the attention mechanism can assign more reasonable weights based on channel information, highlighting the importance of key feature channels; the global pooling operation is stable and robust when extracting the global information of the feature map, reduces the deviation caused by local anomalies or noise, and provides data support for the subsequent processing of the attention mechanism.

[0086] Furthermore, the above step S205 may include: performing feature stitching on the shallow features and weighted features of each sensor in the channel dimension to obtain the stitching features of each sensor; and performing a fusion operation on the stitching features of each sensor to obtain a comprehensive feature vector of the user.

[0087] Specifically, the shallow features from each sensor and the weighted features processed by the attention mechanism are spliced ​​in the channel dimension. The shallow features capture the preliminary temporal information, and the weighted features are adjusted by the attention mechanism to highlight the key features for traffic behavior recognition. These features are spliced ​​in the channel dimension to form a richer feature representation and obtain the spliced ​​features. The splicing operation increases the diversity and information content of the features, ensuring that the model can utilize feature information at different levels in subsequent processing.

[0088] The spliced ​​features contain multi-dimensional information from each sensor. In order to further integrate this information and make it more efficient and useful in the classification stage, the spliced ​​features are fused to transform the high-dimensional spliced ​​features into a more compact and representative comprehensive feature vector. The comprehensive feature vector integrates the information from different sensors and retains the most representative features through the fusion operation, which ultimately provides input for the classification layer.

[0089] In this embodiment, the shallow features and weighted features of each sensor are spliced ​​in the channel dimension, which integrates feature information at different levels, increases the feature capacity of the model, and allows it to capture more complex behavior patterns; further fusion operations convert the spliced ​​high-dimensional features into more refined comprehensive feature vectors, which not only retain the diversity of multi-sensor data, but also, after proper processing and compression, make the comprehensive feature vectors more representative and computationally efficient in the classification stage, thereby ensuring the accuracy of traffic behavior recognition.

[0090] Furthermore, the above-mentioned step of fusing the spliced ​​features of each sensor to obtain a comprehensive feature vector of the user may include: performing convolution processing on the spliced ​​features of each sensor to obtain a fourth feature; performing pooling operation on the fourth feature to obtain a fifth feature; and inputting the fifth feature into a fully connected layer to obtain a comprehensive feature vector of the user.

[0091] Specifically, the concatenated features of each sensor are convolved to further integrate the information in the concatenated features and extract a higher-level and more representative fourth feature.

[0092] The fourth feature usually has a higher dimension and needs to be further compressed and simplified. Through pooling operations (such as maximum pooling or average pooling), the fourth feature is further reduced to a more compact form to obtain the fifth feature. Pooling operations can reduce the spatial dimension of feature maps, eliminate redundant information, and retain the most representative feature parts, which can effectively prevent overfitting and improve the computational efficiency of the model.

[0093] The fifth feature after pooling is flattened and input to the fully connected layer. The fully connected layer further weights and integrates the flattened fifth feature to generate the final comprehensive feature vector. The fully connected layer globally integrates the information of different sensors and feature dimensions, so that the final comprehensive feature vector can reflect the overall characteristics of all input data and provide high-quality input for the subsequent classification layer.

[0094] In this embodiment, convolution processing is performed on the spliced ​​features of each sensor to extract the local correlation between the features and generate a more representative high-level feature map to obtain the fourth feature; the fourth feature is compressed through a pooling operation to generate a more compact fifth feature, thereby reducing redundant information and improving computational efficiency; the pooled fifth feature is input into the fully connected layer to further integrate the information of all feature dimensions to generate a comprehensive feature vector. The comprehensive feature vector has a strong expressive ability and can accurately represent the user's traffic behavior characteristics, providing accurate data input for the subsequent classification layer, thereby improving the accuracy of traffic behavior recognition.

[0095] It should be emphasized that in order to further ensure the privacy and security of the above sensor data, the above sensor data can also be stored in a blockchain node.

[0096] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0097] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0098] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0099] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0100] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0101] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a traffic behavior recognition device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0102] like Figure 3As shown, the traffic behavior recognition device 300 described in this embodiment includes: a data acquisition module 301, a residual processing module 302, a time domain processing module 303, an attention processing module 304, a feature splicing module 305 and a behavior recognition module 306, wherein:

[0103] The data acquisition module 301 is used to acquire sensor data of each sensor in the user terminal.

[0104] The residual processing module 302 is used to input the sensor data of each sensor into the residual network to extract the features of the local receptive field, so as to obtain the shallow features and local features of each sensor.

[0105] The time domain processing module 303 is used to input the shallow features and local features of each sensor into the time domain convolutional neural network to extract the features of the global receptive field, so as to obtain the deep features of each sensor.

[0106] The attention processing module 304 is used to input the deep features of each sensor into the attention network to obtain the weighted features of each sensor.

[0107] The feature concatenation module 305 is used to concatenate the shallow features and weighted features of each sensor to obtain a comprehensive feature vector of the user.

[0108] The behavior recognition module 306 is used to input the comprehensive feature vector into the classification layer to obtain the user's traffic behavior recognition result.

[0109] In this embodiment, the sensor data of each sensor in the user terminal is obtained and input into the residual network for feature extraction of the local receptive field to obtain the shallow features and local features of each sensor, which not only retains the fine-grained information of the sensor data, but also provides a basis for the extraction of global features; the shallow features and local features of each sensor are input into the time domain convolutional neural network for feature extraction of the global receptive field to obtain the deep features of each sensor. The time domain convolutional neural network strengthens the capture of time-dependent features and combines the shallow features to make up for the lack of perception of local details; the deep features of each sensor are input into the attention network. The attention network ensures that the model focuses on the most representative channel features by dynamically adjusting the feature weights, reduces the computational overhead and improves the computational efficiency of the model, and obtains the weighted features of each sensor; the shallow features and weighted features of each sensor are spliced ​​to obtain the user's comprehensive feature vector, which effectively reflects the user's traffic behavior. The comprehensive feature vector is input into the classification layer to accurately identify the user's traffic behavior.

[0110] In some optional implementations of this embodiment, the traffic behavior recognition device 300 may further include:

[0111] The integrity detection module is used to perform integrity detection on the sensor data of each sensor to obtain the missing value of each sensor.

[0112] The missing value filling module is used to fill the detected missing values ​​through Lagrange interpolation.

[0113] In some optional implementations of this embodiment, the time domain processing module 303 may include:

[0114] The feature integration submodule is used to integrate the shallow features and local features of each sensor to obtain integrated features.

[0115] The dilated convolution submodule is used to input the integrated features into the first dilated convolution layer in the time domain convolutional neural network for feature extraction to obtain the first feature.

[0116] The feature processing submodule is used to input the first feature into a nonlinear activation function to obtain a second feature, and perform a normalization operation on the second feature to obtain a third feature.

[0117] The iterative convolution submodule is used to take the third feature as a new integrated feature and input it into the next dilated convolution layer in the time domain convolutional neural network for iteration until the deep feature of the sensor is generated according to the output of the last dilated convolution layer.

[0118] In some optional implementations of this embodiment, the attention processing module 304 may include:

[0119] The information calculation submodule is used to calculate the channel information of each channel according to the feature map of each channel in the deep feature for the deep feature of each sensor.

[0120] The weight calculation submodule is used to calculate the channel information of each channel through the attention mechanism to obtain the attention weight of each channel.

[0121] The weight adding submodule is used to add the attention weight of each channel to the feature map of each channel to obtain the weighted feature.

[0122] In some optional implementations of this embodiment, the information calculation submodule may include:

[0123] The pooling processing unit is used to perform global average pooling or global maximum pooling on the feature maps of each channel in the deep features to obtain the pooling value of each channel.

[0124] The value determination unit is used to use the pooled value of each channel as the channel information of each channel.

[0125] In some optional implementations of this embodiment, the feature splicing module 305 may include:

[0126] The feature splicing submodule is used to perform feature splicing on the shallow features and weighted features of each sensor in the channel dimension to obtain the splicing features of each sensor.

[0127] The feature fusion submodule is used to fuse the spliced ​​features of each sensor to obtain the user's comprehensive feature vector.

[0128] In some optional implementations of this embodiment, the feature fusion submodule may include:

[0129] The convolution processing unit is used to perform convolution processing on the spliced ​​features of each sensor to obtain a fourth feature.

[0130] The pooling operation unit is used to perform a pooling operation on the fourth feature to obtain a fifth feature.

[0131] The comprehensive generation unit is used to input the fifth feature into the fully connected layer to obtain a comprehensive feature vector of the user.

[0132] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0133] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 having a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0134] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0135] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the traffic behavior recognition method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0136] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the traffic behavior recognition method.

[0137] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0138] The computer device provided in this embodiment can execute the above-mentioned traffic behavior recognition method. The traffic behavior recognition method here can be the traffic behavior recognition method of each of the above-mentioned embodiments.

[0139] In this embodiment, the sensor data of each sensor in the user terminal is obtained and input into the residual network for feature extraction of the local receptive field to obtain the shallow features and local features of each sensor, which not only retains the fine-grained information of the sensor data, but also provides a basis for the extraction of global features; the shallow features and local features of each sensor are input into the time domain convolutional neural network for feature extraction of the global receptive field to obtain the deep features of each sensor. The time domain convolutional neural network strengthens the capture of time-dependent features and combines the shallow features to make up for the lack of perception of local details; the deep features of each sensor are input into the attention network. The attention network ensures that the model focuses on the most representative channel features by dynamically adjusting the feature weights, reduces the computational overhead and improves the computational efficiency of the model, and obtains the weighted features of each sensor; the shallow features and weighted features of each sensor are spliced ​​to obtain the user's comprehensive feature vector, which effectively reflects the user's traffic behavior. The comprehensive feature vector is input into the classification layer to accurately identify the user's traffic behavior.

[0140] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the traffic behavior recognition method as described above.

[0141] In this embodiment, the sensor data of each sensor in the user terminal is obtained and input into the residual network for feature extraction of the local receptive field to obtain the shallow features and local features of each sensor, which not only retains the fine-grained information of the sensor data, but also provides a basis for the extraction of global features; the shallow features and local features of each sensor are input into the time domain convolutional neural network for feature extraction of the global receptive field to obtain the deep features of each sensor. The time domain convolutional neural network strengthens the capture of time-dependent features and combines the shallow features to make up for the lack of perception of local details; the deep features of each sensor are input into the attention network. The attention network ensures that the model focuses on the most representative channel features by dynamically adjusting the feature weights, reduces the computational overhead and improves the computational efficiency of the model, and obtains the weighted features of each sensor; the shallow features and weighted features of each sensor are spliced ​​to obtain the user's comprehensive feature vector, which effectively reflects the user's traffic behavior. The comprehensive feature vector is input into the classification layer to accurately identify the user's traffic behavior.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0143] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A traffic behavior recognition method, characterized in that: The steps include: Obtain sensor data of each sensor in the user terminal; Inputting the sensor data of each sensor into the residual network to extract the features of the local receptive field, so as to obtain the shallow features and local features of each sensor; Inputting shallow features and local features of each sensor into a time-domain convolutional neural network to extract features of the global receptive field, thereby obtaining deep features of each sensor; Inputting the deep features of each sensor into the attention network to obtain the weighted features of each sensor; The shallow features and weighted features of each sensor are combined to obtain a comprehensive feature vector of the user; The comprehensive feature vector is input into the classification layer to obtain the traffic behavior recognition result of the user.

2. The traffic behavior recognition method according to claim 1, characterized in that: After the step of obtaining sensor data of each sensor in the user terminal, the method further includes: Performing integrity detection on sensor data of each sensor to obtain missing values ​​of each sensor; The detected missing values ​​are filled in with data through Lagrange interpolation.

3. The traffic behavior recognition method according to claim 1, characterized in that: The step of inputting shallow features and local features of each sensor into a time domain convolutional neural network to extract features of a global receptive field to obtain deep features of each sensor includes: For each sensor, the shallow features and local features of the sensor are integrated to obtain an integrated feature; Inputting the integrated feature into the first dilated convolution layer in the time-domain convolutional neural network for feature extraction to obtain a first feature; Inputting the first feature into a nonlinear activation function to obtain a second feature, and performing a normalization operation on the second feature to obtain a third feature; The third feature is used as a new integrated feature and input into the next dilated convolution layer in the time-domain convolutional neural network for iteration until the deep feature of the sensor is generated according to the output of the last dilated convolution layer.

4. The traffic behavior recognition method according to claim 1, characterized in that: The step of inputting the deep features of each sensor into the attention network to obtain the weighted features of each sensor includes: For each deep feature of the sensor, calculating the channel information of each channel according to the feature map of each channel in the deep feature; Calculating the channel information of each channel through an attention mechanism to obtain an attention weight of each channel; The attention weights of the channels are added to the feature maps of the channels to obtain weighted features.

5. The traffic behavior recognition method according to claim 4, characterized in that: The step of calculating the channel information of each channel according to the feature graph of each channel in the deep feature comprises: Performing global average pooling or global maximum pooling on the feature map of each channel in the deep feature to obtain the pooling value of each channel; The pooled values ​​of the channels are used as the channel information of the channels.

6. The traffic behavior recognition method according to claim 1, characterized in that: The step of combining the shallow features and weighted features of each sensor to obtain a comprehensive feature vector of the user comprises: Performing feature splicing on the shallow features and weighted features of each sensor in the channel dimension to obtain splicing features of each sensor; The spliced ​​features of the sensors are fused to obtain a comprehensive feature vector of the user.

7. The traffic behavior recognition method according to claim 6, characterized in that: The step of fusing the spliced ​​features of the sensors to obtain a comprehensive feature vector of the user comprises: Performing convolution processing on the spliced ​​features of the sensors to obtain a fourth feature; Performing a pooling operation on the fourth feature to obtain a fifth feature; The fifth feature is input into the fully connected layer to obtain a comprehensive feature vector of the user.

8. A traffic behavior recognition device, characterized in that: include: A data acquisition module, used to acquire sensor data of each sensor in the user terminal; A residual processing module, used for inputting the sensor data of each sensor into a residual network to extract features of a local receptive field, and obtaining shallow features and local features of each sensor; A time domain processing module, used for inputting shallow features and local features of each sensor into a time domain convolutional neural network to extract features of the global receptive field, so as to obtain deep features of each sensor; An attention processing module, used for inputting the deep features of each sensor into an attention network to obtain weighted features of each sensor; A feature splicing module, used to splice the shallow features and weighted features of each sensor to obtain a comprehensive feature vector of the user; The behavior recognition module is used to input the comprehensive feature vector into the classification layer to obtain the traffic behavior recognition result of the user.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the traffic behavior recognition method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the traffic behavior recognition method according to any one of claims 1 to 7 are implemented.

Images