Personnel and vehicle monitoring method and system based on capsule network

Abstract

The invention discloses a personnel and vehicle monitoring method and system based on a capsule network, and the method comprises the steps: collecting monitoring audios and videos, and separating the monitoring audios and videos to obtain sound data and image data; performing feature extraction on the sound data and the image data, wherein the feature extraction comprises extraction of low-level features and extraction of high-level abstract features; fusing the abstract features of the sound and the image obtained after feature extraction to generate a feature fusion vector, and processing the feature fusion vector by using a capsule network to identify feature information of personnel and vehicles in the monitored audio and video; and performing data comparison processing on the identified feature information of the personnel and the vehicle and pre-registration information to output monitoring results of the target personnel and the target vehicle. According to the method and system, target identification and classification are carried out by using image features and sound features, so that the identification accuracy can be obviously improved, and an accurate identification result is finally obtained.

Description

technical field [0001] The invention relates to the technical field of security monitoring, in particular to a method and system for monitoring personnel and vehicles based on a capsule network. Background technique [0002] With the continuous acceleration of the urbanization process in our country, the security monitoring and emergency management of urban streets, industrial parks, and residential quarters are important tasks related to social security and stability. In this field, it is an important method to improve the level of security monitoring, and it is also the main direction of technological development, to monitor personnel, vehicles, and the relationship between them, and to find abnormal information in time and give an alarm. [0003] Existing recognition technologies mostly use the single-modal recognition method of visual feature analysis to identify the facial features and clothing features of people; but in practice, most surveillance audio and video canno...

AI Technical Summary

Problems solved by technology

[0005] However, the existing recognition technology ignores the analysis of the sound signal, which limits the further improvement of the recognition accuracy. At present, there are very few researches on the recognition based on the fusion of audio and video features. The main difficulty encountered is that the features of different modalities are not easy to integrate. ;Secondly, the traditional technology needs to use a relatively complete image to analyze, but the monitoring system usually only collects the front and side images of the vehicle, which has fewer features and reduces the recognition accuracy; moreover, the traditional technology only uses traditional image processing and matching technology , difficult to fine-tune and do not have the ability to learn and grow

Images