Non-motor vehicle manned detection and identity verification method and system

Abstract

The invention relates to a non-motor vehicle manned detection and identity verification method and system. The method comprises the steps of collecting riding image data through an intelligent camera, inputting the riding image data into a target detection algorithm, and detecting non-motor vehicle and rider information in the to-be-detected image data; if the non-motor vehicle carries people, cross-camera trajectory tracking is carried out on the rider, and an action trajectory of the rider is determined by using an ReID algorithm; and selecting a target position according to the action track, extracting facial features of the rider at the target position by using a face recognition algorithm, and obtaining identity information of the rider based on the facial features. The intelligent camera is used for collecting riding image data, and a target detection algorithm and a face recognition algorithm are combined, so that the manned behavior of a non-motor vehicle can be found in time, personnel identity verification can be quickly completed, the false detection and missing detection conditions of a non-motor vehicle lane detection scheme can be avoided, the detection precision is improved, and the identity of a violation person is determined.

Description

Technical Field

[0001] The present invention relates to the technical field of non-motor vehicle safety detection, and in particular to a method and system for non-motor vehicle passenger detection and identity verification. Background Art

[0002] With the development of society, people's living standards are constantly improving, and more and more people pay attention to the convenience of travel. As the main means of daily travel, the standardized management of non-motorized vehicles' passenger behavior is of great significance to maintaining travel safety. At present, the supervision of non-motorized vehicles carrying passengers is relatively limited and often relies on manual patrols. This method is not only inefficient, but also difficult to achieve accurate monitoring around the clock and without blind spots. When conducting non-motorized vehicle passenger detection, traditional detection methods face many challenges, including difficulty in adapting to the complexity of the environment and the diversity of personnel flow, insufficient detection accuracy of non-motorized vehicle passenger behavior, and lack of effective identity verification methods, making it difficult to quickly lock in violating individuals.

[0003] Therefore, traditional non-motorized vehicle lane detection schemes have many false detections and missed detections, and have low detection accuracy and are unable to determine the identity of violators. Summary of the invention

[0004] Based on this, in order to solve the above technical problems, a method and system for non-motor vehicle passenger detection and identity verification is provided, which can improve the detection accuracy and determine the identity of the violator.

[0005] A method for detecting and verifying the identity of a person on a non-motor vehicle, the method comprising:

[0006] Collecting riding image data through a smart camera, and performing preprocessing operations on the riding image data to generate image data to be detected;

[0007] Inputting the image data to be detected into a target detection algorithm to detect the non-motor vehicle and rider information in the image data to be detected;

[0008] Determine the relative position relationship and posture characteristics of the non-motor vehicle and the rider according to the non-motor vehicle and the rider information, and determine whether there is a non-motor vehicle carrying a rider based on the relative position relationship and the posture characteristics;

[0009] If there is a non-motor vehicle carrying passengers, the rider is tracked across cameras and the ReID algorithm is used to determine the rider's movement trajectory;

[0010] A target position is selected according to the motion trajectory, facial features of the rider are extracted at the target position using a face recognition algorithm, and identity information of the rider is obtained based on the facial features.

[0011] In one embodiment, riding image data is collected by a smart camera, and preprocessing operations are performed on the riding image data to generate image data to be detected, including:

[0012] Through various intelligent cameras distributed in the collection area, each riding image data in the non-motor vehicle driving area is collected respectively;

[0013] De-noising, cutting and compression processing operations are performed on the riding image data respectively to obtain image data to be detected.

[0014] In one of the embodiments, an attention mechanism is introduced into the target detection algorithm;

[0015] Inputting the image data to be detected into a target detection algorithm to detect non-motor vehicles and rider information in the image data to be detected includes:

[0016] Inputting the image data to be detected into a target detection algorithm, and dividing the image data to be detected into grids by the target detection algorithm;

[0017] Taking non-motor vehicles and cyclists as targets, performing target detection in each of the grids, and outputting the bounding box position and category probability of the target;

[0018] Based on the bounding box position and category probability of the target, the non-motor vehicle and rider information is obtained.

[0019] In one embodiment, the method further comprises:

[0020] Generate a feature tensor based on the number of channels according to the height and width of the image data to be detected by the target detection algorithm;

[0021] Reshape the feature tensor, obtain the spatial attention weight through the attention mechanism, reshape the spatial attention weight back to the spatial dimension, and output the spatial attention;

[0022] The feature tensor is reshaped, the channel attention weight is calculated through the attention mechanism, and the channel attention weight is expanded to the same dimension as the feature tensor, and the channel attention is output.

[0023] In one embodiment, determining the relative position relationship and posture characteristics of the non-motor vehicle and the rider based on the non-motor vehicle and the rider information, and judging whether there is a non-motor vehicle carrying a rider based on the relative position relationship and the posture characteristics, includes:

[0024] Divide the detection area according to the non-motor vehicle and rider information, and determine a human target area and a non-motor vehicle target area in the detection area;

[0025] Expanding the non-motor vehicle target area to obtain a passenger determination area;

[0026] The number of human figures in the human-shaped target area is calculated in the passenger determination area, and whether there is a non-motor vehicle carrying a passenger is determined based on the number of human figures.

[0027] In one embodiment, if there is a non-motor vehicle carrying a passenger, the rider is tracked across cameras, and the ReID algorithm is used to determine the rider's movement trajectory, including:

[0028] If there is a non-motor vehicle carrying a passenger, the ReID algorithm is used to extract the appearance features of the rider;

[0029] Comparing the appearance features with a personnel feature library to complete cross-camera tracking and preliminary identity recognition of the rider;

[0030] The movement trajectory of the rider is determined based on the preliminary identity recognition result and the cross-camera tracking result.

[0031] In one embodiment, the appearance features of the rider are extracted by the ReID algorithm, including:

[0032] Obtaining a test image feature vector according to the image data to be detected;

[0033] Searching for known pedestrian image feature vectors from a database, and respectively calculating vector distances between the known pedestrian image feature vectors and the test image feature vectors;

[0034] The pedestrian image feature vector with the smallest vector distance is output as the appearance feature of the cyclist.

[0035] In one embodiment, extracting facial features of the rider using a face recognition algorithm at the target location, and obtaining identity information of the rider based on the facial features includes:

[0036] Obtaining image information at the target location and inputting it into the face recognition algorithm to extract facial features of the rider;

[0037] Calculating a cosine distance between the facial feature and a reference facial feature in a database;

[0038] Determine a distance threshold, and compare the cosine distance with the distance threshold to obtain a comparison result;

[0039] The identity information of the rider is obtained according to the comparison result.

[0040] In one embodiment, the method further comprises:

[0041] The riding image data and the identity information of the rider are visualized.

[0042] A non-motor vehicle passenger detection and identity verification system, the system comprising:

[0043] An image data acquisition module is used to collect riding image data through a smart camera, and perform preprocessing operations on the riding image data to generate image data to be detected;

[0044] An information detection module, used for inputting the image data to be detected into a target detection algorithm, and detecting the information of non-motor vehicles and riders in the image data to be detected;

[0045] A passenger detection module, used to determine the relative position relationship and posture characteristics of the non-motor vehicle and the rider according to the non-motor vehicle and the rider information, and judge whether there is a non-motor vehicle carrying a passenger based on the relative position relationship and posture characteristics;

[0046] A trajectory tracking module is used to track the rider across cameras if there is a non-motor vehicle carrying passengers, and use the ReID algorithm to determine the rider's movement trajectory;

[0047] The personnel identity verification module is used to select a target position according to the movement trajectory, extract the facial features of the rider at the target position using a face recognition algorithm, and obtain the identity information of the rider based on the facial features.

[0048] The above-mentioned non-motor vehicle passenger detection and identity verification method and system, by using intelligent cameras to collect riding image data and combining target detection algorithms with face recognition algorithms, can promptly detect non-motor vehicle passenger behavior and quickly complete personnel identity verification, which can avoid false detection and missed detection of non-motor vehicle lane detection schemes, thereby improving detection accuracy and determining the identity of violators. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A diagram showing an application environment of a method for detecting and verifying a person on a non-motor vehicle in an embodiment;

[0050] Figure 2 A schematic diagram of a process of a non-motor vehicle passenger detection and identity verification method in one embodiment;

[0051] Figure 3 A schematic diagram of area division in one embodiment;

[0052] Figure 4 A structural block diagram of a non-motor vehicle passenger detection and identity verification system in one embodiment;

[0053] Figure 5 It is a structural block diagram of a non-motor vehicle passenger detection and identity verification system in another embodiment;

[0054] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] The non-motor vehicle passenger detection and identity verification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 As shown, the application environment includes a computer device 110 and various smart cameras 120, and each smart camera 120 is connected to the computer device 110 for communication. The computer device 110 can collect riding image data through the smart camera 120, and perform preprocessing operations on the riding image data to generate image data to be detected; the computer device 110 can input the image data to be detected into the target detection algorithm to detect the non-motor vehicle and rider information in the image data to be detected; the computer device 110 can determine the relative position relationship and posture characteristics of the non-motor vehicle and the rider based on the non-motor vehicle and rider information, and determine whether there is a non-motor vehicle carrying a passenger based on the relative position relationship and posture characteristics; if there is a non-motor vehicle carrying a passenger, the computer device 110 can track the rider across the camera trajectory and determine the rider's movement trajectory using the ReID algorithm; the computer device 110 can select a target position based on the movement trajectory, extract the rider's facial features at the target position using the face recognition algorithm, and obtain the rider's identity information based on the facial features. Among them, the computer device 110 can be but is not limited to various personal computers, laptops, smart phones, robots, unmanned aerial vehicles and other devices; the smart camera 120 can be a high-definition smart camera with high resolution, wide dynamic range and low-light adaptability, which can clearly capture image data of non-motor vehicles and riders under different weather and lighting conditions.

[0057] In one embodiment, Figure 2 As shown, a method for detecting and verifying the identity of a person on a non-motor vehicle is provided, comprising the following steps:

[0058] Step 202: collect riding image data through a smart camera, and perform preprocessing operations on the riding image data to generate image data to be detected.

[0059] Smart cameras can be installed at various major detection points in areas where non-motor vehicle passenger detection is required. Taking the non-motor vehicle passenger detection area as a school as an example, high-definition smart cameras can be deployed at key non-motor vehicle traffic sections around the main entrances and exits of the campus, teaching buildings, dormitory areas, libraries, etc. according to the distribution of campus roads, traffic flow and personnel activity patterns. Among them, smart cameras have high resolution, wide dynamic range and low-light adaptability, and can clearly capture image data of non-motor vehicles and riders under different weather and lighting conditions; at the same time, smart cameras are equipped with adjustable angle mounting brackets to obtain the best shooting angle to ensure full coverage of non-motor vehicle driving areas on campus.

[0060] In one embodiment, a non-motor vehicle passenger detection and identity verification method is provided which may include an image data preprocessing process, the specific process including: collecting various riding image data within the non-motor vehicle driving area through various intelligent cameras distributed in the collection area; performing denoising, cutting, and compression processing operations on the riding image data to obtain the image data to be detected.

[0061] Step 204: input the image data to be detected into a target detection algorithm to detect the information of non-motor vehicles and riders in the image data to be detected.

[0062] Among them, the target detection algorithm can be a YOLO series algorithm that introduces an attention mechanism, which can be used to more accurately detect the status of non-motor vehicles and personnel on campus.

[0063] Specifically, taking the non-motor vehicle passenger detection area as a school as an example, the computer equipment can use a deep convolutional neural network to extract features from video images and learn the unique features of non-motor vehicles and people in the campus environment. Among them, the deep convolutional neural network can be a network model that is customized and trained according to the characteristics of non-motor vehicles and people on campus. The improved YOLO series algorithm introduces an attention mechanism. The YOLO network model reduces the amount of calculation while ensuring the accuracy of feature extraction to meet the requirements of real-time campus monitoring, and can better identify various types of non-motor vehicles on campus as well as teachers and students with different postures and clothing.

[0064] By performing target detection on specific behavior patterns on campus, non-motor vehicles and personnel are detected using the YOLO series of target detection algorithms optimized based on the characteristics of non-motor vehicles carrying passengers on campus. Taking full account of the common driving patterns of non-motor vehicles on campus, through training and optimization, the target detection algorithm can be more focused on key areas and features related to passenger behavior, thereby accurately detecting whether a non-motor vehicle is carrying passengers and effectively reducing false detection and missed detection rates.

[0065] In one embodiment, a non-motor vehicle passenger detection and identity verification method provided may also include a process of detecting non-motor vehicle and rider information, the specific process including: inputting the image data to be detected into a target detection algorithm, dividing the image data to be detected into grids through the target detection algorithm; taking non-motor vehicles and riders as targets, performing target detection in each grid, and outputting the target's bounding box position and category probability; and obtaining non-motor vehicle and rider information based on the target's bounding box position and category probability.

[0066] Specifically, the YOLO series of algorithms mainly achieve fast and accurate target detection by dividing the image into grids, performing target prediction in each grid, and directly outputting the target's bounding box position and category probability. The design of the loss function in the target detection algorithm takes into account coordinate error, confidence error, and category error. By optimizing these errors, the network can learn better target detection capabilities.

[0067] In one embodiment, a non-motor vehicle passenger detection and identity verification method provided may also include a process of using an attention mechanism to enhance model performance, and the specific process includes: generating a feature tensor based on the number of channels according to the height and width of the image data to be detected through a target detection algorithm; reshaping the feature tensor, obtaining a spatial attention weight through an attention mechanism, and reshaping the spatial attention weight back to a spatial dimension, and outputting the spatial attention; reshaping the feature tensor, calculating a channel attention weight through an attention mechanism, and expanding the channel attention weight to the same dimension as the feature tensor, and outputting the channel attention.

[0068] The main purpose of introducing the attention mechanism in the target detection algorithm is to make the model pay more attention to the important parts of the input, thereby improving the performance of the model. The output of a certain layer in the target detection algorithm is a feature tensor X∈R C×H×W , where C is the number of channels, H and W are the height and width of the feature map, respectively. An attention mechanism can be introduced at this layer to enhance the model's attention to important features.

[0069] Specifically, for spatial attention, the computer device can reshape the feature tensor X to obtain X reshaped ∈R C，N , where N = H × W; then, we can define a query vector q s ∈R C , the spatial attention weight a is obtained by the attention calculation method s ∈R N : Among them, j is used to traverse the reshaped vector data, T represents the transposed matrix, and then the attention weight a can be s Reshape back to the spatial dimension, i.e. A s∈R 1×H×W ; where l represents the l channel, which is used to reshape the weight back to 1xHxW; the final output is in Represents element-wise multiplication.

[0070] For channel attention, the computer device can reshape the feature tensor X to obtain X reshaped ∈R N,C ; Next, we can define a query vector q c ∈R N , the channel attention weight a is obtained through attention calculation c ∈R C : Then, the attention weight a c Expanded to the same dimension as the input feature tensor, i.e. A c ∈R C×1×1 The final output is

[0071]

[0072] In this embodiment, through the spatial attention and channel attention mechanisms, the target detection algorithm can pay more attention to the important parts of the input, thereby improving the performance and generalization ability of the model; among them, the specific form and calculation method of the attention mechanism can be adjusted and optimized according to different models.

[0073] Step 206, determining the relative position relationship and posture characteristics of the non-motor vehicle and the rider based on the non-motor vehicle and the rider information, and judging whether there is a passenger on the non-motor vehicle based on the relative position relationship and the posture characteristics.

[0074] Computer equipment can judge the behavior of people. After detecting non-motor vehicles and people, the relative position relationship and posture characteristics of people and non-motor vehicles are further analyzed, and the behavior of people is judged in combination with the actual situation of the campus environment. For example, in the case that students on campus may carry school bags, books and other items, more sophisticated posture analysis and item recognition algorithms are used to accurately distinguish between normal carrying of items and carrying of people.

[0075] In one embodiment, a non-motor vehicle passenger detection and identity verification method provided may also include a process of determining whether there is a non-motor vehicle passenger. The specific process includes: dividing the detection area according to the non-motor vehicle and rider information, and determining a human target area and a non-motor vehicle target area in the detection area; expanding the non-motor vehicle target area to obtain a passenger determination area; calculating the number of human figures in the human target area within the passenger determination area, and determining whether there is a non-motor vehicle passenger based on the number of human figures.

[0076] Computer equipment can use a target detection algorithm optimized based on campus-specific behavior patterns to detect whether a non-motor vehicle is carrying passengers. At the same time, it can combine the relative position and posture characteristics of the person and the non-motor vehicle to determine whether there is any violation of the regulations on campus.

[0077] Specifically, the computer device can divide the detection area, such as Figure 3 As shown, the human target area is divided into P, and the non-motor vehicle target area is divided into N; then, the N area can be expanded upward by 1 / 2 times the height to serve as the passenger determination area K; the number of P NUM is determined by IOU in area K as the basis for passenger; if NUM>1, passenger behavior is detected, otherwise the detection is normal.

[0078] Step 208: If there is a non-motor vehicle carrying passengers, the rider is tracked across cameras and the ReID algorithm is used to determine the rider's movement trajectory.

[0079] In one embodiment, a non-motor vehicle passenger detection and identity verification method provided may also include a process of determining a movement trajectory, the specific process including: if there is a non-motor vehicle passenger, extracting the appearance features of the rider through a ReID algorithm; comparing the appearance features with a personnel feature library to complete cross-camera tracking and preliminary identity recognition of the rider; and determining the movement trajectory of the rider based on the preliminary identity recognition results and the cross-camera tracking results.

[0080] When a non-motor vehicle carrying a passenger is detected, the ReID algorithm is first used to track the rider across cameras and perform preliminary identity recognition to determine their movement trajectory within the campus. By introducing the pedestrian re-identification (ReID) algorithm, the features of the detected non-motor vehicle riders are extracted and matched; multiple cameras are set up at different locations within the campus. When non-motor vehicles and people pass through the monitoring areas of different cameras, the ReID algorithm extracts the appearance features of the person (such as clothing, body shape, posture, etc.) and compares them with the established person feature library to achieve cross-camera tracking and preliminary identity recognition of the person, and determine the person's movement trajectory within the campus.

[0081] Step 210, selecting a target location according to the motion trajectory, extracting facial features of the rider at the target location using a face recognition algorithm, and obtaining identity information of the rider based on the facial features.

[0082] Specifically, in one embodiment, a non-motor vehicle passenger detection and identity verification method provided may also include a process of extracting appearance features of the rider, the specific process including: obtaining a test image feature vector based on the image data to be detected; searching for known pedestrian image feature vectors from a database, and respectively calculating the vector distances between the known pedestrian image feature vectors and the test image feature vectors; and outputting the pedestrian image feature vector with the smallest vector distance as the appearance feature of the rider.

[0083] In this embodiment, for the test image t to be recognized, it is input into the trained model M to obtain its feature vector f t =M(t); Then, the computer device can calculate the distance between the test image feature vector and all known pedestrian image feature vectors in the database: Let the pedestrian image feature vector set in the database be D = {f d1 ,f d2 ,...,f dm}, using Euclidean distance for calculation, for each feature vector f in the database dj , whose distance to the feature vector of the test image is Then, the computer device can find the pedestrian corresponding to the feature vector with the smallest distance as the recognition result, that is, determine the index The identified pedestrian is the pedestrian with the corresponding index k in the database.

[0084] In one embodiment, a non-motor vehicle passenger detection and identity verification method provided may also include a process of obtaining identity information, and the specific process includes: obtaining image information at the target position and inputting it into a face recognition algorithm to extract the facial features of the rider; calculating the cosine distance between the facial features and the reference facial features in the database; determining a distance threshold, and comparing the cosine distance with the distance threshold to obtain a comparison result; and obtaining the rider's identity information based on the comparison result.

[0085] When non-motor vehicles carrying passengers or suspected violations are detected in the human and non-motor vehicle target detection, the face recognition algorithm is activated to extract facial features of the rider. By comparing the extracted facial features with the campus personnel database (including photos and related identity information of students and faculty), the specific identity information of the rider can be quickly and accurately determined, and the violators can be accurately located.

[0086] In this embodiment, the computer device may use a face comparison algorithm to compare the similarity of the face images. The face comparison algorithm is generally intended to determine the similarity between two face images. Specifically, the computer device may set the two face images to be I and I 1 and I 2 , use the feature extraction function f to extract facial features, and the feature vectors are v1 =f(I 1 ) and v 2 =f(I 2 ); Assume that the dimension of the feature vector is d, that is, v 1 =(v 11 ,v 12 ,...,v 1d ) and $v 2 =(v 21 ,v 22 ,...,v 2d ).

[0087] Computer equipment can use Euclidean distance as a measure of face distance. Specifically, Euclidean distance measures the straight-line distance between two feature vectors in space, and the calculation formula is: The smaller the Euclidean distance, the more similar the two face images are.

[0088] In another embodiment, the computer device may use cosine distance as a measure of face distance. Cosine distance measures the cosine value of the angle between two vectors, and the calculation formula is: in, is the vector dot product, and They are vector v 1 and v 2 The closer the cosine distance is to 0, the more similar the two face images are. Preferably, the cosine distance can be used as a measure of face distance.

[0089] In this embodiment, the computer device may set a threshold value T. When the calculated distance is less than the threshold value, it can be considered that the two face images belong to the same person. The specific judgment method is: if d(v 1 ,v 2 )<T, then the two face images are judged to be of the same person; otherwise, they are judged to be different persons. The determination of the threshold T usually needs to be adjusted and optimized according to the specific application scenario and data set. An appropriate threshold can be determined by conducting experiments on the training set or the validation set.

[0090] In one embodiment, a feature registration module can be used for ReID features and facial features. When logging into a non-motor vehicle, ReID features and facial features are extracted and saved in a feature database. The feature database needs to be queried in the personnel identity verification module for identity confirmation.

[0091] In one embodiment, a non-motor vehicle passenger detection and identity verification method provided may also include a result output and feedback process, and the specific process includes: visualizing riding image data and rider identity information.

[0092] Specifically, the computer equipment can output the non-motor vehicle passenger detection results and the rider identity verification results in an intuitive and clear manner. The location of the non-motor vehicle carrying passengers and the relevant personnel information (including name, student number or work number, class or department, etc.) are marked in real time on the display screen of the campus security monitoring center, and alarm information is sent to campus security personnel in a timely manner. At the same time, the computer equipment also has a feedback mechanism. According to the detection results during actual use on campus and the feedback from security personnel, the algorithm model is optimized and adjusted to continuously improve the detection performance and the accuracy of identity verification to better adapt to changes in the campus environment and new behavior patterns.

[0093] A non-motor vehicle passenger detection and identity verification method provided in this embodiment, when applied to a campus environment, detects the non-motor vehicle and rider information in the image data to be detected through a target detection algorithm, can highly accurately detect the non-motor vehicle passenger behavior in the campus, reduce the occurrence of false detection and missed detection, and provide reliable data support for campus safety management; can meet the needs of real-time campus monitoring, timely discover non-motor vehicle passenger behavior and quickly complete personnel identity verification, provide campus security personnel with timely and accurate early warning information, facilitate rapid management measures, and improve the efficiency of campus safety management; due to the combined application of the ReID algorithm and the face recognition algorithm, the accurate verification of the identity of non-motor vehicle riders can be achieved; the identity information of the violators can be quickly and accurately determined, providing a strong basis for subsequent education and management work, which is conducive to improving the pertinence and effectiveness of campus safety management; and the system has good scalability and compatibility, and can be easily integrated with other campus security management systems (such as access control systems, monitoring systems, etc.) to achieve information sharing and collaborative work, forming an organic campus safety prevention and control system, and further improving the comprehensive safety management level of the campus.

[0094] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0095] In one embodiment, Figure 4As shown, a non-motor vehicle passenger detection and identity verification system is provided, including: an image data acquisition module 410, an information detection module 420, a passenger detection module 430, a trajectory tracking module 440 and a personnel identity verification module 450, wherein:

[0096] The image data acquisition module 410 is used to collect riding image data through a smart camera, and perform preprocessing operations on the riding image data to generate image data to be detected;

[0097] The information detection module 420 is used to input the image data to be detected into the target detection algorithm to detect the information of non-motor vehicles and riders in the image data to be detected;

[0098] The passenger detection module 430 is used to determine the relative position relationship and posture characteristics of the non-motor vehicle and the rider according to the non-motor vehicle and the rider information, and judge whether there is a non-motor vehicle carrying a passenger based on the relative position relationship and the posture characteristics;

[0099] The trajectory tracking module 440 is used to track the rider across cameras if there is a non-motor vehicle carrying a passenger, and determine the rider's movement trajectory using a ReID algorithm;

[0100] The person identity verification module 450 is used to select a target location according to the movement trajectory, extract the facial features of the rider at the target location using a face recognition algorithm, and obtain the identity information of the rider based on the facial features.

[0101] In one embodiment, the image data acquisition module 410 is also used to collect various riding image data within the non-motor vehicle driving area through various smart cameras distributed in the acquisition area; and perform denoising, cutting, and compression processing operations on the riding image data to obtain the image data to be detected.

[0102] In one embodiment, an attention mechanism is introduced into the target detection algorithm; the information detection module 420 is also used to input the image data to be detected into the target detection algorithm, and divide the image data to be detected into grids through the target detection algorithm; non-motor vehicles and cyclists are taken as targets, target detection is performed in each grid, and the bounding box position and category probability of the target are output; based on the bounding box position and category probability of the target, the non-motor vehicle and cyclist information is obtained.

[0103] In one embodiment, the information detection module 420 is also used to generate a feature tensor based on the number of channels according to the height and width of the image data to be detected through a target detection algorithm; reshape the feature tensor, obtain the spatial attention weight through an attention mechanism, and reshape the spatial attention weight back to the spatial dimension, and output the spatial attention; reshape the feature tensor, calculate the channel attention weight through an attention mechanism, and expand the channel attention weight to the same dimension as the feature tensor, and output the channel attention.

[0104] In one embodiment, the passenger detection module 430 is also used to divide the detection area according to the information of non-motor vehicles and riders, and determine the human target area and non-motor vehicle target area in the detection area; expand the non-motor vehicle target area to obtain the passenger determination area; calculate the number of human figures in the human target area in the passenger determination area, and determine whether there are non-motor vehicle passengers based on the number of human figures.

[0105] In one embodiment, the trajectory tracking module 440 is also used to extract the appearance features of the rider through the ReID algorithm if there is a non-motor vehicle carrying passengers; compare the appearance features with the personnel feature library to complete cross-camera tracking and preliminary identity recognition of the rider; and determine the movement trajectory of the rider based on the preliminary identity recognition results and the cross-camera tracking results.

[0106] In one embodiment, the trajectory tracking module 440 is also used to obtain a test image feature vector based on the image data to be detected; search for known pedestrian image feature vectors from a database, and calculate the vector distances between the known pedestrian image feature vectors and the test image feature vectors; and output the pedestrian image feature vector with the smallest vector distance as the appearance feature of the cyclist.

[0107] In one embodiment, the personnel identity verification module 450 is also used to obtain image information at the target location and input it into the face recognition algorithm to extract the facial features of the rider; calculate the cosine distance between the facial features and the reference facial features in the database; determine the distance threshold, and compare the cosine distance with the distance threshold to obtain a comparison result; and obtain the identity information of the rider based on the comparison result.

[0108] In one embodiment, Figure 5 As shown, a non-motor vehicle passenger detection and identity verification system provided may also include a feature registration module 460 and a result output and feedback module 470, wherein:

[0109] The feature registration module 460 is used to utilize the feature registration module for ReID features and face features, extract ReID features and face features when logging into a non-motor vehicle, and save them to the feature database; the personnel identity verification module needs to query the feature database for identity confirmation;

[0110] The result output and feedback module 470 is used to visualize the riding image data and the identity information of the rider.

[0111] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting and verifying the passenger on a non-motor vehicle is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0112] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of a method for detecting and verifying passengers on a non-motor vehicle are implemented.

[0114] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for detecting and verifying the identity of a person on a non-motor vehicle are implemented.

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting and verifying the identity of a person on a non-motor vehicle, characterized in that: The method comprises: Collecting riding image data through a smart camera, and performing preprocessing operations on the riding image data to generate image data to be detected; Inputting the image data to be detected into a target detection algorithm to detect the non-motor vehicle and rider information in the image data to be detected; Determine the relative position relationship and posture characteristics of the non-motor vehicle and the rider according to the non-motor vehicle and the rider information, and determine whether there is a non-motor vehicle carrying a rider based on the relative position relationship and the posture characteristics; If there is a non-motor vehicle carrying passengers, the rider is tracked across cameras and the ReID algorithm is used to determine the rider's movement trajectory; A target position is selected according to the motion trajectory, facial features of the rider are extracted at the target position using a face recognition algorithm, and identity information of the rider is obtained based on the facial features.

2. The non-motor vehicle passenger detection and identity verification method according to claim 1, characterized in that: The riding image data is collected by a smart camera, and a preprocessing operation is performed on the riding image data to generate image data to be detected, including: Through various intelligent cameras distributed in the collection area, each riding image data in the non-motor vehicle driving area is collected respectively; De-noising, cutting and compression processing operations are performed on the riding image data respectively to obtain image data to be detected.

3. The non-motor vehicle passenger detection and identity verification method according to claim 1, characterized in that: An attention mechanism is introduced into the target detection algorithm; Inputting the image data to be detected into a target detection algorithm to detect non-motor vehicles and rider information in the image data to be detected includes: Inputting the image data to be detected into a target detection algorithm, and dividing the image data to be detected into grids by the target detection algorithm; Taking non-motor vehicles and cyclists as targets, performing target detection in each of the grids, and outputting the bounding box position and category probability of the target; Based on the bounding box position and category probability of the target, the non-motor vehicle and rider information is obtained.

4. The non-motor vehicle passenger detection and identity verification method according to claim 3, characterized in that: The method further comprises: Generate a feature tensor based on the number of channels according to the height and width of the image data to be detected by the target detection algorithm; Reshape the feature tensor, obtain the spatial attention weight through the attention mechanism, reshape the spatial attention weight back to the spatial dimension, and output the spatial attention; The feature tensor is reshaped, the channel attention weight is calculated through the attention mechanism, and the channel attention weight is expanded to the same dimension as the feature tensor, and the channel attention is output.

5. The non-motor vehicle passenger detection and identity verification method according to claim 1, characterized in that: Determining the relative position relationship and posture characteristics of the non-motor vehicle and the rider based on the non-motor vehicle and the rider information, and judging whether there is a non-motor vehicle carrying a rider based on the relative position relationship and the posture characteristics, including: Divide the detection area according to the non-motor vehicle and rider information, and determine a human target area and a non-motor vehicle target area in the detection area; Expanding the non-motor vehicle target area to obtain a passenger determination area; The number of human figures in the human-shaped target area is calculated in the passenger determination area, and whether there is a non-motor vehicle carrying a passenger is determined based on the number of human figures.

6. The non-motor vehicle passenger detection and identity verification method according to claim 1, characterized in that: If there is a non-motor vehicle carrying a passenger, the rider is tracked across cameras and the ReID algorithm is used to determine the rider's movement trajectory, including: If there is a non-motor vehicle carrying a passenger, the ReID algorithm is used to extract the appearance features of the rider; Comparing the appearance features with a personnel feature library to complete cross-camera tracking and preliminary identity recognition of the rider; The movement trajectory of the rider is determined based on the preliminary identity recognition result and the cross-camera tracking result.

7. The non-motor vehicle passenger detection and identity verification method according to claim 6, characterized in that: The ReID algorithm is used to extract the appearance features of the rider, including: Obtaining a test image feature vector according to the image data to be detected; Searching for known pedestrian image feature vectors from a database, and calculating vector distances between the known pedestrian image feature vectors and the test image feature vectors; The pedestrian image feature vector with the smallest vector distance is output as the appearance feature of the cyclist.

8. The non-motor vehicle passenger detection and identity verification method according to claim 1, characterized in that: Extracting facial features of the rider using a face recognition algorithm at the target location, and obtaining identity information of the rider based on the facial features, including: Acquire image information at the target position and input it into the face recognition algorithm to extract facial features of the rider; Calculating a cosine distance between the facial feature and a reference facial feature in a database; Determine a distance threshold, and compare the cosine distance with the distance threshold to obtain a comparison result; The identity information of the rider is obtained according to the comparison result.

9. The non-motor vehicle passenger detection and identity verification method according to claim 1, characterized in that: The method further comprises: The riding image data and the identity information of the rider are visualized.

10. A non-motor vehicle passenger detection and identity verification system, characterized in that: The system comprises: An image data acquisition module is used to collect riding image data through a smart camera, and perform preprocessing operations on the riding image data to generate image data to be detected; An information detection module, used for inputting the image data to be detected into a target detection algorithm, and detecting the non-motor vehicle and rider information in the image data to be detected; A passenger detection module, used to determine the relative position relationship and posture characteristics of the non-motor vehicle and the rider according to the non-motor vehicle and the rider information, and judge whether there is a non-motor vehicle carrying a passenger based on the relative position relationship and posture characteristics; A trajectory tracking module is used to track the rider across cameras if there is a non-motor vehicle carrying passengers, and determine the rider's movement trajectory using a ReID algorithm; The personnel identity verification module is used to select a target position according to the movement trajectory, extract the facial features of the rider at the target position using a face recognition algorithm, and obtain the identity information of the rider based on the facial features.

Images