Intelligent vehicle security inspection method and device, computer device and storage medium

Abstract

The application relates to a human-vehicle intelligent security method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring heart rate variation information of a driver, each facial image of the driver and multi-angle photographed vehicle images, and identifying facial expression information corresponding to each facial image; identifying psychological state information of the driver based on the heart rate variation information of the driver, and identifying emotion change distribution information of the driver based on each facial expression information; identifying vehicle abnormal information of the vehicle based on each vehicle image, and identifying intention information of the driver through an abnormal emotion analysis strategy based on the psychological state information of the driver and the emotion change distribution information of the driver; and determining a target detection result of the vehicle based on the intention information of the driver and the vehicle abnormal information of the vehicle. The method can improve the comprehensiveness of human-vehicle security.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and Internet of Things technologies, and in particular to a method, device, computer equipment, and storage medium for intelligent security inspection of people and vehicles. Background Technology

[0002] In recent years, police checkpoints have played a crucial role in addressing the current pressures of social control and the security challenges of major events. Currently, police checkpoints mostly rely on manual inspection and verification, followed by manual control and release. This approach requires a large workforce, is inefficient, involves fragmented workflows, and is highly demanding, easily causing traffic congestion.

[0003] Traditional vehicle and passenger security checks utilize security checkpoints, in-vehicle personnel information collection subsystems, vehicle information collection subsystems, under-vehicle image acquisition subsystems, and vehicle blocking subsystems to automate the checks, allowing occupants to undergo security verification without leaving the vehicle. However, this method can only perform security checks on basic vehicle and personnel information and cannot detect hidden items inside the vehicle, resulting in a lack of comprehensiveness in vehicle and passenger security checks. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, device, computer equipment, computer-readable storage medium, and computer program product for intelligent security inspection of people and vehicles to address the above-mentioned technical problems.

[0005] Firstly, this application provides an intelligent security inspection method for people and vehicles. The method includes:

[0006] The system acquires information on the driver's heart rate changes, images of the driver's faces, and vehicle images taken from multiple angles, and identifies facial expression information corresponding to each facial image.

[0007] Based on the driver's heart rate change information, the driver's psychological state information is identified, and based on each of the facial expression information, the driver's emotional change distribution information is identified.

[0008] Based on the vehicle images, abnormal vehicle information is identified, and based on the driver's psychological state information and the driver's emotional change distribution information, the driver's intention information is identified through an abnormal emotion analysis strategy.

[0009] Based on the driver's intent information and the vehicle's abnormality information, the target detection result of the vehicle is determined.

[0010] Optionally, the identification of facial expression information corresponding to each facial image includes:

[0011] For each facial image, an image feature recognition network is used to identify the facial features in the facial image, and for each facial feature image, a micro-expression analysis algorithm is used to identify the micro-expression information corresponding to each facial feature image.

[0012] Based on all micro-expression information, the driver's facial expression information is determined.

[0013] Optionally, identifying the driver's psychological state information based on the driver's heart rate change information includes:

[0014] The driver's heart rate change information is sorted in chronological order to obtain the driver's heart rate change distribution information;

[0015] Identify the heart rate trend information and the heart rate distribution range of the heart rate change distribution information, and identify the driver's psychological state type based on the heart rate distribution range;

[0016] Based on the driver's heart rate trend information, the driver's corresponding psychological state information is matched within each psychological state range corresponding to the psychological state type.

[0017] Optionally, the step of identifying the driver's emotional change distribution information based on each of the facial expression information includes:

[0018] The facial expression information is sorted according to time sequence to obtain the driver's expression change distribution information, and the emotional information corresponding to each facial expression information is identified respectively.

[0019] According to the order of the facial expression change distribution information, the emotional information is distributed and sorted to obtain the driver's emotional change distribution information.

[0020] Optionally, identifying vehicle anomaly information based on each of the vehicle images includes:

[0021] Identify the vehicle region corresponding to each vehicle image, and for each vehicle image, based on the vehicle region corresponding to the vehicle image, identify the image feature content in each vehicle image through an image feature analysis algorithm;

[0022] The image classification network identifies the content category corresponding to each image feature and the probability value of each content category. If there is an abnormal content category and the probability value of the abnormal content category is greater than a preset probability threshold, the vehicle image, the abnormal content category, and the probability value of the abnormal content category are used as the vehicle abnormality information of the vehicle.

[0023] Optionally, the step of identifying the driver's intention information based on the driver's psychological state information and the driver's emotional change distribution information through an abnormal emotion analysis strategy includes:

[0024] Based on the driver's emotional change distribution information, the driver's emotional trend information and the range of the driver's emotional fluctuations are identified;

[0025] Based on the driver's emotional trend information, the target emotional information of the driver is identified, and based on the driver's emotional fluctuation range, the probability of the driver's emotional change is identified.

[0026] Based on the driver's psychological state information, analyze the initial action intentions corresponding to the driver's psychological state information, and identify the emotional range of each initial action intention;

[0027] Based on the driver's target emotion information, within the emotion range of each initial action intention, the action intention corresponding to the emotion range containing the driver's target emotion information is selected as the driver's target intention, and the target intention, along with the driver's emotion change probability, is used as the driver's intention information.

[0028] Optionally, determining the target detection result of the vehicle based on the driver's intent information and the vehicle's anomaly information includes:

[0029] If the vehicle has abnormal information, the detection result of the vehicle is determined to be an abnormal vehicle, and based on the driver's intention information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified.

[0030] The probability of the driver breaking through the checkpoint, the route taken by the driver to break through the checkpoint, and the detection result of the vehicle are used as the target detection result of the vehicle.

[0031] If there is no abnormal vehicle information, the driver's level of tension is identified based on the driver's target emotion information and the driver's psychological state information. If the driver's level of tension is greater than a preset tension threshold, the vehicle is determined to be an abnormal vehicle.

[0032] Based on the driver's intent information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified, and the probability of the driver running the checkpoint, the route of the driver running the checkpoint, and the detection result of the vehicle are used as the target detection result of the vehicle.

[0033] Optionally, the method further includes:

[0034] Based on the probability of the driver running the checkpoint, the target vehicle blocking devices that need to be activated are determined, and based on the route taken by the driver to run the checkpoint, the activation sequence of each target vehicle blocking device is determined.

[0035] Collect the current operating information of the vehicle, and based on the current operating information of the vehicle, adjust each of the target vehicle blocking devices and the activation sequence of each of the target vehicle blocking devices;

[0036] Return to the step of collecting the current operating information of the vehicle until the vehicle stops in the target area or in response to the staff's closing operation, and then close all target vehicle blocking devices.

[0037] Secondly, this application also provides an intelligent security inspection device for people and vehicles. The device includes:

[0038] The acquisition module is used to acquire the driver's heart rate change information, the driver's facial images, and vehicle images taken from multiple angles, and to identify the facial expression information corresponding to each facial image.

[0039] The recognition module is used to recognize the driver's psychological state information based on the driver's heart rate change information, and to recognize the driver's emotional change distribution information based on each of the facial expression information;

[0040] The analysis module is used to identify vehicle anomaly information based on the vehicle images, and to identify the driver's intention information based on the driver's psychological state information and the driver's emotional change distribution information through an abnormal emotion analysis strategy.

[0041] The determination module is used to determine the target detection result of the vehicle based on the driver's intention information and the vehicle's abnormal information.

[0042] Optionally, the acquisition module is specifically used for:

[0043] For each facial image, an image feature recognition network is used to identify the facial features in the facial image, and for each facial feature image, a micro-expression analysis algorithm is used to identify the micro-expression information corresponding to each facial feature image.

[0044] Based on all micro-expression information, the driver's facial expression information is determined.

[0045] Optionally, the identification module is specifically used for:

[0046] The driver's heart rate change information is sorted in chronological order to obtain the driver's heart rate change distribution information;

[0047] Identify the heart rate trend information and the heart rate distribution range of the heart rate change distribution information, and identify the driver's psychological state type based on the heart rate distribution range;

[0048] Based on the driver's heart rate trend information, the driver's corresponding psychological state information is matched within each psychological state range corresponding to the psychological state type.

[0049] Optionally, the identification module is specifically used for:

[0050] The facial expression information is sorted according to time sequence to obtain the driver's expression change distribution information, and the emotional information corresponding to each facial expression information is identified respectively.

[0051] According to the order of the facial expression change distribution information, the emotional information is distributed and sorted to obtain the driver's emotional change distribution information.

[0052] Optionally, the analysis module is specifically used for:

[0053] Identify the vehicle region corresponding to each vehicle image, and for each vehicle image, based on the vehicle region corresponding to the vehicle image, identify the image feature content in each vehicle image through an image feature analysis algorithm;

[0054] The image classification network identifies the content category corresponding to each image feature and the probability value of each content category. If there is an abnormal content category and the probability value of the abnormal content category is greater than a preset probability threshold, the vehicle image, the abnormal content category, and the probability value of the abnormal content category are used as the vehicle abnormality information of the vehicle.

[0055] Optionally, the analysis module is specifically used for:

[0056] Based on the driver's emotional change distribution information, the driver's emotional trend information and the range of the driver's emotional fluctuations are identified;

[0057] Based on the driver's emotional trend information, the target emotional information of the driver is identified, and based on the driver's emotional fluctuation range, the probability of the driver's emotional change is identified.

[0058] Based on the driver's psychological state information, analyze the initial action intentions corresponding to the driver's psychological state information, and identify the emotional range of each initial action intention;

[0059] Based on the driver's target emotion information, within the emotion range of each initial action intention, the action intention corresponding to the emotion range containing the driver's target emotion information is selected as the driver's target intention, and the target intention, along with the driver's emotion change probability, is used as the driver's intention information.

[0060] Optionally, the determining module is specifically used for:

[0061] If the vehicle has abnormal information, the detection result of the vehicle is determined to be an abnormal vehicle, and based on the driver's intention information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified.

[0062] The probability of the driver breaking through the checkpoint, the route taken by the driver to break through the checkpoint, and the detection result of the vehicle are used as the target detection result of the vehicle.

[0063] If there is no abnormal vehicle information, the driver's level of tension is identified based on the driver's target emotion information and the driver's psychological state information. If the driver's level of tension is greater than a preset tension threshold, the vehicle is determined to be an abnormal vehicle.

[0064] Based on the driver's intent information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified, and the probability of the driver running the checkpoint, the route of the driver running the checkpoint, and the detection result of the vehicle are used as the target detection result of the vehicle.

[0065] Optionally, the device further includes:

[0066] The vehicle blocking determination module is used to determine the target vehicle blocking devices that need to be activated based on the probability of the driver running through the checkpoint, and to determine the activation sequence of each target vehicle blocking device based on the route taken by the driver running through the checkpoint.

[0067] The detection module is used to collect the current operating information of the vehicle and, based on the current operating information of the vehicle, adjust each of the target vehicle blocking devices and the activation sequence of each of the target vehicle blocking devices;

[0068] The cyclic adjustment module is used to return to the step of collecting the current operating information of the vehicle until the vehicle stops in the target area or in response to the staff's closing operation, and then close all target vehicle blocking devices.

[0069] Thirdly, this application provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any one of the first aspects.

[0070] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0071] Fifthly, this application provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0072] The aforementioned intelligent vehicle and personnel security inspection method, device, computer equipment, storage medium, and computer program product acquire and scan the driver's heart rate change information, various facial images of the driver, and vehicle images captured from multiple angles, and identify facial expression information corresponding to each facial image; based on the driver's heart rate change information, identify the driver's psychological state information, and based on the facial expression information, identify the driver's emotional change distribution information; based on the vehicle images, identify vehicle anomaly information, and based on the driver's psychological state information and the driver's emotional change distribution information, identify the driver's intention information through an abnormal emotion analysis strategy; based on the driver's intention information and the vehicle anomaly information, determine the target detection result of the vehicle. In this embodiment, by combining the driver's facial images, heart rate change information, and actual vehicle images, a comprehensive analysis is conducted to determine whether there are any vehicle anomalies. This ensures that the driver does not need to get out of the vehicle for inspection, achieving both comprehensive detection of abnormal information in the vehicle's external structure and comprehensive detection of potential anomalies, thereby improving the comprehensiveness of vehicle and personnel security inspections. Attached Figure Description

[0073] Figure 1 This is a flowchart illustrating an intelligent security inspection method for people and vehicles in one embodiment;

[0074] Figure 2 This is a structural block diagram of a smart security inspection device for people and vehicles in one embodiment;

[0075] Figure 3 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0077] The intelligent vehicle and passenger security inspection method provided in this application can be applied to vehicle security inspection environments. This method can be applied to a terminal, a server, or a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The terminal combines the driver's facial image, heart rate change information, and actual vehicle image to comprehensively analyze whether there are any abnormalities in the vehicle. Without requiring the driver to get out of the vehicle for inspection, it can achieve comprehensive detection of abnormal information in the vehicle's external structure, as well as comprehensive detection of potential abnormalities, thereby improving the comprehensiveness of vehicle and passenger security inspections.

[0078] In one embodiment, such as Figure 1 As shown, an intelligent security inspection method for people and vehicles is provided. Taking the application of this method to a terminal as an example, the method includes the following steps:

[0079] Step S101: Obtain the driver's heart rate change information, various facial images of the driver, and vehicle images taken from multiple angles, and identify the facial expression information corresponding to each facial image.

[0080] In this embodiment, the terminal repeatedly collects the driver's heart rate information using heart rate detection devices installed at the checkpoint and at different target locations in front of the checkpoint, obtaining heart rate change information. Then, the terminal uses infrared scanning devices installed at the checkpoint and at different target locations in front of the checkpoint to scan each driver's face multiple times, generating a facial image for each driver. This facial image is a three-dimensional facial image of the driver. Next, the terminal uses multi-angle cameras installed at the checkpoint to collect image data of the vehicle from different angles, obtaining vehicle images. These vehicle images are two-dimensional images. To check for items concealed inside the vehicle structure (items concealed inside the seats, fuel tank, tires, or vehicle interlayers, etc.), X-ray scan images of the vehicle from different angles transmitted by a large vehicle X-ray scanning device can be used as vehicle images.

[0081] Finally, the terminal identifies the facial expression information corresponding to each facial image. This identification is achieved by analyzing the driver's facial expressions after facial feature recognition using an image feature recognition network. The specific analysis process will be explained in detail later. This image feature recognition network is a reinforcement learning-based convolutional neural network (BackPropagation, BP). The facial expression information includes micro-expressions corresponding to different facial feature images, such as the eyes, eyebrows, mouth, nose, and ears.

[0082] Step S102: Based on the driver's heart rate change information, identify the driver's psychological state information, and based on each facial expression information, identify the driver's emotional change distribution information.

[0083] In this embodiment, the terminal identifies the driver's psychological state information based on heart rate changes and identifies the driver's emotional distribution information based on facial expressions. The psychological state information includes agitation, calmness, fear, anxiety, hesitation, and depression, while the emotional distribution information includes multiple emotional states such as attention, fatigue, tension, relaxation, sadness, and joy. The specific identification process will be explained in detail later.

[0084] Step S103: Based on the images of each vehicle, identify abnormal vehicle information, and based on the driver's psychological state information and the distribution information of the driver's emotional changes, identify the driver's intention information through an abnormal emotion analysis strategy.

[0085] In this embodiment, the terminal identifies vehicle anomaly information based on vehicle images and, based on the driver's psychological state and emotional change distribution information, identifies the driver's intent information through an abnormal emotion analysis strategy. The vehicle anomaly information includes anomalies on the vehicle's external structural surfaces and internal structural surfaces that can be directly observed inside the lathe. These anomalies include abnormal vehicle modifications, abnormal seatbelt wearing, and abnormal additions to non-original vehicles (e.g., concealed or illegal items). The specific identification process will be described in detail later. The driver's intent information includes the driver's behavioral intention and the probability information of that intention. This behavioral intention includes actions such as vehicle acceleration, vehicle deceleration, stopping, U-turn, and following guidance.

[0086] Step S104: Based on the driver's intent information and the vehicle's abnormal information, determine the target detection result of the vehicle.

[0087] In this embodiment, the terminal determines the target detection result of the vehicle based on the driver's intention information and the vehicle's abnormal information.

[0088] Based on the above scheme, by combining the driver's facial image, heart rate change information, and actual vehicle image, a comprehensive analysis is conducted to determine whether there are any abnormalities in the vehicle. This allows for comprehensive detection of abnormal information in the vehicle's external structure and comprehensive detection of potential abnormalities without requiring the driver to get out of the vehicle for inspection, thereby improving the comprehensiveness of vehicle and personnel safety checks.

[0089] Optionally, identifying facial expression information corresponding to each facial image includes: for each facial image, identifying facial feature images in the facial image through an image feature recognition network, and for each facial feature image, identifying micro-expression information corresponding to each facial feature image through a micro-expression analysis algorithm; and determining the driver's facial expression information based on all micro-expression information.

[0090] In this embodiment, for each facial image, the terminal uses an image feature recognition network to identify facial feature images, and for each facial feature image, uses a micro-expression analysis algorithm to identify the corresponding micro-expression information. The facial feature images include organ feature images of each facial organ. The micro-expression analysis algorithm can be, but is not limited to, the Local Binary Pattern (LBP) algorithm.

[0091] Then, the terminal summarizes all the micro-expression information and, according to the distribution position of each facial organ in the facial image, splices and combines the micro-expression information corresponding to the organ feature images of each facial organ to obtain the driver's facial expression information.

[0092] Specifically, the process of identifying the micro-expression information corresponding to each organ feature image is as follows: The terminal acquires the feature image of each organ feature, and then extracts the expression texture features corresponding to each feature image using the LBP algorithm. Next, the terminal queries the micro-expression database for the micro-expression texture features of the facial organs corresponding to each feature image at different age groups. Then, using a convolutional neural network (CNN) model, the expression texture features corresponding to each feature image are matched against the micro-expression texture features of the facial organs corresponding to each feature image at different age groups to obtain the micro-expression information corresponding to each expression texture feature.

[0093] Based on the above scheme, by extracting micro-expression information corresponding to facial feature images, the driver's facial expression information can be identified, thereby improving the accuracy of identifying the driver's facial expression information.

[0094] Optionally, based on the driver's heart rate change information, the driver's psychological state information is identified, including: sorting the driver's heart rate change information in chronological order to obtain the driver's heart rate change distribution information; identifying the heart rate trend information and the heart rate distribution range of the heart rate change distribution information, and identifying the driver's psychological state type based on the heart rate distribution range; and matching the driver's corresponding psychological state information in each psychological state range corresponding to the psychological state type based on the driver's heart rate trend information.

[0095] In this embodiment, the terminal sorts the driver's heart rate change information in chronological order to obtain the driver's heart rate change distribution information. Then, the terminal identifies the heart rate trend information and the heart rate distribution range of the heart rate change distribution information. The method for identifying the heart rate trend information is to use an Autoregressive Integrated Moving Average Model (ARIMA) to predict the driver's heart rate change trend after the current moment (i.e., after the last moment of the heart rate change distribution information) based on the heart rate change distribution information, thereby obtaining the driver's heart rate trend information.

[0096] Then, the terminal identifies the sample heart rate distribution range to which the heart rate distribution range belongs in the psychological state database, and uses the psychological state type corresponding to that sample heart rate distribution range as the driver's psychological state type for that heart rate distribution range. This psychological state type includes normal psychological state types and abnormal psychological state types. Normal psychological states include stable psychological states, regular psychological states, and excited psychological states, etc. Abnormal psychological states include impatient psychological states, stable psychological states, fearful psychological states, anxious psychological states, hesitant psychological states, and depressed psychological states, etc.

[0097] Finally, based on the driver's heart rate trend information, the corresponding psychological state information is matched within each psychological state range corresponding to the psychological state type. Within each psychological state type, different heart rate ranges correspond to different psychological states. The terminal identifies the driver's future heart rate based on the driver's heart rate trend information and queries the heart rate range to which that heart rate belongs, thereby obtaining the driver's psychological state information.

[0098] Based on the above scheme, by predicting heart rate trends and querying the psychological state types corresponding to heart rate distributions, the driver's psychological state information can be determined, thereby improving the accuracy of the determined driver's psychological state information.

[0099] Optionally, based on each facial expression information, the driver's emotional change distribution information is identified, including: sorting each facial expression information in chronological order to obtain the driver's emotional change distribution information, and identifying the emotional information corresponding to each facial expression information; sorting each emotional information according to the order of the emotional change distribution information to obtain the driver's emotional change distribution information.

[0100] In this embodiment, the terminal sorts the facial expression information in chronological order to obtain the driver's expression change distribution information, and identifies the emotion information corresponding to each facial expression. Then, the terminal sorts the emotion information according to the order of the expression change distribution information to obtain the driver's emotion change distribution information.

[0101] Specifically, the process of identifying the emotional information corresponding to each facial expression is as follows: the terminal identifies the micro-expression texture features corresponding to each facial organ in each facial expression, and based on the micro-expression information corresponding to the expression texture features (e.g., furrowed brows, dull eyes, downturned corners of the mouth, slightly parted lips, furrowed brows, pursed corners of the mouth, etc.), it queries the emotion database for the emotional information of all micro-expression information containing the facial expression information, and uses this as the emotional information of the facial expression information.

[0102] For example, when the emotional information is happiness, the facial expression is bright eyes and upturned corners of the mouth; when the emotional information is sadness, the facial expression is furrowed brows, mournful eyes, and downturned corners of the mouth; when the emotional information is anger, the facial expression is furrowed brows, sharp eyes, and a closed or open mouth; when the emotional information is bewilderment, the facial expression is furrowed brows, wandering eyes, and pursed corners of the mouth; when the emotional information is disappointment, the facial expression is furrowed brows, dim eyes, and slightly downturned corners of the mouth; when the emotional information is embarrassment, the facial expression is shifty eyes and slightly parted lips.

[0103] Based on the above scheme, by identifying the emotional information corresponding to the micro-expression information of each facial expression, the driver's emotional change distribution information is generated, thereby improving the comprehensiveness and accuracy of the obtained driver's emotional change distribution information.

[0104] Optionally, based on each vehicle image, identify vehicle anomaly information, including: identifying the vehicle region corresponding to each vehicle image, and for each vehicle image, based on the vehicle region corresponding to the vehicle image, identifying the image feature content in each vehicle image through an image feature analysis algorithm; identifying the content category corresponding to each image feature content and the probability value of each content category through an image classification network, and if there is an abnormal content category and the probability value of the abnormal content category is greater than a preset probability threshold, using the vehicle image, the abnormal content category, and the probability value of the abnormal content category as the vehicle anomaly information.

[0105] In this embodiment, the terminal identifies the vehicle region corresponding to each vehicle image, and for each vehicle image, based on the vehicle region corresponding to the vehicle image, it identifies the image feature content in each vehicle image through an image feature analysis algorithm.

[0106] Next, the terminal uses an image classification network to identify the content category corresponding to each image feature and the probability value of each content category. This image classification network is the VGG-Net image classification network, and its output layer includes a probabilistic recognition algorithm to identify the probability value corresponding to each classified content category. This probabilistic recognition algorithm includes, but is not limited to, numerical probabilistic algorithms, Monte Carlo algorithms, Las Vegas algorithms, and Sherwood algorithms.

[0107] Then, the terminal queries the database to find the content category whose identifier is abnormal content. If the content category of each image feature is abnormal and the probability value of the abnormal content category is greater than a preset probability threshold, the terminal uses the vehicle image, the abnormal content category, and the probability value of the abnormal content category as the vehicle's abnormal information. The preset probability threshold is a probability value preset by the staff on the terminal.

[0108] Based on the above scheme, by classifying and recognizing image features and determining the probability of each category, abnormal vehicle information can be identified, thereby improving the accuracy of the identified abnormal vehicle information.

[0109] Optionally, based on the driver's psychological state information and the driver's emotional change distribution information, an abnormal emotion analysis strategy is used to identify the driver's intention information, including: identifying the driver's emotional trend information and the driver's emotional fluctuation range based on the driver's emotional change distribution information; identifying the driver's target emotional information based on the driver's emotional trend information, and identifying the driver's emotional change probability based on the driver's emotional fluctuation range; analyzing each initial action intention corresponding to the driver's psychological state information based on the driver's psychological state information, and identifying the emotional range of each initial action intention; and selecting the action intention corresponding to the emotional range containing the driver's target emotional information from the emotional range of each initial action intention as the driver's target intention, and using the target intention and the driver's emotional change probability as the driver's intention information.

[0110] In this embodiment, the terminal identifies the driver's emotional trend information and emotional fluctuation range based on the driver's emotional change distribution information. Based on the driver's emotional trend information, it identifies the driver's target emotional information, and based on the driver's emotional fluctuation range, it identifies the probability of the driver's emotional change. The method for identifying the driver's emotional trend information is the same as the method for identifying heart rate trend information described above, and will not be repeated here. The method for identifying the probability of emotional change is as follows: the terminal identifies the heart rate change information corresponding to the emotional change distribution information, and based on the change information of the heart rate change range corresponding to this heart rate change information, the terminal calculates the slope of the change information of the heart rate change range and normalizes the slope to obtain the probability of the driver's emotional change.

[0111] Next, based on the driver's psychological state information, the terminal analyzes the initial action intentions corresponding to the driver's psychological state information. The intention database contains various action intentions corresponding to different psychological state information. The terminal queries the intention database for the initial action intentions corresponding to the driver's psychological state information.

[0112] The terminal identifies the emotional range of each initial action intention. Then, based on the driver's target emotional information, the terminal filters the action intentions corresponding to the emotional ranges containing the driver's target emotional information within each initial action intention's emotional range, and uses these as the driver's target intentions. The target intentions, along with the probability of the driver's emotional change, are used as the driver's intention information.

[0113] Based on the above scheme, by identifying the trend and probability of drivers' emotional changes, the intentions of drivers can be analyzed to obtain information about their intentions, thereby preventing the risk of drivers running away from checkpoints and improving the prevention effect against different drivers' intentions to run away from checkpoints.

[0114] Optionally, the vehicle target detection result is determined based on the driver's intent information and the vehicle's abnormal information, including: if the vehicle has abnormal information, the vehicle is determined to be an abnormal vehicle, and based on the driver's intent information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified; the probability of the driver running the checkpoint, the route of the driver running the checkpoint, and the vehicle's detection result are used as the vehicle's target detection result; if the vehicle does not have abnormal information, the driver's level of tension is identified based on the driver's target emotion information and the driver's psychological state information, and if the driver's level of tension is greater than a preset tension threshold, the vehicle is determined to be an abnormal vehicle; based on the driver's intent information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified, and the probability of the driver running the checkpoint, the route of the driver running the checkpoint, and the vehicle's detection result are used as the vehicle's target detection result.

[0115] In this embodiment, when abnormal vehicle information is detected, the terminal determines the vehicle to be an abnormal vehicle and identifies the driver's probability of ramming the checkpoint and the possible route based on the driver's intent information. Specifically, if the corresponding action intent information in the driver's intent information indicates an intent to ram the checkpoint, the driver's probability of ramming the checkpoint is 1 minus the driver's emotional change probability. If the corresponding action intent information does not indicate an intent to ram the checkpoint, the driver's probability of ramming the checkpoint is the driver's emotional change probability. The driver's route is determined by the terminal collecting the current vehicle position and the road conditions ahead of the vehicle, and using a route planning algorithm to calculate the optimal driving route after the vehicle avoids the suspects and the roadblocks.

[0116] Then, the terminal uses the driver's probability of running the checkpoint, the driver's route of running the checkpoint, and the vehicle detection results as the vehicle target detection results.

[0117] When no abnormal vehicle information is found, the terminal identifies the driver's level of tension based on the driver's target emotional information and psychological state information. The level of tension is identified by calculating the similarity between the driver's psychological state information and information indicating a state of tension, and using this similarity as the driver's level of tension. The similarity algorithm can be, but is not limited to, cosine similarity, Euclidean distance, and Mahalanobis distance algorithms.

[0118] The terminal determines the vehicle's detection result as abnormal when the driver's level of tension exceeds a preset tension threshold. This tension threshold is preset by the staff on the terminal and can be adjusted as needed.

[0119] Then, based on the driver's intent information, the terminal identifies the probability of the driver attempting to evade the checkpoint and the route the driver might take. This probability, route, and vehicle detection result are then used as the vehicle's target detection result. (Ibid.)

[0120] Based on the above scheme, by analyzing the driver's psychological state information and the driver's intention information, the probability and route of the driver running the checkpoint are analyzed, thereby improving the early warning effect of preventing drivers from running the checkpoint.

[0121] Optionally, the method further includes: determining the target vehicle blocking devices to be activated based on the driver's probability of ramming through the checkpoint, and determining the activation sequence of each target vehicle blocking device based on the driver's route of ramming through the checkpoint; collecting the current operating information of the vehicle, and adjusting each target vehicle blocking device and the activation sequence of each target vehicle blocking device based on the current operating information of the vehicle; returning to the step of collecting the current operating information of the vehicle until the vehicle stops in the target area or responds to the closing operation of the staff, and then closing all target vehicle blocking devices.

[0122] In this embodiment, the terminal determines the target vehicle blocking devices to be activated based on the probability of the driver attempting to evade the checkpoint, and determines the activation sequence of each target vehicle blocking device based on the driver's route. The vehicle blocking devices are arranged sequentially according to the road conditions to prevent vehicle movement. These devices include mechanical vehicle blocking devices, sensor-activated vehicle blocking devices, fixed vehicle blocking devices, electronic vehicle blocking devices, and alarm devices. Mechanical vehicle blocking devices include vehicle blocking barriers, telescopic spikes, and mechanically gravity-activated spikes. Sensor-activated vehicle blocking devices include infrared vehicle blocking devices (vehicle blocking posts, barriers, or fan-shaped devices activated by sensing vehicle movement). Fixed vehicle blocking devices can be concrete pillars or guardrails. Electronic vehicle blocking devices include signal jamming devices (used to interfere with the vehicle's computer and electronic signals, or laser equipment). The alarm device sends basic vehicle information and its location directly to an early warning center (police station, traffic police center, road traffic control center, etc.) when the vehicle begins to evade the checkpoint.

[0123] Then, the terminal collects the vehicle's current operating information and, based on this information, adjusts each target vehicle blocking device and its activation sequence. The activation sequence refers to the sequence of target vehicle blocking devices that need to be activated after the previous target vehicle blocking device has been activated, provided the vehicle's speed has not yet reached zero.

[0124] Then, the terminal returns to the step of collecting the current operating information of the vehicle until the vehicle stops in the target area or in response to the staff's closing operation, at which point all target vehicle blocking devices are closed.

[0125] Based on the above scheme, by progressively blocking vehicles that attempt to ram through checkpoints from multiple directions, angles, and levels, the effectiveness of blocking such vehicles has been improved.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0127] Based on the same inventive concept, this application also provides an intelligent vehicle security inspection device for implementing the aforementioned intelligent vehicle security inspection method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the intelligent vehicle security inspection device provided below can be found in the limitations of the intelligent vehicle security inspection method described above, and will not be repeated here.

[0128] In one embodiment, such as Figure 2 As shown, an intelligent security inspection device for people and vehicles is provided, including: an acquisition module 210, an identification module 220, an analysis module 230, and a determination module 240, wherein:

[0129] The acquisition module 210 is used to acquire the heart rate change information of the scanned driver, various facial images of the driver, and vehicle images taken from multiple angles, and to identify the facial expression information corresponding to each facial image.

[0130] The recognition module 220 is used to recognize the driver's psychological state information based on the driver's heart rate change information, and to recognize the driver's emotional change distribution information based on each of the facial expression information;

[0131] The analysis module 230 is used to identify vehicle abnormality information based on each of the vehicle images, and to identify the driver's intention information based on the driver's psychological state information and the driver's emotional change distribution information through an abnormal emotion analysis strategy.

[0132] The determination module 240 is used to determine the target detection result of the vehicle based on the driver's intention information and the vehicle's abnormal information.

[0133] Optionally, the acquisition module 210 is specifically used for:

[0134] For each facial image, an image feature recognition network is used to identify the facial features in the facial image, and for each facial feature image, a micro-expression analysis algorithm is used to identify the micro-expression information corresponding to each facial feature image.

[0135] Based on all micro-expression information, the driver's facial expression information is determined.

[0136] Optionally, the identification module 220 is specifically used for:

[0137] The driver's heart rate change information is sorted in chronological order to obtain the driver's heart rate change distribution information;

[0138] Identify the heart rate trend information and the heart rate distribution range of the heart rate change distribution information, and identify the driver's psychological state type based on the heart rate distribution range;

[0139] Based on the driver's heart rate trend information, the driver's corresponding psychological state information is matched within each psychological state range corresponding to the psychological state type.

[0140] Optionally, the identification module 220 is specifically used for:

[0141] The facial expression information is sorted according to time sequence to obtain the driver's expression change distribution information, and the emotional information corresponding to each facial expression information is identified respectively.

[0142] According to the order of the facial expression change distribution information, the emotional information is distributed and sorted to obtain the driver's emotional change distribution information.

[0143] Optionally, the analysis module 230 is specifically used for:

[0144] Identify the vehicle region corresponding to each vehicle image, and for each vehicle image, based on the vehicle region corresponding to the vehicle image, identify the image feature content in each vehicle image through an image feature analysis algorithm;

[0145] The image classification network identifies the content category corresponding to each image feature and the probability value of each content category. If there is an abnormal content category and the probability value of the abnormal content category is greater than a preset probability threshold, the vehicle image, the abnormal content category, and the probability value of the abnormal content category are used as the vehicle abnormality information of the vehicle.

[0146] Optionally, the analysis module 230 is specifically used for:

[0147] Based on the driver's emotional change distribution information, the driver's emotional trend information and the range of the driver's emotional fluctuations are identified;

[0148] Based on the driver's emotional trend information, the target emotional information of the driver is identified, and based on the driver's emotional fluctuation range, the probability of the driver's emotional change is identified.

[0149] Based on the driver's psychological state information, analyze the initial action intentions corresponding to the driver's psychological state information, and identify the emotional range of each initial action intention;

[0150] Based on the driver's target emotion information, within the emotion range of each initial action intention, the action intention corresponding to the emotion range containing the driver's target emotion information is selected as the driver's target intention, and the target intention, along with the driver's emotion change probability, is used as the driver's intention information.

[0151] Optionally, the determining module 240 is specifically used for:

[0152] If the vehicle has abnormal information, the detection result of the vehicle is determined to be an abnormal vehicle, and based on the driver's intention information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified.

[0153] The probability of the driver breaking through the checkpoint, the route taken by the driver to break through the checkpoint, and the detection result of the vehicle are used as the target detection result of the vehicle.

[0154] If there is no abnormal vehicle information, the driver's level of tension is identified based on the driver's target emotion information and the driver's psychological state information. If the driver's level of tension is greater than a preset tension threshold, the vehicle is determined to be an abnormal vehicle.

[0155] Based on the driver's intent information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified, and the probability of the driver running the checkpoint, the route of the driver running the checkpoint, and the detection result of the vehicle are used as the target detection result of the vehicle.

[0156] Optionally, the device further includes:

[0157] The vehicle blocking determination module is used to determine the target vehicle blocking devices that need to be activated based on the probability of the driver running through the checkpoint, and to determine the activation sequence of each target vehicle blocking device based on the route taken by the driver running through the checkpoint.

[0158] The detection module is used to collect the current operating information of the vehicle and, based on the current operating information of the vehicle, adjust each of the target vehicle blocking devices and the activation sequence of each of the target vehicle blocking devices;

[0159] The cyclic adjustment module is used to return to the step of collecting the current operating information of the vehicle until the vehicle stops in the target area or in response to the staff's closing operation, and then close all target vehicle blocking devices.

[0160] Each module in the aforementioned intelligent security inspection device for people and vehicles can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0161] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a smart security inspection method for people and vehicles. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0162] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any one of the first aspects.

[0164] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0165] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0167] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for intelligent security inspection of people and vehicles, characterized in that, The method includes: Heart rate detection devices are installed at checkpoints and at different target locations in front of checkpoints to collect the driver's heart rate change information multiple times; the driver's face is scanned multiple times by infrared scanning devices to obtain images of the driver's face; and vehicle images are collected from different angles by multi-angle camera devices, including vehicle internal structure scan images transmitted by large vehicle X-ray scanning equipment. For each facial image, the facial feature image is identified by an image feature recognition network, and the micro-expression analysis algorithm is used to extract the micro-expression information corresponding to each facial feature image. The facial expression information corresponding to each facial image is determined by combining all the micro-expression information. The heart rate change information is sorted in chronological order to obtain heart rate change distribution information; the heart rate trend information corresponding to the heart rate change distribution information is predicted by an autoregressive moving average model, and the heart rate distribution range of the heart rate change distribution information is identified; the driver's psychological state type is matched based on the heart rate distribution range, and the driver's psychological state information is determined by combining the heart rate trend information. The facial expression information is sorted in chronological order to obtain expression change distribution information; the emotion information corresponding to each facial expression information is identified, and the emotion information is sorted in the order of the expression change distribution information to obtain the driver's emotion change distribution information. Based on the vehicle images, the vehicle region is identified and image feature content is extracted. The category and corresponding probability value of the image feature content are identified through an image classification network. When there is an abnormal content category and its probability value is greater than a preset threshold, the vehicle image, abnormal content category and probability value are used as vehicle abnormal information. The vehicle abnormal information includes abnormal vehicle modification, abnormal seat belt wearing, and abnormal vehicle additives for non-original vehicles. Based on the emotional change distribution information, emotional trend information and emotional fluctuation range are identified, target emotional information is determined and the probability of emotional change is calculated; combined with the psychological state information, the initial action intention and corresponding emotional range are queried, action intentions containing the target emotional information are selected as target intentions, and the target intentions and the probability of emotional change are used as intention information. The target detection result is determined based on the intent information and vehicle anomaly information: If there is abnormal vehicle information, the vehicle is identified as an abnormal vehicle. Based on the driver's intent information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified. The probability of the driver running the checkpoint, the route of the driver running the checkpoint, and the vehicle detection result are used as the target detection result of the vehicle. In the absence of any abnormal vehicle information, the driver's level of tension is identified based on the driver's target emotional information and psychological state information. When the driver's level of tension exceeds a preset tension threshold, the vehicle is determined to be an abnormal vehicle. Based on the driver's intention information, the probability of the driver running the checkpoint and the route of the driver running the checkpoint are identified. The probability of the driver running the checkpoint, the route of the driver running the checkpoint, and the vehicle's detection result are used as the target detection result of the vehicle.

2. The method according to claim 1, characterized in that, The method further includes: Based on the probability of the driver running the checkpoint, the target vehicle blocking devices that need to be activated are determined, and based on the route taken by the driver to run the checkpoint, the activation sequence of each target vehicle blocking device is determined. Collect the current operating information of the vehicle, and based on the current operating information of the vehicle, adjust each of the target vehicle blocking devices and the activation sequence of each of the target vehicle blocking devices; Return to the step of collecting the current operating information of the vehicle until the vehicle stops in the target area or in response to the staff's closing operation, and then close all target vehicle blocking devices.

3. A smart security inspection device for people and vehicles, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the method of any one of claims 1 to 2.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 2.

Images