Security check quality evaluation method, system and equipment based on human body key point detection

Abstract

The invention provides a security check quality evaluation method, system and equipment based on human body key point detection. The method comprises the following steps: extracting a single-frame image and a multi-frame image according to security check video data; inputting the single-frame image and the initial spatial position feature to at least one preset spatial diagram encoder to output a key point spatial feature; inputting the multi-frame image and the initial time sequence position feature to at least one preset time sequence diagram encoder to output a key point time feature; generating a first prediction key point according to the key point spatial feature, the key point time feature and a preset space-time encoder; determining target loss and a key point coverage rate according to the first prediction key point and the reference key point; and generating a security check quality score according to the key point effective contact rate, the target loss and the key point coverage rate. According to the invention, automatic evaluation of the normalization of the personal inspection of the security inspector is realized, the process and the standard of the security inspection personal inspection are further standardized, and the security level of security inspection is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of security inspection technology, and in particular to a security inspection quality assessment method, system and equipment based on human body key point detection. Background Art

[0002] In modern society, security checks are often implemented at entrances and exits of public places, transportation hubs, government agencies, and other locations to improve safety. Personal checks are particularly important during security checks to detect hidden dangerous or contraband items and prevent them from entering protected areas and potentially causing public safety risks.

[0003] Currently, personal inspections rely primarily on manual operations by security inspectors, and the evaluation of security inspectors' performance relies primarily on human oversight, making it difficult to promptly detect irregularities in the security process. Furthermore, manual inspections are subject to discrepancies in operational standards and lack objective, quantitative evaluation criteria for security inspectors' performance, making it impossible to objectively assess security inspection quality. Therefore, a security inspection quality assessment method is urgently needed to address these technical issues in existing technologies. Summary of the Invention

[0004] In order to address the deficiencies of the prior art, the main purpose of the present invention is to provide a security inspection quality assessment method, system and equipment based on human body key point detection to solve the above-mentioned technical problems of the prior art.

[0005] In order to achieve the above objectives, the present invention provides a first aspect of a security inspection quality assessment method based on human key point detection, comprising:

[0006] Extract single-frame images and multi-frame images based on security inspection video data;

[0007] Inputting a single frame image and initial spatial position features to at least one preset spatial graph encoder to output key point spatial features;

[0008] Input multiple frames of images and initial temporal position features to at least one preset temporal graph encoder to output key point temporal features;

[0009] Generate a first predicted key point according to the key point spatial feature, the key point temporal feature and a preset spatiotemporal encoder;

[0010] Determining a target loss and a keypoint coverage rate based on the first predicted keypoint and the reference keypoint, wherein the reference keypoint is a keypoint in a reference 3D human pose, and the target loss includes a keypoint position loss and a keypoint detection order loss;

[0011] The security inspection quality score is generated based on the effective contact rate and target loss of key points and the key point coverage rate.

[0012] In some embodiments, before determining the target loss and the keypoint coverage, the method further includes:

[0013] Updating the first prediction key to generate a second prediction key point;

[0014] Determine the target loss and key point coverage based on the second predicted key point and the benchmark key point;

[0015] The updating of the first prediction key to generate the second prediction key point includes:

[0016] removing noise from key point features contained in single-frame images and multi-frame images to generate transition key point features;

[0017] A second predicted key point is generated according to the transition key point feature, the first predicted key point, and the spatiotemporal encoder.

[0018] In some embodiments, the method is applied to a spatial encoder, inputting a single frame image and initial spatial position features to at least one preset spatial graph encoder to output keypoint spatial features, including:

[0019] Extracting a first spatial graph feature from a single frame image;

[0020] Performing a first processing on the first spatial graph feature to generate a second spatial graph feature, wherein the first spatial graph feature represents a connection state of the key point from the global space to the local space, the first processing comprising normalization and two-dimensional convolution;

[0021] Determine the first spatial position feature based on the multi-head self-attention mechanism and the initial spatial position feature;

[0022] fusing the first spatial position feature and the initial spatial position feature to generate a second spatial position feature and performing a second processing on the second spatial position feature to obtain a third spatial position feature, wherein the second processing includes normalization processing and feedforward neural network processing;

[0023] Performing a dot product operation on the second spatial position feature and the second spatial graph feature to generate a first transition spatial feature;

[0024] fusing the first transition space feature and the second spatial graph feature to generate a second transition space feature;

[0025] The second transition space feature, the third space position feature and the first space graph feature are fused to generate the key point space feature.

[0026] In some embodiments, the method is applied to a timing graph encoder, extracting timing graph features and initial timing position features from multiple frames of images and inputting them into at least one preset timing graph encoder to generate key point time features, including:

[0027] Extracting timing graph features from multiple frames of images;

[0028] Determine a first temporal position feature based on the multi-head attention mechanism and the initial temporal position feature, wherein in the process of determining the first temporal position feature, the input of the softmax function is the temporal graph feature and the attention score;

[0029] fusing the first temporal position feature and the initial temporal position feature to generate a second temporal position feature and performing a third processing on the second temporal position feature to generate a third temporal position feature, wherein the third processing includes normalization and linear layer processing;

[0030] The third temporal position feature and the second temporal position feature are fused to generate a key point temporal feature.

[0031] In some embodiments, the levels of the spatial image encoder and the temporal image encoder are matched one-to-one, and the method includes:

[0032] replacing the first spatial graph feature in the next-level spatial graph encoder of the current spatial graph encoder according to the key point spatial feature generated by the current spatial graph encoder;

[0033] The key point timing features generated by the current timing graph encoder are used to replace the timing graph features in the next level timing graph encoder of the current timing graph encoder.

[0034] In some embodiments, generating a first predicted key point based on the key point spatial feature and the key point temporal feature and a preset spatiotemporal encoder includes:

[0035] The key point spatial features output by the spatial graph encoder at the same level and the key point temporal features output by the temporal graph encoder are concatenated to generate multiple fused key point features;

[0036] Input multiple fused keypoint features to the spatiotemporal encoder to output the first keypoint feature.

[0037] In some embodiments, inputting a plurality of fused keypoint features into a spatiotemporal encoder to output a first keypoint feature includes:

[0038] Determine the first transition fusion feature based on the multi-head attention mechanism and the fusion key point features;

[0039] fusing the first transition fusion feature and the fusion key point feature to generate a second transition fusion feature and normalizing the second transition fusion feature to generate a third transition fusion feature;

[0040] processing the third transition fusion feature according to a feedforward neural network to generate a fourth transition fusion feature;

[0041] The fourth transition fused feature and the second transition fused feature are fused to generate a first predicted key point feature.

[0042] In some embodiments, generating a security inspection quality score based on the effective contact rate and target loss of key points and the key point coverage rate includes:

[0043] The security inspection quality score is calculated according to the preset formula, where the preset formula is:

[0044]

[0045] Among them, S represents the security inspection quality score, Represents the target loss, w1 represents the first weight matching the target loss, C represents the key point coverage, w2 represents the second weight matching the key point coverage, T represents the key point effective contact rate, and w3 represents the third weight matching the key point effective contact rate.

[0046] In a second aspect, the present application provides a security inspection quality assessment system based on human key point detection, the system comprising:

[0047] A preprocessing module is used to extract single-frame images and multi-frame images based on security inspection video data;

[0048] A feature processing module, configured to input a single-frame image and initial spatial position features into at least one preset spatial graph encoder to output keypoint spatial features;

[0049] The feature processing module is further used to input multiple frames of images and initial temporal position features to at least one preset temporal graph encoder to output key point time features;

[0050] The feature processing module is further used to generate a first predicted key point based on the key point spatial feature and the key point temporal feature and a preset spatiotemporal encoder;

[0051] a quality assessment module, configured to determine a target loss and a keypoint coverage rate based on the first predicted keypoints and the reference keypoints, wherein the reference keypoints are keypoints in a reference 3D human pose, and the target loss includes a keypoint position loss and a keypoint detection order loss;

[0052] The quality assessment module is also used to generate security inspection quality scores based on the effective contact rate and target loss of key points and the key point coverage rate.

[0053] In a third aspect, the present application provides an electronic device, the electronic device comprising:

[0054] one or more processors;

[0055] and a memory associated with one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, perform the following operations:

[0056] Extract single-frame images and multi-frame images based on security inspection video data;

[0057] Inputting a single frame image and initial spatial position features to at least one preset spatial graph encoder to output key point spatial features;

[0058] Input multiple frames of images and initial temporal position features to at least one preset temporal graph encoder to output key point temporal features;

[0059] Generate a first predicted key point according to the key point spatial feature, the key point temporal feature and a preset spatiotemporal encoder;

[0060] Determine target loss and key point coverage based on the first predicted key point and the benchmark key point;

[0061] The security inspection quality score is generated based on the effective contact rate and target loss of key points and the key point coverage rate.

[0062] The beneficial effects achieved by this application are:

[0063] The present application provides a security inspection quality assessment method based on human key point detection. The method uses the human key point detection method to analyze security inspection video data to evaluate the quality of the security inspection personal inspection process, including: extracting single-frame images and multi-frame images based on the security inspection video data; inputting the single-frame image and initial spatial position features into at least one preset spatial graph encoder to output key point spatial features; inputting multi-frame images and initial temporal position features into at least one preset temporal graph encoder to output key point temporal features; generating a first predicted key point based on the key point spatial features, key point temporal features, and a preset spatiotemporal encoder; determining the target loss and key point coverage rate based on the first predicted key point and the benchmark key point; and generating a security inspection quality score based on the key point effective contact rate, target loss, and key point coverage rate. This method realizes automatic assessment of the standardization of security inspectors' personal inspections, further standardizes the process and standards for security inspection personal inspections, and effectively improves the safety level of security inspections.

[0064] Not all products in this application need to have all the above effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0066] Figure 1 Schematic diagram of a security inspection quality assessment method based on human key point detection provided in an embodiment of the present application;

[0067] Figure 2 This is a partial flow chart of a security inspection quality assessment method based on human key point detection in an implementation scenario provided by an embodiment of the present application;

[0068] Figure 3 This is a partial flow chart of a security inspection quality assessment method based on human key point detection in another implementation scenario provided by an embodiment of the present application;

[0069] Figure 4 is a schematic diagram of a spatial encoder provided in an embodiment of the present application;

[0070] Figure 5 Schematic diagram of a timing encoder provided in an embodiment of the present application;

[0071] Figure 6 Schematic diagram of a space-time encoder provided by an embodiment of the present application;

[0072] Figure 7 This is an architecture diagram of a security inspection quality assessment system based on human key point detection provided by an embodiment of the present application;

[0073] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0075] It should be understood that in the description of this application, unless the context clearly requires otherwise, words such as "include", "comprises", and the like throughout the specification and claims should be interpreted as inclusive rather than exclusive or exhaustive; that is, as "including but not limited to".

[0076] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0077] It should be noted that the terms "S1", "S2", etc. are used only for the purpose of describing the steps and do not specifically refer to the order or sequence, nor are they used to limit this application. They are merely for the convenience of describing the method of this application and should not be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0078] As mentioned in the background technology, current personal inspections mainly rely on manual operations by security inspectors. This approach currently faces many challenges, including the following:

[0079] (1) Human factors have a significant impact: Security inspectors have a high workload and long hours of repetitive work can lead to fatigue, which in turn affects the quality of security inspections. Furthermore, different security inspectors have different levels of operational standardization and meticulousness, making it difficult to ensure the consistency of security inspection results.

[0080] (2) Lack of objective evaluation standards: Currently, the operational evaluation of security inspectors mainly relies on manual supervision. There is a lack of objective and quantitative evaluation standards, making it difficult to detect irregular behaviors in the security process in a timely manner.

[0081] To solve the above technical problems, this application provides a standardized security inspection quality assessment method. It uses security inspection video data to detect and track the key points of security inspectors' inspections, and further constructs an evaluation rule to automatically evaluate the normativeness and standardization of security inspectors' personal inspections, and score the security inspectors' personal inspection results, thereby standardizing the process and standards of security inspection personal inspections and effectively improving the safety level of security inspections.

[0082] Example 1

[0083] In order to improve the security level of security inspection, the embodiment of the present application provides a security inspection quality assessment method based on human key point detection, such as Figure 1 As shown, the method disclosed in this embodiment is used to evaluate the quality of personal security inspection, which mainly includes:

[0084] S1. Extract single-frame images and multi-frame images based on security inspection video data.

[0085] Specifically, in the embodiments of this application, security inspection video data primarily refers to video footage captured by cameras installed at security checkpoints, showing security inspectors performing personnel inspections. This video data is a video sequence containing human gestures. This application does not limit the specific type and configuration of the camera; it can be a spherical camera installed on the security gate, a separate camera, or the like.

[0086] In order to fully extract the key point spatial features and key point temporal features that are representative of the key points of the human body in the security inspection video data, in this application, the collected security inspection video data is first converted into a series of frame images, including single-frame images and multi-frame images. This can be achieved in a variety of ways, such as using Python and OpenCV to extract frames from video data files and save them as image formats, or using FFmpeg command line tools. This application does not limit the specific image extraction method. It can be understood that a single-frame image is a frame at a specific time point, and a multi-frame image is all frames within a period of time. In this application, each frame image can also be pre-processed, such as scaling and normalization, for subsequent processing.

[0087] S2. Input a single-frame image and initial spatial position features to at least one preset spatial graph encoder to output key point spatial features.

[0088] For a single-frame image, this application pre-designs a spatial graph encoder to explore the influence and dependency of the key points of the human body contained in each frame of the image on the spatial structure and position, thereby extracting representative key point spatial features. The spatial graph encoder outputs the key point spatial features based on the key point X1 (i.e., the key point position coordinates) and the initial spatial position feature X1′ in the input single-frame image; that is, it can be understood that the initial spatial position feature X1′ in the spatial graph encoder can be transformed into X1′=X1+E SPos Among them, the pre-embedded spatial position encoding matrix E SPos To retain the original spatial position information of all human key points contained in the image frame, the matrix is initialized as a learnable random matrix, and the parameters will be adaptively learned and updated as the network iterates.

[0089] S3. Input multiple frames of images and initial temporal position features to at least one preset temporal graph encoder to output key point temporal features.

[0090] For multi-frame images, this application pre-designs a timing graph encoder to mine the position change relationship of the same key point of the human body in different frame images, thereby extracting representative key point timing features. Specifically, the timing graph encoder outputs the key point timing features based on the key point X2 in the input multi-frame images and the initial timing position feature X2'. It can be understood that the position feature X2' in the timing graph encoder can also be transformed into X2'=X2+ETPos Among them, the pre-embedded temporal position coding matrix E SPos To retain the original temporal position information of each frame of the image; the matrix is initialized as a learnable random matrix, and then adaptively learns and updates the parameters as the network iterates.

[0091] S4. Generate a first predicted key point according to the key point spatial features, the key point temporal features, and a preset spatiotemporal encoder.

[0092] The present application proposes that multiple spatial graph encoders and temporal graph encoders can be set for security inspection video data, so as to further mine the multi-level rich information from shallow to deep layers in the security inspection video data. Specifically, multiple spatial graph encoders are connected in series to obtain the key point spatial features of multiple levels of key points, and multiple temporal graph encoders are connected in series to obtain the key point temporal features of multiple levels of key points. That is, the number and level of spatial graph encoders and temporal graph encoders correspond one to one. Setting a spatial graph encoder requires setting a corresponding temporal graph encoder. Of course, it is also possible to set only one spatial graph encoder and one temporal graph encoder. The present application does not limit the number of specific spatial graph encoders and temporal graph encoders.

[0093] It can be understood that in the scenario where there are multiple spatial map encoders and multiple temporal map encoders, the output of the current spatial map encoder is part of the input features of the spatial map encoder of the next level of the current spatial map encoder, and the output of the current temporal map encoder is the input of the temporal map encoder of the next level of the current temporal map encoder. Therefore, the spatial map encoder with a preamble spatial map encoder no longer needs to extract the first spatial map features for a single frame image, and directly uses the key point spatial features output by the preamble spatial map encoder to replace the first idle map features, that is, the output of the current spatial map encoder is part of the input features of the spatial map encoder of the next level of the current spatial map encoder. The temporal map encoder with a preamble temporal map encoder no longer needs to extract temporal map features for multiple frames of images, and directly uses the key point temporal features output by the preamble spatial map encoder to replace the temporal map features; the key point temporal features generated by the current temporal map encoder replace the temporal map features in the temporal map encoder of the next level of the current temporal map encoder.

[0094] In order to improve the accuracy of the predicted key points of the final output, the spatial features of the output key points of the current spatial graph encoder can be enhanced before being input to the next-level spatial graph encoder, and the temporal features of the output key points of the current temporal graph encoder can be enhanced before being input to the next-level temporal graph encoder. Of course, this application does not limit whether feature enhancement is required. Feature enhancement can be performed or not, which is decided by those skilled in the art.

[0095] In a specific implementation scenario, generating the first predicted key point based on the key point spatial features and key point temporal features and the preset spatiotemporal encoder includes: splicing the key point spatial features output by the spatial graph encoder at the same level and the key point temporal features output by the temporal graph encoder to generate multiple fused key point features; inputting the multiple fused key point features into the spatiotemporal encoder to output the first key point feature. Figure 2 In the multi-level encoder scenario shown, the shallow key point spatial features and shallow key point temporal features output by the shallow spatial graph encoder and the temporal graph encoder are spliced to generate shallow fused key point features; the middle-level key point spatial features and middle-level key point temporal features output by the middle-level spatial graph encoder and the temporal graph encoder are spliced to generate middle-level fused key point features; the high-level key point spatial features and high-level key point temporal features output by the high-level spatial graph encoder and the temporal graph encoder are spliced to generate high-level fused key point features, and then the spliced shallow fused key point features, middle-level fused key point features and high-level fused key point features are input into the spatiotemporal encoder for further processing.

[0096] S5. Determine the target loss and key point coverage based on the first predicted key point and the benchmark key point.

[0097] Reference key points are key points in a reference 3D human pose, defined by those skilled in the art. In the security inspection scenarios disclosed herein, these can be understood as key points that security personnel must inspect under standard security checks. To achieve intelligent detection and tracking of key points on the human body, this application eliminates constraints on key point position loss and key point detection order. Specifically, target loss can be used to represent key point position loss and key point detection order loss.

[0098] Specifically, it can be expressed by the following formula:

[0099]

[0100]

[0101] Among them, I is the number of image frames, J is the number of key points, is the first preset key point, p i (j) is the benchmark key point, represents the key point position loss, represents the key point detection order loss, represents the target loss, and λ is the adjustment parameter of different loss functions.

[0102] Specifically, the key point coverage rate refers to the ratio of the number of reference key points covered (i.e., correctly covered) by the security inspector during the security check to the total number of reference key points, which can be expressed by the following formula: Among them, C represents the key point coverage, N correct Indicates the number of correctly covered benchmark key points, N total Represents the total number of reference keypoints. Correct coverage can be determined by setting an overlap threshold and obtaining the overlap ratio between the area centered on the predicted keypoint and a predetermined area centered on the reference keypoint. If the overlap ratio exceeds the overlap threshold, the reference keypoint is deemed to overlap with the predicted keypoint, indicating that the reference keypoint was covered during the inspection. The overlap threshold is adaptively adjusted by those skilled in the art based on the specific type of security inspection equipment and is not limited in this application.

[0103] It is understandable that in order to further improve the accuracy of the final security inspection quality score, the present application also proposes to update the first predicted key point to generate a second predicted key point, and then determine the target loss and key point coverage based on the second predicted key point and the benchmark key point;

[0104] Among them, the first prediction key is updated to generate a second prediction key point, including: removing the noise of the key point features contained in the single-frame image and the multi-frame image to generate a transition key point feature; generating a second prediction key point based on the transition key point feature and the first prediction key point and the spatiotemporal encoder.

[0105] Preferably, Figure 3 An extended model can be constructed to remove noise from key point features contained in single-frame images and multi-frame images:

[0106] For each key point X in each frame, according to the predetermined probability distribution q(X t |X (t-1) ) adds random noise to it to generate noisy data. Usually, the noise distribution q(X t |X (t-1) ) is defined as a Gaussian distribution, expressed as: where β i Is the noise ratio added at the tth time step, usually a monotonically increasing sequence in the interval [0,1]. I is the identity matrix. In each frame, from the standard normal distribution Noise ∈ is extracted from , and then added to the data according to the following formula: in So, X t It's X (t-1) The result of adding Gaussian noise. Next, the reverse denoising process is used to gradually restore the feature data of the key points. First, the noise is estimated. For each time step t (from t = T to t = 1), the denoising network ∈ θ (X t ,t) to estimate the current data X tNoise in θ (X t ,t)=NN(X t ,e t ,F). The noise estimation network can adopt any of the neural networks such as CNN, ResNet, GAN, etc.; e t is the embedding of time step t, which is used to indicate the current time step. Then, at each time step t, according to the noise estimate ∈ θ (X t ,t) Update data X t , get the denoised data X t-1 The denoising process can be expressed as:

[0107]

[0108] X (t-1) =μ θ (X t ,t)+σ t z

[0109] Among them, α t =1-β t , z is from the standard normal distribution Finally, the key point features output by the diffusion model and the first predicted key point features obtained previously are fused across attention through a spatiotemporal encoder to fully exploit their implicit knowledge to enhance the final second key point features.

[0110] Of course, the above denoising method can also be implemented through generative models such as generative adversarial networks and variational autoencoders. This application will not elaborate on this. The above means belong to conventional technical means in this field, and this application will not elaborate on them here.

[0111] S6. Generate a security inspection quality score based on the effective contact rate and target loss of key points and the key point coverage rate.

[0112] The effective contact rate of key points indicates the ratio of the number of benchmark key points whose contact duration exceeds the set time threshold (i.e., effective contact is achieved) to the total number of benchmark key stores, and is expressed by the following formula: Among them, T represents the effective contact rate of key points, M correct Indicates the number of effectively contacted benchmark key points, M total Represents the total number of benchmark key points. Whether a contact is valid can be determined by setting a time threshold to detect whether the security inspector's contact time with each key point exceeds the time threshold. If so, the value is 1, indicating that the detection is valid, that is, the number of benchmark key points that are effectively touched is increased by one; otherwise, the value is 0, indicating that the detection is invalid.

[0113] Specifically, the security inspection quality score is generated based on the effective contact rate of key points, target loss, and key point coverage, and is calculated by setting an evaluation function, where the evaluation function is expressed as:

[0114]

[0115] Wherein, S represents the security inspection quality score, w1∈[0,1], w2∈[0,1], w3∈[0,1], w1+w2+w3=1, respectively representing the adjustment weights corresponding to each factor, which are set by technical personnel in this field according to actual conditions. For example, w1 is set to 0.8, w2 is set to 0.1, and w3 is set to 0.1. This application does not limit the specific setting scheme.

[0116] It can be understood that the security inspection quality assessment method based on human key point detection disclosed in this application can score the security inspection standardization of security inspectors and provide timely feedback on the security inspectors' operational standardization. Subsequently, a more scientific training plan can be specified based on the security inspection quality score to improve the training efficiency of new employees; in addition, through the analysis of a large amount of commodity data, common types of operational errors can also be analyzed, so as to improve the security inspection process in a targeted manner and improve overall work efficiency.

[0117] This application discloses a security inspection quality assessment method based on human body key point detection. For security inspection video data, an algorithm is used to detect and track the predicted key points of the security inspector's inspection, and then an evaluation function is constructed to automatically evaluate the normativeness and standardization of the security inspector's personal inspection to obtain the final security inspection quality score, thereby achieving the standardization of security inspection personal inspections and effectively improving the security inspection level.

[0118] In a specific implementation scenario, the above step S2 is applied in the spatial encoder, and the key point X1 and the initial spatial position feature X1′ in the single frame image are processed by multi-head self-attention (MHA), layer normalization (LN) and feedforward neural network layer (FFN) to Figure 4 The fusion is performed in the manner shown, specifically including:

[0119] Extract the first spatial graph feature from a single frame image; the first spatial graph feature represents the connection state of the key points from the global space to the local space, and the first spatial graph feature can be expressed as:

[0120]

[0121] F S =concat(F G ,FL );

[0122] X represents the key points of the human body in a single frame image, F G represents the global spatial graph feature, F L represents the local spatial graph feature, F S Represents the first spatial graph feature from global to local formed by all key points of the human body in a frame of image. It is an N×N dimensional normalized symmetric matrix, which is converted based on the connection adjacency matrix between all key points, emphasizing the interconnection between all key points of the human body. I is an N×N dimensional identity matrix that describes the self-connection of the joint points. yes The degree matrix satisfies It consists of an N×N dimensional learnable adjacency matrix whose elements are initially assigned random values and are automatically updated during network training based on learning progress. K represents the local feature categories, and each category represents a different skip connection between two keypoints. and Similarly, the local interconnection between different joints, such as first-order connection, second-order connection, etc., is represented by the local adjacency matrix B between different joints. k Comes from change. and Similarly, is the adjustment matrix. σ(·) represents the nonlinear activation function. W is the weight matrix corresponding to the network when solving the corresponding feature.

[0123] A first processing is performed on the first spatial graph feature to generate a second spatial graph feature, wherein the first processing includes normalization and two-dimensional convolution (Conv2d).

[0124] The first spatial position feature is determined based on the multi-head self-attention mechanism and the initial spatial position feature. Specifically, the position feature X1′ is converted into the attention Q, K and V matrices through layer normalization and different linear layers. The multi-head attention mechanism of the encoder is used to dynamically model the correlation between them, which is expressed as:

[0125] Q=LN(ψ1(X1′)), K=LN(ψ2(X1′)), V=LN(ψ3(X1′));

[0126]

[0127] Then, the first spatial position feature and the initial spatial position feature are fused to generate a second spatial position feature and the second spatial position feature is subjected to a second processing to obtain a third spatial position feature; wherein the second processing includes normalization processing and feedforward neural network processing.

[0128] Perform a dot multiplication operation on the second spatial position feature and the second spatial graph feature to generate a first transition spatial feature; fuse the first transition spatial feature and the second spatial graph feature to generate a second transition spatial feature; fuse the second transition spatial feature, the third spatial position feature and the first spatial graph feature to generate a key point spatial feature. Specifically, the first spatial graph feature F from the global to the local of the joint point is introduced. S , using the layer normalization layer and 1×1 convolution layer to transform it, and then effectively fuse it with the third spatial position feature at the perceptron position. The specific implementation process is expressed as:

[0129] X1 l′ =MHA(X1 l-1 )+X1 l-1 ;

[0130]

[0131] in, represents the cross product, μ and τ are the feature transformation and mapping functions in the feedforward network.

[0132] In a specific implementation scenario, the above step S3 is applied in the spatial encoder, and the temporal graph features of the key points and the initial temporal position features are combined through the multi-head self-attention (MHA), layer normalization (LN) and feedforward neural network (FFN) layers to Figure 5 The method shown is used to fuse the spatial features to obtain representative features, including:

[0133] Extract the time sequence graph features from multiple frames of images. Specifically, this application proposes to construct a time sequence graph adjacency matrix based on the visibility in different image frames, thereby generating the time sequence graph feature F T , further explore the position change relationship of the same human key point in different image frames:

[0134]

[0135] Among them, j represents the number of key points, and Similarly, it describes the visibility of the same keypoint in different image frames.

[0136] Then, the first temporal position feature is determined based on the multi-head attention mechanism and the initial temporal position feature. In the process of determining the first temporal position feature, the input of the softmax function is the temporal graph feature and the attention score. That is, the temporal graph feature and the initial temporal position feature are fused in the attention mechanism of the temporal graph encoder, and the temporal graph feature is used as a related bias term of the initial temporal position feature. By increasing the information involved in the attention calculation, the important semantic information hidden in the spatial and temporal features of all human joints is mined, thereby realizing the adaptive dynamic fusion of different categories of joint feature parts. That is, the initial position feature X2′ is converted into the attention Q, K and V matrices through layer normalization and different linear layers, and the correlation between them is dynamically modeled using the multi-head attention mechanism of the encoder, which is expressed as:

[0137] Q=LN(ψ1(X2′)), K=LN(ψ2(X2′)), V=LN(ψ3(X2′));

[0138]

[0139] The first temporal position feature and the initial temporal position feature are fused to generate the second temporal position feature and the second temporal position feature is subjected to a third processing to generate the third temporal position feature; the third temporal position feature and the second temporal position feature are fused to generate the key point temporal feature. Specifically, the third processing includes normalization and linear layer processing, that is, the output second temporal position feature is transformed by layer normalization and linear layer to generate the third temporal position feature, and then the third temporal position feature is further fused with the second temporal position feature. The network adopts residual connection to facilitate the training of the model and alleviate the problem of gradient disappearance or explosion. The specific implementation process is expressed as:

[0140] X1′=MHA(X2 l-1 )+X2 l-1 ;

[0141] X l =X l′ +μ l (LN(X2 l′ ));

[0142] in, represents the cross product, μ and τ are the feature transformation and mapping functions in the feedforward network.

[0143] In a specific implementation scenario, when the spatiotemporal encoder performs the above step S4, the above input multiple fusion key point features are input to the spatiotemporal encoder to output the first key point feature, such as Figure 6 As shown, specifically including:

[0144] The first transition fusion feature is determined based on the multi-head attention mechanism and the fused key point feature; the first transition fusion feature and the fused key point feature are fused to generate the second transition fusion feature and the second transition fusion feature is normalized to generate the third transition fusion feature; the third transition fusion feature is processed according to the forward feedback neural network to generate the fourth transition fusion feature; the fourth transition fusion feature and the second transition fusion feature are fused to generate the first predicted key point feature. In the multi-head attention mechanism within the spatiotemporal encoder, the fused key point feature input to the spatiotemporal encoder is normalized, and multiple Q, K, and V matrices are extracted. The Q and K matrices are dot-producted and scaled before being input into the softMax function. The output of the softMax function is then dot-producted with the V matrix. Finally, the features output by the multiple heads are concatenated and linearized before being output.

[0145] It can be understood that in the implementation scenario of updating the first prediction key point, the fused key point features input into the spatiotemporal encoder are the fused first prediction key point features and transition key point features; the steps of the spatiotemporal encoder to generate the second prediction key point are similar to the steps of generating the first prediction key point, and this application will not repeat them here.

[0146] Example 2

[0147] Corresponding to the above embodiment 1, the present application also provides a security inspection quality assessment system based on human key point detection, such as Figure 7 As shown, specifically including:

[0148] A pre-processing module 710 is used to extract single-frame images and multi-frame images based on security inspection video data;

[0149] The feature processing module 720 is configured to input a single-frame image and initial spatial position features into at least one preset spatial graph encoder to output keypoint spatial features;

[0150] The feature processing module 720 is further configured to input multiple frames of images and initial temporal position features into at least one preset temporal graph encoder to output key point temporal features;

[0151] The feature processing module 720 is further configured to generate a first predicted key point based on the key point spatial feature, the key point temporal feature, and a preset spatiotemporal encoder;

[0152] a quality assessment module 730 for determining a target loss and a keypoint coverage rate based on the first predicted keypoints and the reference keypoints, wherein the reference keypoints are keypoints in a reference 3D human pose, and the target loss includes a keypoint position loss and a keypoint detection order loss;

[0153] The quality assessment module 730 is further configured to generate a security inspection quality score based on the effective contact rate and target loss of key points and the key point coverage rate.

[0154] In some implementation scenarios, the feature processing module 720 is also used to: update the first predicted key to generate a second predicted key point; determine the target loss and key point coverage based on the second predicted key point and the benchmark key point; wherein, updating the first predicted key to generate the second predicted key point includes: removing the noise of the key point features contained in the single-frame image and the multi-frame image to generate a transition key point feature; generating the second predicted key point based on the transition key point feature and the first predicted key point and the spatiotemporal encoder.

[0155] In some implementation scenarios, the feature processing module 720 is also used to: extract a first spatial graph feature from a single-frame image; perform a first processing on the first spatial graph feature to generate a second spatial graph feature, the first processing including normalization and two-dimensional convolution, wherein the first spatial graph feature represents the connection state of the key point from the global space to the local space; determine the first spatial position feature based on the multi-head self-attention mechanism and the initial spatial position feature; fuse the first spatial position feature and the initial spatial position feature to generate a second spatial position feature and perform a second processing on the second spatial position feature to obtain a third spatial position feature, the second processing including normalization and feedforward neural network processing; perform a dot multiplication operation on the second spatial position feature and the second spatial graph feature to generate a first transition spatial feature; fuse the first transition spatial feature and the second spatial graph feature to generate a second transition spatial feature; fuse the second transition spatial feature, the third spatial position feature and the first spatial graph feature to generate a key point spatial feature.

[0156] In some implementation scenarios, the feature processing module 720 is also used to: extract temporal graph features from multiple frame images; determine a first temporal position feature based on a multi-head attention mechanism and an initial temporal position feature, wherein in the process of determining the first temporal position feature, the input of the softmax function is the temporal graph feature and the attention score; fuse the first temporal position feature and the initial temporal position feature to generate a second temporal position feature and perform a third processing on the second temporal position feature to generate a third temporal position feature, wherein the third processing includes normalization and linear layer processing; fuse the third temporal position feature and the second temporal position feature to generate a key point temporal feature.

[0157] In some implementation scenarios, the feature processing module 720 is also used to: replace the first spatial graph feature in the next-level spatial graph encoder of the current spatial graph encoder according to the key point spatial feature generated by the current spatial graph encoder; replace the timing graph feature in the next-level timing graph encoder of the current timing graph encoder according to the key point timing feature generated by the current timing graph encoder.

[0158] In some implementation scenarios, the feature processing module 720 is also used to: splice the key point spatial features output by the spatial graph encoder at the same level and the key point temporal features output by the temporal graph encoder to generate multiple fused key point features; input multiple fused key point features into the spatiotemporal encoder to output the first key point feature.

[0159] In some implementation scenarios, the feature processing module 720 is also used to: determine a first transition fusion feature based on a multi-head attention mechanism and fused key point features; fuse the first transition fusion feature and the fused key point feature to generate a second transition fusion feature and normalize the second transition fusion feature to generate a third transition fusion feature; process the third transition fusion feature according to a forward feedback neural network to generate a fourth transition fusion feature; and fuse the fourth transition fusion feature and the second transition fusion feature to generate a first predicted key point feature.

[0160] In some implementation scenarios, the quality assessment module 730 is further configured to calculate the security inspection quality score according to a preset formula, where the preset formula is:

[0161]

[0162] Among them, S represents the security inspection quality score, Represents the target loss, w1 represents the first weight matching the target loss, C represents the key point coverage, w2 represents the second weight matching the key point coverage, T represents the key point effective contact rate, and w3 represents the third weight matching the key point effective contact rate.

[0163] Example 3

[0164] Corresponding to all the above embodiments, an embodiment of the present application provides an electronic device, including: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions. When the program instructions are read and executed by the one or more processors, the program instructions perform the following operations:

[0165] Extract single-frame images and multi-frame images based on security inspection video data;

[0166] Inputting a single frame image and initial spatial position features to at least one preset spatial graph encoder to output key point spatial features;

[0167] Input multiple frames of images and initial temporal position features to at least one preset temporal graph encoder to output key point temporal features;

[0168] Generate a first predicted key point according to the key point spatial feature, the key point temporal feature and a preset spatiotemporal encoder;

[0169] Determining a target loss and a keypoint coverage rate based on the first predicted keypoint and the reference keypoint, wherein the reference keypoint is a keypoint in a reference 3D human pose, and the target loss includes a keypoint position loss and a keypoint detection order loss;

[0170] The security inspection quality score is generated based on the effective contact rate and target loss of key points and the key point coverage rate.

[0171] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following operations:

[0172] Updating the first prediction key to generate a second prediction key point;

[0173] Determine the target loss and key point coverage based on the second predicted key point and the benchmark key point;

[0174] The updating of the first prediction key to generate the second prediction key point includes:

[0175] removing noise from key point features contained in single-frame images and multi-frame images to generate transition key point features;

[0176] A second predicted key point is generated according to the transition key point feature, the first predicted key point, and the spatiotemporal encoder.

[0177] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following operations:

[0178] Extracting a first spatial graph feature from a single frame image;

[0179] Performing a first processing on the first spatial graph feature to generate a second spatial graph feature, wherein the first spatial graph feature represents a connection state of the key point from the global space to the local space, the first processing comprising normalization and two-dimensional convolution;

[0180] Determine the first spatial position feature based on the multi-head self-attention mechanism and the initial spatial position feature;

[0181] fusing the first spatial position feature and the initial spatial position feature to generate a second spatial position feature and performing a second processing on the second spatial position feature to obtain a third spatial position feature, wherein the second processing includes normalization processing and feedforward neural network processing;

[0182] Performing a dot product operation on the second spatial position feature and the second spatial graph feature to generate a first transition spatial feature;

[0183] fusing the first transition space feature and the second spatial graph feature to generate a second transition space feature;

[0184] The second transition space feature, the third space position feature and the first space graph feature are fused to generate the key point space feature.

[0185] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following operations:

[0186] Extracting timing graph features from multiple frames of images;

[0187] Determine a first temporal position feature based on the multi-head attention mechanism and the initial temporal position feature, wherein in the process of determining the first temporal position feature, the input of the softmax function is the temporal graph feature and the attention score;

[0188] fusing the first temporal position feature and the initial temporal position feature to generate a second temporal position feature and performing a third processing on the second temporal position feature to generate a third temporal position feature, wherein the third processing includes normalization and linear layer processing;

[0189] The third temporal position feature and the second temporal position feature are fused to generate a key point temporal feature.

[0190] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following operations:

[0191] replacing the first spatial graph feature in the next-level spatial graph encoder of the current spatial graph encoder according to the key point spatial feature generated by the current spatial graph encoder;

[0192] The key point timing features generated by the current timing graph encoder are used to replace the timing graph features in the next level timing graph encoder of the current timing graph encoder.

[0193] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following operations:

[0194] The key point spatial features output by the spatial graph encoder at the same level and the key point temporal features output by the temporal graph encoder are concatenated to generate multiple fused key point features;

[0195] Input multiple fused keypoint features to the spatiotemporal encoder to output the first keypoint feature.

[0196] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following operations:

[0197] Determine the first transition fusion feature based on the multi-head attention mechanism and the fusion key point features;

[0198] fusing the first transition fusion feature and the fusion key point feature to generate a second transition fusion feature and normalizing the second transition fusion feature to generate a third transition fusion feature;

[0199] processing the third transition fusion feature according to a feedforward neural network to generate a fourth transition fusion feature;

[0200] The fourth transition fused feature and the second transition fused feature are fused to generate a first predicted key point feature.

[0201] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following operations:

[0202] The security inspection quality score is calculated according to the preset formula, where the preset formula is:

[0203]

[0204] Among them, S represents the security inspection quality score, Represents the target loss, w1 represents the first weight matching the target loss, C represents the key point coverage, w2 represents the second weight matching the key point coverage, T represents the key point effective contact rate, and w3 represents the third weight matching the key point effective contact rate.

[0205] in, Figure 8 The electronic device architecture is shown as an example, and may include a processor 810, a video display adapter 811, a disk drive 812, an input / output interface 813, a network interface 814, and a memory 820. The processor 810, the video display adapter 811, the disk drive 812, the input / output interface 813, the network interface 814, and the memory 820 may be communicatively connected via a bus 830.

[0206] Among them, the processor 810 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in this application.

[0207] The memory 820 can be implemented in the form of ROM (Read Only Memory, programmable memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, etc. The memory 820 can store an operating system 821 for controlling the execution of the electronic device 800 and a basic input and output system (BIOS) 822 for controlling the low-level operations of the electronic device 800. In addition, a data storage management system 823 and the like can also be stored. In short, when the technical solution provided in this application is implemented by software or firmware, the relevant program code is stored in the memory 820 and is called and executed by the processor 810.

[0208] The input / output interface 813 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0209] The network interface 814 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0210] The bus 830 comprises a pathway for transmitting information between the various components of the device (eg, the processor 810 , the video display adapter 811 , the disk drive 812 , the input / output interface 813 , the network interface 814 , and the memory 820 ).

[0211] In addition, the electronic device 800 can also obtain information on specific collection conditions from the virtual resource object collection condition information database for use in condition judgment, etc.

[0212] It should be noted that although the above device only shows the processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, memory 820, bus 830, etc., in the specific implementation process, the device may also include other components necessary for normal execution. In addition, it will be understood by those skilled in the art that the above device may also include only the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0213] Through the description of the above implementation methods, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a cloud service terminal, or a network device, etc.) to execute the methods of each embodiment of the present application or certain parts of the embodiments.

[0214] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without making any creative efforts.

[0215] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A security inspection quality assessment method based on human key point detection, characterized in that: The method comprises: Extract single-frame images and multi-frame images based on security inspection video data; Inputting the single frame image and the initial spatial position features into at least one preset spatial graph encoder to output key point spatial features; Inputting the multiple frames of images and the initial temporal position features into at least one preset temporal graph encoder to output key point temporal features; Generate a first predicted key point according to the key point spatial feature and the key point temporal feature and a preset spatiotemporal encoder; Determining a target loss and a key point coverage rate based on the first predicted key point and the reference key point, wherein the reference key point is a key point in a reference 3D human posture, and the target loss includes a key point position loss and a key point detection order loss; A security inspection quality score is generated according to the effective contact rate of key points, the target loss, and the key point coverage rate.

2. The method according to claim 1, characterized in that Before determining the target loss and the key point coverage, the method further includes: updating the first prediction key to generate a second prediction key point; Determining target loss and key point coverage based on the second predicted key points and the benchmark key points; The updating of the first prediction key to generate a second prediction key point includes: removing noise from key point features contained in the single-frame image and the multiple-frame images to generate transition key point features; A second predicted key point is generated according to the transition key point feature, the first predicted key point, and the spatiotemporal encoder.

3. The method according to claim 1, characterized in that Applied to the spatial encoder, the inputting of the single-frame image and the initial spatial position features to at least one preset spatial graph encoder to output keypoint spatial features includes: Extracting a first spatial graph feature from a single frame image; Performing a first processing on the first spatial graph feature to generate a second spatial graph feature, wherein the first spatial graph feature represents a connection state of a key point from a global space to a local space, the first processing comprising normalization and two-dimensional convolution; Determine a first spatial position feature according to the multi-head self-attention mechanism and the initial spatial position feature; fusing the first spatial position feature and the initial spatial position feature to generate a second spatial position feature and performing a second processing on the second spatial position feature to obtain a third spatial position feature, wherein the second processing includes normalization processing and feedforward neural network processing; Performing a dot product operation on the second spatial position feature and the second spatial graph feature to generate a first transition spatial feature; fusing the first transition space feature and the second spatial graph feature to generate a second transition space feature; The second transition space feature, the third space position feature, and the first space graph feature are fused to generate a key point space feature.

4. The method according to claim 3, characterized in that Applied to a timing graph encoder, the step of extracting timing graph features and initial timing position features from the multi-frame images and inputting them into at least one preset timing graph encoder to generate key point time features includes: Extracting timing graph features from multiple frames of images; Determining a first temporal position feature according to the multi-head attention mechanism and the initial temporal position feature, wherein in the process of determining the first temporal position feature, the input of the softmax function is the temporal graph feature and the attention score; fusing the first temporal position feature and the initial temporal position feature to generate a second temporal position feature, and performing a third processing on the second temporal position feature to generate a third temporal position feature, wherein the third processing includes normalization and linear layer processing; The third temporal position feature and the second temporal position feature are fused to generate a key point temporal feature.

5. The method according to claim 4, wherein the levels of the spatial image encoder and the temporal image encoder are matched one-to-one, wherein: The method comprises: Replacing the first spatial graph feature in the next-level spatial graph encoder of the current spatial graph encoder according to the key point spatial feature generated by the current spatial graph encoder; According to the key point timing features generated by the current timing diagram encoder, the timing diagram features in the next-level timing diagram encoder of the current timing diagram encoder are replaced.

6. The method according to claim 5, characterized in that Generating a first predicted key point according to the key point spatial feature and the key point temporal feature and a preset spatiotemporal encoder includes: The key point spatial features output by the spatial graph encoder at the same level and the key point temporal features output by the temporal graph encoder are concatenated to generate multiple fused key point features; Input a plurality of fused keypoint features to the spatiotemporal encoder to output a first keypoint feature.

7. The method according to claim 1, characterized in that Applied to a spatiotemporal encoder, the inputting of multiple fused key point features to the spatiotemporal encoder to output a first key point feature includes: Determine a first transition fusion feature according to the multi-head attention mechanism and the fusion key point feature; fusing the first transition fusion feature and the fusion key point feature to generate a second transition fusion feature, and performing normalization processing on the second transition fusion feature to generate a third transition fusion feature; Processing the third transition fusion feature according to a feedforward neural network to generate a fourth transition fusion feature; The fourth transition fused feature and the second transition fused feature are fused to generate a first predicted key point feature.

8. The method according to claim 1, characterized in that Generating a security inspection quality score according to the effective contact rate of key points, the target loss, and the key point coverage rate includes: The security inspection quality score is calculated according to a preset formula, where the preset formula is: Among them, S represents the security inspection quality score, Represents the target loss, w1 represents the first weight matching the target loss, C represents the key point coverage, w2 represents the second weight matching the key point coverage, T represents the key point effective contact rate, and w3 represents the third weight matching the key point effective contact rate.

9. A security inspection quality assessment system based on human key point detection, characterized in that: The system comprises: A preprocessing module is used to extract single-frame images and multi-frame images based on security inspection video data; a feature processing module, configured to input the single-frame image and the initial spatial position feature into at least one preset spatial graph encoder to output keypoint spatial features; The feature processing module is further configured to input the multiple frames of images and the initial temporal position features into at least one preset temporal graph encoder to output key point temporal features; The feature processing module is further configured to generate a first predicted key point based on the key point spatial feature and the key point temporal feature and a preset spatiotemporal encoder; a quality assessment module, configured to determine a target loss and a keypoint coverage rate based on the first predicted keypoint and a reference keypoint, wherein the reference keypoint is a keypoint in a reference 3D human pose, and the target loss includes a keypoint position loss and a keypoint detection order loss; The quality assessment module is further configured to generate a security inspection quality score based on the effective contact rate of key points, the target loss, and the key point coverage rate.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; And a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, executes the security inspection quality assessment method based on human key point detection as described in any one of claims 1-8.

Images