A streaming mechanical blind spot liveness detection system

Abstract

The present invention discloses a streaming mechanical visual field blind spot liveness detection system, including a data acquisition module, a data processing and fusion module, a liveness detection module, a behavior analysis module, a single-stage integrated detection module and an exception handling module. The present invention belongs to the field of visual field blind spot detection technology, specifically a streaming mechanical visual field blind spot liveness detection system, the present invention adopts two task sub-models for single-stage integrated detection, improves the real-time performance of detection by respectively improving the liveness detection and behavior analysis models as sub-models, and performs integrated training to obtain a single-stage integrated detection model; adopts a lightweight visual detection model combined with dilated convolution and adaptive gradient improvement for liveness detection, improves the overall rate and real-time performance of subsequent integrated detection; adopts a fine-grained data efficient image transformer model combined with local selection optimization for behavior analysis, improves the fine-grained degree of the behavior analysis process.

Description

Technical Field

[0001] The invention belongs to the technical field of visual field blind spot detection, and in particular relates to a flow-type mechanical visual field blind spot living body detection system. Background Art

[0002] The streaming mechanical field of view blind spot liveness detection system is an intelligent system for real-time monitoring and identification of moving objects in blind spots, and is usually used in security monitoring and automation equipment. The system uses sensors and image processing technology to automatically detect living targets (such as people or animals) in the field of view, and promptly issues alarms or takes corresponding measures to prevent accidents or ensure safety. This system effectively makes up for the lack of monitoring of traditional monitoring equipment in blind spots and improves the safety and responsiveness of smart places.

[0003] However, in the existing streaming mechanical blind spot liveness detection system, there is a technical problem that the existing method constructs multiple multi-stage detection models, and realizes a general blind spot liveness detection task through the gradual detection of multiple models. However, the streaming mechanical blind spot liveness detection has extremely high requirements for real-time performance, and multi-stage detection is likely to delay the detection time, and the accuracy is inevitably reduced due to the combination of multiple stages; in the existing liveness detection methods, there is a technical problem that the traditional methods do not perform well in the face of blind spot detection details, and the blind spot liveness detection must accurately detect the live body and mark it before performing subsequent tasks; in the existing behavior analysis methods, there is a technical problem that the behavior analysis for liveness detection is a fine-grained detection and classification process, and the classification granularity of the existing methods is insufficient and it is easy to ignore the potential characteristics of abnormal behavior after liveness detection, thereby reducing the availability of the overall method. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a streaming mechanical blind spot liveness detection system. In the existing streaming mechanical blind spot liveness detection system, there is an existing method that constructs multiple stage detection models, and realizes a general blind spot liveness detection task through the gradual detection of multiple models. However, the streaming mechanical blind spot liveness detection has extremely high requirements for real-time performance, and multi-stage detection is likely to delay the detection time, and the accuracy is inevitably reduced due to the combination of multiple stages. Technical problems, this scheme creatively adopts sub-models of two tasks for single-stage integrated detection, and improves the liveness detection and behavior analysis models as sub-models respectively, and performs integrated training to obtain a single-stage integrated detection model, thereby improving the real-time performance of the detection, and further providing a guarantee for the availability of the streaming mechanical blind spot liveness detection system; in the existing liveness detection methods, there is a traditional method that performs poorly in the detection details of the blind spot, and the blind spot liveness detection must accurately detect the liveness. In order to solve the technical problem that the live body must be detected and marked before subsequent tasks can be carried out, this solution creatively adopts a lightweight visual detection model combined with dilated convolution and adaptive gradient improvement for liveness detection. On the one hand, the performance of the standard model is improved by dilated convolution and adaptive gradient algorithm, and on the other hand, the overall computational efficiency of the model is further optimized by introducing Gaussian error linear unit, thereby improving the overall rate and real-time performance of subsequent integrated detection; in view of the technical problem that in the existing behavior analysis methods, the behavior analysis for liveness detection is a fine-grained detection and classification process, and the classification fine-grainedness of the existing methods is insufficient and it is easy to ignore the potential characteristics of abnormal behavior after liveness detection, thereby reducing the availability of the overall method, this solution creatively adopts a fine-grained data efficient image transformer model combined with local selection optimization for behavior analysis. By introducing local selection sub-blocks and constructing a knowledge distillation model structure, the fine-grained degree of the behavior analysis process is improved, and feasible technical support is also provided for subsequent abnormal processing and risk alarm.

[0005] The technical solution adopted by the present invention is as follows: A streaming mechanical visual field blind spot liveness detection system provided by the present invention comprises a data acquisition module, a data processing fusion module, a liveness detection module, a behavior analysis module, a single-stage integrated detection module and an abnormality processing module;

[0006] The data acquisition module is used to collect the raw data required for liveness detection in the blind area of ​​the flow machine vision, obtain the raw data for liveness detection in the blind area of ​​the vision through data acquisition, and send the raw data for liveness detection in the blind area of ​​the vision to the data processing fusion module and the single-stage integrated detection module;

[0007] The data processing and fusion module is used to enhance and optimize the original data, obtain an optimized liveness detection data set through data processing and fusion, and send the optimized liveness detection data set to the liveness detection module and the behavior analysis module;

[0008] The liveness detection module is used to improve the target detection model and serve as a liveness target detection sub-model of the final model. Through liveness detection, a liveness detection output and a liveness detection sub-model are obtained, and the liveness detection output is sent to the behavior analysis module, and the liveness detection sub-model is sent to the single-stage integrated detection module;

[0009] The behavior analysis module is used to construct a fine-grained model and serve as a fine-grained classification sub-model of the living behavior of the final model, obtain a behavior analysis output and a behavior analysis sub-model through behavior analysis, and send the behavior analysis sub-model to the single-stage integrated detection module;

[0010] The single-stage integrated detection module is used to integrate the liveness detection and behavior analysis sub-models as a single-stage detection model, obtain the liveness detection reference data of the blind area of ​​the flow mechanical field of view through the single-stage integrated detection, and send the liveness detection reference data of the blind area of ​​the flow mechanical field of view to the abnormality processing module;

[0011] The exception handling module is used to provide alarm feedback according to the detection results, and to perform real-time monitoring and exception alarm through exception handling.

[0012] Furthermore, the data collection is specifically to obtain raw data of blind spot detection in visual field by setting a streaming mechanical visual field blind spot sensor to perform real-time data collection and preliminary preprocessing in multiple environments;

[0013] The raw data of liveness detection in the blind area of ​​vision specifically includes RGB camera data, infrared camera data and acoustic signal auxiliary data;

[0014] The multi-environment real-time data collection is specifically carried out by manually observing, dynamically adjusting the viewing angle of the sensor camera, and maximizing the coverage of the blind spots of the field of vision;

[0015] The preliminary preprocessing specifically includes a median filtering denoising operation and an infrared image normalization operation.

[0016] Furthermore, the data processing and fusion is specifically to extract features from the camera data and acoustic signal data in the raw data of the blind spot liveness detection, and perform multimodal data feature fusion to obtain an optimized liveness detection data set, and the specific steps include: camera feature extraction, acoustic signal feature extraction and feature fusion processing;

[0017] The camera feature extraction specifically uses a standard convolutional neural network to extract features of RGB camera data and infrared camera data in the raw data of the blind spot living body detection to obtain camera feature data;

[0018] The acoustic signal feature extraction is specifically to extract the frequency and amplitude features of the acoustic signal data through spectrum analysis, perform acoustic signal feature extraction, and obtain audio feature data;

[0019] The feature fusion processing specifically comprises: using a weighted average method to fuse the camera feature data and the audio feature data with feature vectors, calculating the comprehensive feature data, and obtaining an optimized liveness detection data set through data set integration;

[0020] The optimized liveness detection data set specifically includes original camera data, camera feature data and comprehensive feature data;

[0021] The original camera data specifically refers to the RGB camera data and infrared camera data in the blind spot liveness detection original data;

[0022] The calculation formula for obtaining the comprehensive characteristic data is:

[0023] ;

[0024] In the formula, F fused is the comprehensive feature data, is the feature weight, F I is the camera feature data, F S It is audio feature data.

[0025] Furthermore, the liveness detection is specifically performed based on the original camera data and comprehensive feature data in the optimized liveness detection data set, using a lightweight visual detection model combined with dilated convolution and adaptive gradient improvement to obtain a liveness detection output and a liveness detection sub-model;

[0026] The lightweight visual detection model combining dilated convolution and adaptive gradient improvement specifically includes a basic input layer, an improved dilated convolution layer, a standard convolution layer, a spatial pyramid fusion layer, an upsampling pooling layer, a fully connected layer, an adaptive gradient improvement operator and an output layer;

[0027] The step of performing liveness detection by using a lightweight visual detection model combining dilated convolution and adaptive gradient improvement to obtain liveness detection output and a liveness detection sub-model specifically includes: constructing a standard model structure, dilated convolution improvement, adaptive gradient improvement, and liveness detection model optimization training;

[0028] The constructing of the standard model structure specifically involves constructing a standard lightweight visual detection model including a basic input layer, a standard convolutional layer, a spatial pyramid fusion layer, an upsampling pooling layer, a fully connected layer, and an output layer;

[0029] The standard lightweight visual detection model specifically refers to the YOLOv8s model;

[0030] The spatial pyramid fusion layer is used for multi-scale feature fusion; the upsampling pooling layer is used for restoring the feature map size and merging the feature maps;

[0031] The dilated convolution improvement specifically replaces the standard convolution layer of the standard lightweight visual detection model with an improved dilated convolution layer that is a combination of a dilated convolution layer and a standard convolution layer, and obtains a dilated improved lightweight visual detection model by constructing the improved dilated convolution layer, and performs feature depth extraction to obtain deep living body detection feature data;

[0032] The adaptive gradient improvement is specifically to update the parameters of the expanded and improved lightweight visual detection model by introducing adaptive parameters to obtain the expanded parameter improved lightweight visual detection model;

[0033] Loss function, specifically using binary cross entropy loss function;

[0034] The optimization training of the liveness detection model is specifically to construct an expanded and improved lightweight visual detection model and adaptive gradient improvement, and use a Gaussian error linear unit to replace the standard nonlinear activation function as the activation function of the model to perform optimization training of the liveness detection model to obtain the liveness detection sub-model Model C1 , and by using the liveness detection sub-model Model C1 , obtaining a liveness detection output according to the original camera data in the optimized liveness detection data set;

[0035] The calculation formula of the Gaussian error linear unit is:

[0036] ;

[0037] Where GELU(·) is the Gaussian error linear unit function, It is the input data to be activated, which is used to represent the feature map data after the improved dilated convolution layer, standard convolution layer, spatial pyramid fusion layer, upsampling pooling layer and fully connected layer.

[0038] Further, the behavior analysis is specifically performed based on the optimized liveness detection data set, using a fine-grained data efficient image transformer model combined with local selection optimization to obtain a behavior analysis output and a behavior analysis sub-model;

[0039] The fine-grained data-efficient image transformer model combined with local selection optimization specifically includes a teacher subnet, a backbone subnet, and a local selection optimization sub-block;

[0040] The steps of using the fine-grained data efficient image transformer model combined with local selection optimization to perform behavior analysis and obtain behavior analysis output and behavior analysis submodel include: constructing the basic structure of the knowledge distillation model, constructing the backbone subnet, constructing the teacher subnet, constructing the local selection optimization subblock and optimizing the behavior analysis model training;

[0041] The constructing of the basic structure of the knowledge distillation model specifically includes constructing the basic structure of the knowledge distillation model including a student subnet and a teacher subnet, and using the backbone subnet as the student subnet;

[0042] The constructing of the backbone subnet is specifically to construct a feature extraction subnet based on a standard fine-grained data efficient image transformer model, wherein the standard fine-grained data efficient image transformer model specifically refers to a convolution transformer model combining a multi-head self-attention mechanism and a multi-layer perceptron, and by performing multi-head self-attention feature capture and multi-layer perceptron feature capture, a behavior analysis feature extraction is performed to obtain the backbone subnet output feature data;

[0043] The constructing of the teacher subnet specifically involves constructing a standard visual transformer model as the teacher subnet to obtain the teacher subnet feature data;

[0044] The constructing of the local selection optimization sub-block specifically involves constructing the local selection optimization sub-block by combining the attention weights in the multi-head self-attention feature capture, performing feature map optimization, and obtaining fine-grained behavior analysis feature data;

[0045] The behavior analysis model optimization training is specifically to optimize the behavior analysis model through the basic structure of the knowledge distillation model, the backbone subnet, the teacher subnet and the local selection optimization sub-block, combined with the model loss function to obtain the behavior analysis sub-model Model C2 , and by using the behavior analysis sub-model Model C2 , get the behavior analysis output;

[0046] The model loss function specifically refers to a combination of a cross entropy loss function and a contrast loss function;

[0047] The contrast loss function specifically includes a class loss function and a distillation feature loss function.

[0048] Furthermore, the single-stage integrated detection is specifically to obtain the liveness detection sub-model and the behavior analysis sub-model, perform model integration and training, obtain a single-stage integrated detection model, and perform single-stage blind spot liveness detection based on the blind spot liveness detection raw data by using the single-stage integrated detection model to obtain the streaming mechanical blind spot liveness detection reference data;

[0049] The liveness detection reference data of the blind spot of the streaming mechanical field of view specifically includes liveness detection mark data, behavior analysis type reference data and abnormal behavior reference data;

[0050] The behavior analysis type reference data specifically includes abnormal movement, sudden stop, long-term residence, living body fall and multi-person gathering;

[0051] The abnormal behavior reference data specifically refers to the risk quantification value of the behavior analysis type reference data.

[0052] Furthermore, the abnormal processing is specifically based on the reference data of liveness detection in the blind spot of the flow mechanical field of view, and setting a risk alarm quantification index. When the risk quantification value in the abnormal behavior reference data is higher than the risk alarm quantification index, an abnormal alarm is issued.

[0053] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0054] (1) In the existing streaming mechanical blind spot liveness detection system, there are existing methods that construct multiple multi-stage detection models, and implement a general blind spot liveness detection task through the gradual detection of multiple models. However, streaming mechanical blind spot liveness detection has extremely high requirements for real-time performance, and multi-stage detection is likely to delay the detection timing, and the accuracy is inevitably reduced due to the combination of multiple stages. This solution creatively uses sub-models of two tasks for single-stage integrated detection. By improving the liveness detection and behavior analysis models as sub-models respectively, and performing integrated training to obtain a single-stage integrated detection model, the real-time performance of the detection is improved, and the availability of the streaming mechanical blind spot liveness detection system is further guaranteed;

[0055] (2) In view of the technical problem that the traditional methods in existing liveness detection methods have poor performance in detecting details in blind spots, and liveness detection in blind spots must accurately detect liveness and mark it before subsequent tasks can be carried out, this solution creatively adopts a lightweight visual detection model that combines dilated convolution and adaptive gradient improvement for liveness detection. On the one hand, the performance of the standard model is improved by dilated convolution and adaptive gradient algorithm, and on the other hand, the overall computational efficiency of the model is further optimized by introducing Gaussian error linear unit, thereby improving the overall rate and real-time performance of subsequent integrated detection;

[0056] (3) In the existing behavior analysis methods, there is a technical problem that the behavior analysis for liveness detection is a fine-grained detection and classification process, but the classification granularity of the existing methods is insufficient and it is easy to ignore the potential characteristics of abnormal behavior after liveness detection, thereby reducing the usability of the overall method. This solution creatively adopts a fine-grained data-efficient image transformer model combined with local selection optimization to perform behavior analysis. By introducing local selection sub-blocks and constructing a knowledge distillation model structure, the fine-grained level of the behavior analysis process is improved, and feasible technical support is also provided for subsequent abnormal processing and risk alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the structure of a streaming mechanical visual field blind spot living body detection system provided by the present invention;

[0058] Figure 2 A flowchart of the steps performed by the data processing fusion module;

[0059] Figure 3 A schematic diagram of the flow of steps performed by the living body detection module;

[0060] Figure 4 A flowchart of the steps performed by the behavior analysis module;

[0061] Figure 5 This is a schematic diagram of the quantitative indicators of abnormal alarm risk in the abnormal handling module.

[0062] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0064] Example 1, see Figure 1 ,The technical solution adopted by the present invention is as follows: A streaming mechanical visual field blind spot liveness detection system provided by the present invention includes a data acquisition module, a data processing fusion module, a liveness detection module, a behavior analysis module, a single-stage integrated detection module and an abnormality processing module;

[0065] The data acquisition module is used to collect the raw data required for liveness detection in the blind area of ​​the flow machine vision, obtain the raw data for liveness detection in the blind area of ​​the vision through data acquisition, and send the raw data for liveness detection in the blind area of ​​the vision to the data processing fusion module and the single-stage integrated detection module;

[0066] The data processing and fusion module is used to enhance and optimize the original data, obtain an optimized liveness detection data set through data processing and fusion, and send the optimized liveness detection data set to the liveness detection module and the behavior analysis module;

[0067] The liveness detection module is used to improve the target detection model and serve as a liveness target detection sub-model of the final model. Through liveness detection, a liveness detection output and a liveness detection sub-model are obtained, and the liveness detection output is sent to the behavior analysis module, and the liveness detection sub-model is sent to the single-stage integrated detection module;

[0068] The behavior analysis module is used to construct a fine-grained model and serve as a fine-grained classification sub-model of the living behavior of the final model, obtain a behavior analysis output and a behavior analysis sub-model through behavior analysis, and send the behavior analysis sub-model to the single-stage integrated detection module;

[0069] The single-stage integrated detection module is used to integrate the liveness detection and behavior analysis sub-models as a single-stage detection model, obtain the liveness detection reference data of the blind area of ​​the flow mechanical field of view through the single-stage integrated detection, and send the liveness detection reference data of the blind area of ​​the flow mechanical field of view to the abnormality processing module;

[0070] The exception handling module is used to provide alarm feedback according to the detection results, and to perform real-time monitoring and exception alarm through exception handling.

[0071] Embodiment 2, this embodiment is based on the above embodiment, see Figure 1 , the data collection is specifically to obtain raw data of live body detection in the blind area of ​​the visual field by setting a streaming mechanical visual field blind area sensor to perform real-time data collection and preliminary preprocessing in multiple environments;

[0072] The raw data of liveness detection in the blind area of ​​vision specifically includes RGB camera data, infrared camera data and acoustic signal auxiliary data;

[0073] The multi-environment real-time data collection is specifically carried out by manually observing, dynamically adjusting the viewing angle of the sensor camera, and maximizing the coverage of the blind spots of the field of vision;

[0074] The preliminary preprocessing specifically includes a median filtering denoising operation and an infrared image normalization operation.

[0075] Embodiment 3: This embodiment is based on the above embodiment. The data processing fusion is specifically to extract features from the camera data and acoustic signal data in the raw data of the blind spot liveness detection, and perform multimodal data feature fusion to obtain an optimized liveness detection data set. The specific steps include: camera feature extraction, acoustic signal feature extraction and feature fusion processing;

[0076] The camera feature extraction specifically uses a standard convolutional neural network to extract features of RGB camera data and infrared camera data in the raw data of the blind spot living body detection to obtain camera feature data;

[0077] The acoustic signal feature extraction is specifically to extract the frequency and amplitude features of the acoustic signal data through spectrum analysis, perform acoustic signal feature extraction, and obtain audio feature data;

[0078] The feature fusion processing specifically comprises: using a weighted average method to fuse the camera feature data and the audio feature data with feature vectors, calculating the comprehensive feature data, and obtaining an optimized liveness detection data set through data set integration;

[0079] The optimized liveness detection data set specifically includes original camera data, camera feature data and comprehensive feature data;

[0080] The original camera data specifically refers to the RGB camera data and infrared camera data in the blind spot liveness detection original data;

[0081] The calculation formula for obtaining the comprehensive characteristic data is:

[0082] ;

[0083] In the formula, F fused is the comprehensive feature data, is the feature weight, F I is the camera feature data, F S It is audio feature data.

[0084] Embodiment 4, this embodiment is based on the above embodiment, see Figure 3 The liveness detection is specifically performed based on the original camera data and comprehensive feature data in the optimized liveness detection data set, using a lightweight visual detection model combined with dilated convolution and adaptive gradient improvement to obtain a liveness detection output and a liveness detection sub-model;

[0085] The lightweight visual detection model combining dilated convolution and adaptive gradient improvement specifically includes a basic input layer, an improved dilated convolution layer, a standard convolution layer, a spatial pyramid fusion layer, an upsampling pooling layer, a fully connected layer, an adaptive gradient improvement operator and an output layer;

[0086] The step of performing liveness detection by using a lightweight visual detection model combining dilated convolution and adaptive gradient improvement to obtain liveness detection output and a liveness detection sub-model specifically includes: constructing a standard model structure, dilated convolution improvement, adaptive gradient improvement, and liveness detection model optimization training;

[0087] The constructing of the standard model structure specifically involves constructing a standard lightweight visual detection model including a basic input layer, a standard convolutional layer, a spatial pyramid fusion layer, an upsampling pooling layer, a fully connected layer, and an output layer;

[0088] The standard lightweight visual detection model specifically refers to the YOLOv8s model;

[0089] The spatial pyramid fusion layer is used for multi-scale feature fusion; the upsampling pooling layer is used for restoring the feature map size and merging the feature maps;

[0090] The dilated convolution improvement specifically replaces the standard convolution layer of the standard lightweight visual detection model with an improved dilated convolution layer that is a combination of a dilated convolution layer and a standard convolution layer, and obtains a dilated improved lightweight visual detection model by constructing the improved dilated convolution layer, and performs feature depth extraction to obtain deep living body detection feature data;

[0091] The calculation formula for the improved dilated convolution layer to extract feature depth is:

[0092] ;

[0093] Where y(·) is the feature depth extraction operation function, K is the total number of convolution kernel levels of the dilated convolution layer, k is the convolution kernel level index, w(k) is the convolution kernel weight of the kth level, x(·) is the input feature map, which is used to represent the feature data in the comprehensive feature data, p is the input feature map position information index, which is used to represent the specific feature value at position p in the input feature map, and r is the expansion rate, which is specifically 1.8;

[0094] The adaptive gradient improvement is specifically to introduce adaptive parameters to expand and improve the parameters of the lightweight visual detection model to update the parameters, so as to obtain the expanded parameter improved lightweight visual detection model, and the calculation formula is:

[0095] ;

[0096] In the formula, is the updated model parameter group, t is the iteration index, is the model parameter group before updating, is the initial model learning rate parameter, G tis the sum of squared gradients from the start of training to the tth iteration, are the squared gradient and the bias term, is the loss function at the tth iteration relative to the model parameter set The gradient of

[0097] The loss function specifically adopts a binary cross entropy loss function;

[0098] The optimization training of the liveness detection model is specifically to construct an expanded and improved lightweight visual detection model and adaptive gradient improvement, and use a Gaussian error linear unit to replace the standard nonlinear activation function as the activation function of the model to perform optimization training of the liveness detection model to obtain the liveness detection sub-model Model C1 , and by using the living body detection sub-model Model C1 , obtaining a liveness detection output according to the original camera data in the optimized liveness detection data set;

[0099] The calculation formula of the Gaussian error linear unit is:

[0100] ;

[0101] Where GELU(·) is the Gaussian error linear unit function, It is the input data to be activated, which is used to represent the feature map data after the improved dilated convolution layer, standard convolution layer, spatial pyramid fusion layer, upsampling pooling layer and fully connected layer.

[0102] By performing the above operations, in order to solve the technical problem that in the existing liveness detection methods, traditional methods have poor performance in detecting details in blind spots of the field of vision, and liveness detection in blind spots must accurately detect liveness and mark it before performing subsequent tasks, this solution creatively adopts a lightweight visual detection model that combines dilated convolution and adaptive gradient improvement for liveness detection. On the one hand, the performance of the standard model is improved by dilated convolution and adaptive gradient algorithms, and on the other hand, the overall computational efficiency of the model is further optimized by introducing Gaussian error linear units, thereby improving the overall rate and real-time performance of subsequent integrated detection.

[0103] Embodiment 5, this embodiment is based on the above embodiment, see Figure 4 The behavior analysis is specifically performed based on the optimized liveness detection data set, using a fine-grained data efficient image transformer model combined with local selection optimization to obtain a behavior analysis output and a behavior analysis sub-model;

[0104] The fine-grained data-efficient image transformer model combined with local selection optimization specifically includes a teacher subnet, a backbone subnet, and a local selection optimization sub-block;

[0105] The steps of using the fine-grained data efficient image transformer model combined with local selection optimization to perform behavior analysis and obtain behavior analysis output and behavior analysis submodel include: constructing the basic structure of the knowledge distillation model, constructing the backbone subnet, constructing the teacher subnet, constructing the local selection optimization subblock and optimizing the behavior analysis model training;

[0106] The constructing of the basic structure of the knowledge distillation model specifically includes constructing the basic structure of the knowledge distillation model including a student subnet and a teacher subnet, and using the backbone subnet as the student subnet;

[0107] The constructing of the backbone subnet is specifically to construct a feature extraction subnet based on a standard fine-grained data efficient image transformer model, wherein the standard fine-grained data efficient image transformer model specifically refers to a convolution transformer model combining a multi-head self-attention mechanism and a multi-layer perceptron, and by performing multi-head self-attention feature capture and multi-layer perceptron feature capture, a behavior analysis feature extraction is performed to obtain the backbone subnet output feature data;

[0108] The calculation formula for multi-head self-attention feature capture and multi-layer perception feature capture is:

[0109] ;

[0110] In the formula, is the multi-head self-attention feature capture output, where l is the transformer level index, MHSA(·) is the multi-head self-attention mechanism operation function, LN(·) is the layer normalization function, and Z l-1 is the multi-layer perceptual feature capture output of the l-1 transformer layer, Z l is the multi-layer perceptron feature capture output of the l-layer transformer layer, and MLP(·) is the multi-layer perceptron operation function;

[0111] The constructing of the teacher subnet specifically involves constructing a standard visual transformer model as the teacher subnet to obtain the teacher subnet feature data;

[0112] The construction of the local selection optimization sub-block is specifically to construct the local selection optimization sub-block by combining the attention weights in the multi-head self-attention feature capture, perform feature map optimization, and obtain fine-grained behavior analysis feature data. The calculation formula is:

[0113] ;

[0114] In the formula, Z output is the fine-grained behavior analysis feature data, M PS (·) is the local selection optimization sub-block operation function, Z input It is the local selection optimization sub-block input data, which is used to represent the output feature data of the backbone sub-network and the feature data of the teacher sub-network;

[0115] The behavior analysis model optimization training is specifically to optimize the behavior analysis model through the basic structure of the knowledge distillation model, the backbone subnet, the teacher subnet and the local selection optimization sub-block, combined with the model loss function to obtain the behavior analysis sub-model Model C2 , and by using the behavior analysis sub-model Model C2 , get the behavior analysis output;

[0116] The model loss function specifically refers to a combination of a cross entropy loss function and a contrast loss function;

[0117] The contrast loss function specifically includes a class loss function and a distillation feature loss function.

[0118] By performing the above operations, in view of the technical problem that in the existing behavior analysis methods, the behavior analysis for liveness detection is a fine-grained detection and classification process, and the classification granularity of the existing methods is insufficient and it is easy to ignore the potential characteristics of abnormal behavior after liveness detection, thereby reducing the usability of the overall method, this solution creatively adopts a fine-grained data efficient image transformer model combined with local selection optimization to perform behavior analysis. By introducing local selection sub-blocks and constructing a knowledge distillation model structure, the fine-grained degree of the behavior analysis process is improved, and feasible technical support is provided for subsequent abnormal processing and risk alarms.

[0119] Embodiment 6, this embodiment is based on the above embodiment, see Figure 5 The single-stage integrated detection is specifically to obtain the liveness detection sub-model and the behavior analysis sub-model, perform model integration and training, obtain a single-stage integrated detection model, and perform single-stage liveness detection in the blind area of ​​the visual field according to the raw data of liveness detection in the blind area of ​​the visual field by using the single-stage integrated detection model to obtain reference data for liveness detection in the blind area of ​​the visual field of the flow machine;

[0120] The liveness detection reference data of the blind spot of the streaming mechanical field of view specifically includes liveness detection mark data, behavior analysis type reference data and abnormal behavior reference data;

[0121] The behavior analysis type reference data specifically includes abnormal movement, sudden stop, long-term residence, living body fall and multi-person gathering;

[0122] The abnormal behavior reference data specifically refers to the risk quantification value of the behavior analysis type reference data.

[0123] By executing the above operations, in the existing streaming mechanical blind spot liveness detection system, there is an existing method that constructs multiple multi-stage detection models, and realizes a general blind spot liveness detection task through the gradual detection of multiple models. However, the streaming mechanical blind spot liveness detection has extremely high requirements for real-time performance, and multi-stage detection is likely to delay the detection timing, and the accuracy is inevitably reduced due to the combination of multiple stages. Technical problems, this scheme creatively adopts sub-models of two tasks for single-stage integrated detection, and improves the liveness detection and behavior analysis models as sub-models respectively, and performs integrated training to obtain a single-stage integrated detection model, which improves the real-time performance of the detection and further provides guarantee for the availability of the streaming mechanical blind spot liveness detection system.

[0124] Embodiment 7, this embodiment is based on the above embodiment, see Figure 5 The abnormal processing is specifically based on the reference data of the liveness detection in the blind spot of the flow mechanical field of view, and setting a risk alarm quantification index. When the risk quantification value in the abnormal behavior reference data is higher than the risk alarm quantification index, an abnormal alarm is issued;

[0125] The value range of the risk alarm quantitative index is [0,1];

[0126] The specific value range of the risk quantification value of the abnormal movement is [0.65, 1], and the risk quantification index corresponding to the abnormal movement is 0.7;

[0127] The specific value range of the risk alarm quantization value of the sudden stop is [0.5, 1], and the risk alarm quantization index corresponding to the sudden stop is 0.6;

[0128] The specific value range of the risk alarm quantification value of the long-term residence is [0.6, 1], and the risk alarm quantification index corresponding to the long-term residence is 0.65;

[0129] The specific value range of the risk alarm quantization value of the living body falling is [0.8, 1], and the risk alarm quantization index corresponding to the sudden stop is 0.85;

[0130] The specific value range of the risk alarm quantification value of the sudden stop is [0.4, 1], and the risk alarm quantification index corresponding to the sudden stop is 0.55.

[0131] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process and method including a series of elements include not only those elements, but also other elements not explicitly listed, or also include elements inherent to such process and method.

[0132] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0133] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A streaming mechanical visual field blind spot living body detection system, characterized by: It includes data acquisition module, data processing and fusion module, liveness detection module, behavior analysis module, single-stage integrated detection module and abnormality handling module; The data acquisition module performs real-time data acquisition and preliminary preprocessing in multiple environments to obtain raw data of liveness detection in blind areas of the visual field; The data processing and fusion module extracts features from the camera data and acoustic signal data in the raw data of the blind spot liveness detection, and performs multimodal data feature fusion to obtain an optimized liveness detection data set; The liveness detection module performs liveness detection based on the original camera data and comprehensive feature data in the optimized liveness detection data set, using a lightweight visual detection model combined with dilated convolution and adaptive gradient improvement to obtain liveness detection output and liveness detection sub-model, the steps including: constructing a standard model structure, dilated convolution improvement, adaptive gradient improvement and liveness detection model optimization training; The lightweight visual detection model combining dilated convolution and adaptive gradient improvement specifically includes a basic input layer, an improved dilated convolution layer, a standard convolution layer, a spatial pyramid fusion layer, an upsampling pooling layer, a fully connected layer, an adaptive gradient improvement operator and an output layer; The behavior analysis module is used to construct a fine-grained model and serve as a fine-grained classification sub-model of the living behavior of the final model. Through behavior analysis, the behavior analysis output and the behavior analysis sub-model are obtained, and the behavior analysis sub-model is sent to the single-stage integrated detection module. The steps include: constructing the basic structure of the knowledge distillation model, constructing the backbone subnet, constructing the teacher subnet, constructing the local selection optimization sub-block and optimizing the behavior analysis model training; The single-stage integrated detection module obtains the liveness detection sub-model and the behavior analysis sub-model, performs model integration and training, obtains a single-stage integrated detection model, and performs single-stage blind-spot liveness detection based on the blind-spot liveness detection raw data using the single-stage integrated detection model to obtain streaming mechanical blind-spot liveness detection reference data; The exception handling module is used to provide alarm feedback according to the detection results, and to perform real-time monitoring and exception alarm through exception handling.

2. According to claim 1, a flow-type mechanical visual field blind spot living body detection system is characterized by: The data collection is specifically to set up a streaming mechanical visual field blind spot sensor to perform multi-environment real-time data collection and preliminary preprocessing to obtain the original data of visual field blind spot liveness detection; The raw data of liveness detection in the blind area of ​​vision specifically includes RGB camera data, infrared camera data and acoustic signal auxiliary data; The multi-environment real-time data collection is specifically carried out by manually observing, dynamically adjusting the viewing angle of the sensor camera, and maximizing the coverage of the blind spots of the field of vision; The preliminary preprocessing specifically includes a median filtering denoising operation and an infrared image normalization operation.

3. A streaming mechanical visual field blind spot living body detection system according to claim 2, characterized in that: The data processing and fusion is specifically to extract features from the camera data and acoustic signal data in the raw data of the blind spot liveness detection, and perform multimodal data feature fusion to obtain an optimized liveness detection data set, and the specific steps include: camera feature extraction, acoustic signal feature extraction and feature fusion processing; The camera feature extraction specifically uses a standard convolutional neural network to extract features of RGB camera data and infrared camera data in the raw data of the blind spot living body detection to obtain camera feature data; The acoustic signal feature extraction is specifically to extract the frequency and amplitude features of the acoustic signal data through spectrum analysis, perform acoustic signal feature extraction, and obtain audio feature data; The feature fusion processing specifically comprises: using a weighted average method to fuse the camera feature data and the audio feature data with feature vectors, calculating the comprehensive feature data, and obtaining an optimized liveness detection data set through data set integration; The optimized liveness detection data set specifically includes original camera data, camera feature data and comprehensive feature data; The original camera data specifically refers to the RGB camera data and infrared camera data in the blind spot liveness detection original data; The calculation formula for obtaining the comprehensive characteristic data is: ； In the formula, F fused is the comprehensive feature data, is the feature weight, F I is the camera feature data, F S It is audio feature data.

4. A streaming mechanical visual field blind spot living body detection system according to claim 3, characterized in that: The step of performing liveness detection by using a lightweight visual detection model combining dilated convolution and adaptive gradient improvement to obtain liveness detection output and a liveness detection sub-model specifically includes: constructing a standard model structure, dilated convolution improvement, adaptive gradient improvement, and liveness detection model optimization training; The constructing of the standard model structure specifically involves constructing a standard lightweight visual detection model including a basic input layer, a standard convolutional layer, a spatial pyramid fusion layer, an upsampling pooling layer, a fully connected layer, and an output layer; The standard lightweight visual detection model specifically refers to the YOLOv8s model; The spatial pyramid fusion layer is used for multi-scale feature fusion; the upsampling pooling layer is used for restoring the feature map size and merging the feature maps; The dilated convolution improvement specifically replaces the standard convolution layer of the standard lightweight visual detection model with an improved dilated convolution layer that is a combination of a dilated convolution layer and a standard convolution layer, and obtains a dilated improved lightweight visual detection model by constructing the improved dilated convolution layer, and performs feature depth extraction to obtain deep living body detection feature data; The adaptive gradient improvement is specifically to introduce adaptive parameters to expand and improve the lightweight visual detection model to update the parameters, so as to obtain the expanded parameter improved lightweight visual detection model; the loss function is specifically a binary cross entropy loss function; The optimization training of the liveness detection model is specifically to construct an expanded and improved lightweight visual detection model and adaptive gradient improvement, and use a Gaussian error linear unit to replace the standard nonlinear activation function as the activation function of the model to perform optimization training of the liveness detection model to obtain the liveness detection sub-model Model C1 , and by using the liveness detection sub-model Model C1 , obtaining a liveness detection output according to the original camera data in the optimized liveness detection data set; The calculation formula of the Gaussian error linear unit is: ； Where GELU(·) is the Gaussian error linear unit function, It is the input data to be activated, which is used to represent the feature map data after the improved dilated convolution layer, standard convolution layer, spatial pyramid fusion layer, upsampling pooling layer and fully connected layer.

5. The streaming mechanical visual field blind spot living body detection system according to claim 4 is characterized in that: The behavior analysis uses a fine-grained data efficient image transformer model combined with local selection optimization to perform behavior analysis, and the fine-grained data efficient image transformer model combined with local selection optimization specifically includes a teacher subnet, a backbone subnet and a local selection optimization sub-block; The steps of using the fine-grained data efficient image transformer model combined with local selection optimization to perform behavior analysis and obtain behavior analysis output and behavior analysis submodel include: constructing the basic structure of the knowledge distillation model, constructing the backbone subnet, constructing the teacher subnet, constructing the local selection optimization subblock and optimizing the behavior analysis model training; The constructing of the basic structure of the knowledge distillation model specifically includes constructing the basic structure of the knowledge distillation model including a student subnet and a teacher subnet, and using the backbone subnet as the student subnet; The constructing of the backbone subnet is specifically to construct a feature extraction subnet based on a standard fine-grained data efficient image transformer model, wherein the standard fine-grained data efficient image transformer model specifically refers to a convolution transformer model combining a multi-head self-attention mechanism and a multi-layer perceptron, and by performing multi-head self-attention feature capture and multi-layer perceptron feature capture, a behavior analysis feature extraction is performed to obtain the backbone subnet output feature data; The constructing of the teacher subnet specifically involves constructing a standard visual transformer model as the teacher subnet to obtain the teacher subnet feature data; The constructing of the local selection optimization sub-block specifically involves constructing the local selection optimization sub-block by combining the attention weights in the multi-head self-attention feature capture, performing feature map optimization, and obtaining fine-grained behavior analysis feature data; The behavior analysis model optimization training is specifically to optimize the behavior analysis model through the basic structure of the knowledge distillation model, the backbone subnet, the teacher subnet and the local selection optimization sub-block, combined with the model loss function to obtain the behavior analysis sub-model Model C2 , and by using the behavior analysis sub-model Model C2 , get the behavior analysis output; The model loss function specifically refers to a combination of a cross entropy loss function and a contrast loss function; The contrast loss function specifically includes a class loss function and a distillation feature loss function.

6. The streaming mechanical visual field blind spot living body detection system according to claim 5, characterized in that: The single-stage integrated detection is specifically to obtain the liveness detection sub-model and the behavior analysis sub-model, perform model integration and training, obtain a single-stage integrated detection model, and perform single-stage blind spot liveness detection based on the blind spot liveness detection raw data by using the single-stage integrated detection model to obtain the streaming mechanical blind spot liveness detection reference data; The liveness detection reference data of the blind spot of the streaming mechanical field of view specifically includes liveness detection mark data, behavior analysis type reference data and abnormal behavior reference data; The behavior analysis type reference data specifically includes abnormal movement, sudden stop, long-term residence, living body fall and multi-person gathering; The abnormal behavior reference data specifically refers to the risk quantification value of the behavior analysis type reference data.

7. The streaming mechanical visual field blind spot living body detection system according to claim 6, characterized in that: The abnormal processing is specifically based on the reference data of liveness detection in the blind spot of the flow mechanical field of view, and setting a risk alarm quantification index. When the risk quantification value in the abnormal behavior reference data is higher than the risk alarm quantification index, an abnormal alarm is issued.

8. The streaming mechanical visual field blind spot living body detection system according to claim 7 is characterized by: The data acquisition module is used to collect the raw data required for liveness detection in the blind area of ​​the flow machine vision, obtain the raw data for liveness detection in the blind area of ​​the vision through data acquisition, and send the raw data for liveness detection in the blind area of ​​the vision to the data processing fusion module and the single-stage integrated detection module; The data processing and fusion module is used to enhance and optimize the original data, obtain an optimized liveness detection data set through data processing and fusion, and send the optimized liveness detection data set to the liveness detection module and the behavior analysis module; The liveness detection module is used to improve the target detection model and serve as a liveness target detection sub-model of the final model. Through liveness detection, a liveness detection output and a liveness detection sub-model are obtained, and the liveness detection output is sent to the behavior analysis module, and the liveness detection sub-model is sent to the single-stage integrated detection module; The behavior analysis module is used to construct a fine-grained model and serve as a fine-grained classification sub-model of the living behavior of the final model, obtain a behavior analysis output and a behavior analysis sub-model through behavior analysis, and send the behavior analysis sub-model to the single-stage integrated detection module; The single-stage integrated detection module is used to integrate the liveness detection and behavior analysis sub-models as a single-stage detection model, obtain the liveness detection reference data of the blind area of ​​the flow mechanical field of view through the single-stage integrated detection, and send the liveness detection reference data of the blind area of ​​the flow mechanical field of view to the abnormality processing module; The exception handling module is used to provide alarm feedback according to the detection results, and to perform real-time monitoring and exception alarm through exception handling.

Images