Intelligent security and protection method based on machine vision

Abstract

The invention relates to the technical field of machine vision, and discloses an intelligent security and protection method and system based on machine vision, and the method comprises the steps: extracting a security and protection video frame sequence corresponding to a target security and protection video, and calculating a reference enhancement parameter; performing inter-frame image enhancement on the security video frame sequence according to the reference enhancement parameter to obtain an enhanced image frame sequence; performing interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence, and calculating a multi-scale full-dimensional feature of each target image frame; calculating scale weights of the multi-scale full-dimensional features, and performing normalized fusion on the multi-scale full-dimensional features according to the scale weights to obtain target features of the target image frame; and identifying a security target state in the target image frame according to the target feature, and performing security early warning on the target security video according to the security target state. According to the invention, the accuracy of intelligent security and protection can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to an intelligent security method based on machine vision. Background Art

[0002] Intelligent security systems play an increasingly important role in protecting people's lives and property. Intelligent security is a method that uses computers to automatically detect, track and identify. On the premise of minimizing reliance on humans, it obtains useful information from a large number of surveillance videos, understands and analyzes them, and detects, identifies and tracks targets, thereby achieving security monitoring and alarm functions. With the rapid development of science and technology, the application of machine vision in the field of intelligent security will become more extensive and in-depth, providing more powerful protection for social security and personal safety.

[0003] The existing intelligent security methods mainly use Kinect sensors to obtain behavioral data in surveillance videos and detect unsafe human behaviors through pattern recognition; or extract data features of behavioral data as input to feedforward neural networks to identify different unsafe behaviors; and on the basis of detecting human targets in surveillance video images, use the acquired directional gradient histogram features to complete the prediction and identification of unsafe behaviors. In the existing technology, the detection effect is mainly determined by the results of video image feature extraction, and the behavior of the target reflected by a single feature is extremely one-sided, resulting in poor accuracy of intelligent security. Summary of the invention

[0004] The present invention provides an intelligent security method based on machine vision, the main purpose of which is to solve the problem of poor accuracy of intelligent security.

[0005] To achieve the above object, the present invention provides an intelligent security method based on machine vision, comprising:

[0006] Acquire a target security video, extract a security video frame sequence corresponding to the target security video, and calculate a baseline enhancement parameter of the security video frame sequence;

[0007] Performing baseline enhancement on the security video frame sequence according to the baseline enhancement parameters to obtain baseline enhanced image frames, and performing inter-frame image enhancement on the security video frame sequence according to the baseline enhanced image frames to obtain an enhanced image frame sequence;

[0008] Performing interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence, and calculating multi-scale full-dimensional features of each target image frame in the target image frame sequence;

[0009] Calculating the scale weight of the multi-scale full-dimensional features, and normalizing and fusing the multi-scale full-dimensional features according to the scale weight to obtain the target features of the target image frame;

[0010] The security target state in the target image frame is identified according to the target feature, and a security warning is performed on the target security video according to the security target state.

[0011] Optionally, the calculating the baseline enhancement parameter of the security video frame sequence includes:

[0012] Converting the first security video frame in the security video frame sequence into a preset color space to obtain a spatial video frame;

[0013] Extracting the illumination component of the spatial video frame, and constructing an image evaluation function according to the illumination component;

[0014] The image evaluation function is optimized to obtain a baseline enhancement parameter of the security video frame sequence.

[0015] Optionally, optimizing the image evaluation function to obtain a baseline enhancement parameter of the security video frame sequence includes:

[0016] The image evaluation function is shown in the following formula:

[0017]

[0018] Among them, F represents the image evaluation function, α, β represent the preset initial enhancement parameters, and L in represents the illumination value in the illumination component, num(L in ) indicates that the illumination value in the spatial video frame is L in , where n represents the number of pixels in the spatial video frame, t represents the integral variable, and dt represents the differential element of the integral.

[0019] Optionally, performing baseline enhancement on the security video frame sequence according to the baseline enhancement parameter to obtain a baseline enhanced image frame includes:

[0020] Extracting a first security video frame from the security video frame sequence, and performing illumination component mapping on the first security video frame using the reference enhancement parameter to obtain a reference illumination component;

[0021] The illumination component of the first security video frame is mapped using the following formula:

[0022]

[0023] Wherein, T represents the reference illumination component, u represents the normalized illumination value of the first security video frame, represents the reference enhancement parameter, D represents the illumination value range corresponding to the first security video frame, and dt represents the differential element of the integral;

[0024] The first security video frame is reconstructed according to the reference illumination component to obtain a reference enhanced image frame.

[0025] Optionally, performing inter-frame image enhancement on the security video frame sequence according to the reference enhanced image frame to obtain an enhanced image frame sequence includes:

[0026] Taking the reference enhanced image frame as a reference frame, calculating a histogram of a security video frame next to the reference frame and the reference enhanced image frame, and calculating a Euclidean distance between the histograms;

[0027] When the Euclidean distance is less than a preset threshold, performing image enhancement on the next security video frame using the benchmark enhancement parameters of the benchmark enhanced image frame to obtain a second enhanced image frame, and using the second enhanced image frame as a benchmark frame;

[0028] When the Euclidean distance is greater than or equal to the threshold, calculating an updated enhanced image frame corresponding to the next security video frame, and using the updated enhanced image frame as a reference frame;

[0029] Return to the above step of calculating the histogram of the next security video frame of the reference frame and the reference enhanced image frame, and calculating the Euclidean distance between the histograms, until the next security video frame is the last security video frame in the security video frame sequence, to obtain an enhanced image frame sequence.

[0030] Optionally, performing interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence includes:

[0031] Converting the enhanced image frame sequence into a grayscale image to obtain a grayscale image frame sequence;

[0032] Extracting continuous differential image frames of each grayscale image frame in the grayscale image frame sequence, and calculating the median grayscale value of the grayscale image frame;

[0033] Calculating an interval differential image between the differential image frames, and converting the interval differential image into a binary image according to the median of the grayscale value;

[0034] Perform image detection on the binary image to obtain a target image frame, and collect the target image frames to obtain a target image frame sequence.

[0035] Optionally, calculating the multi-scale full-dimensional features of each target image frame in the target image frame sequence includes:

[0036] Convert the target image frame into a feature vector, perform vector mapping and full connection on the feature vector, and obtain a target feature vector;

[0037] Calculating the convolution attention of the preset multi-layer convolution kernel using a pre-built full-dimensional dynamic convolution module according to the target feature vector;

[0038] Adjusting the attention of the multiple convolution kernels according to the convolution attention to obtain a target multi-layer convolution kernel;

[0039] The target multi-layer convolution kernel is used to perform layered convolution on the feature vector to obtain multi-scale full-dimensional features of each target image frame.

[0040] Optionally, calculating the scale weight of the multi-scale full-dimensional feature includes:

[0041] The scale weight of the multi-scale full-dimensional feature is calculated using the following formula:

[0042]

[0043] Where W represents the scale weight, w represents the preset initial weight, Resize represents the feature map size adjustment parameter, R represents the feature in the multi-scale full-dimensional feature, ε represents the preset minimum value, δ represents the activation function, and f 7×7 represents a 7×7 convolution operation, G max represents the maximum pooling operation, G ave Represents an average pooling operation.

[0044] Optionally, the normalizing and fusing the multi-scale full-dimensional features according to the scale weights to obtain target features of the target image frame includes:

[0045] The multi-scale full-dimensional features are normalized and fused using the following formula:

[0046]

[0047] Among them, O represents the target feature, w j Represents the jth feature R in the multi-scale full-dimensional feature j The corresponding scale weight, ε represents the preset minimum value.

[0048] Optionally, performing a security warning on the target security video according to the security target state includes:

[0049] Tracking the security target state to obtain a moving target state set;

[0050] Detecting the motion state behavior of the motion target in the motion target state set;

[0051] A security warning is performed on the target security video according to the motion state behavior.

[0052] The embodiment of the present invention can obtain a more accurate enhanced image frame sequence while reducing the amount of calculation by performing baseline enhancement on the target security video; performing interval differential optimization on the enhanced image frame sequence to obtain a target image frame sequence with a moving target, and calculating the multi-scale full-dimensional features of each target image frame in the target image frame sequence, which can extract feature information of different scales, further improve the richness and comprehensiveness of feature extraction, and then more accurately identify the security target state in the target image frame, quickly and accurately identify the unsafe behavior of the moving target in the target security video, and realize accurate intelligent security. Therefore, the intelligent security method and system based on machine vision proposed by the present invention can solve the problem of poor accuracy of intelligent security. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a process flow of an intelligent security method based on machine vision provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of a flow chart of calculating a baseline enhancement parameter of a security video frame sequence provided by an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of a process for performing interval difference optimization on an enhanced image frame sequence provided by an embodiment of the present invention;

[0056] Figure 4 A functional module diagram of a machine vision-based intelligent security system provided in one embodiment of the present invention.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0059] The embodiment of the present application provides an intelligent security method based on machine vision. The execution subject of the intelligent security method based on machine vision includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the intelligent security method based on machine vision can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0060] Reference Figure 1 FIG. 1 is a flow chart of an intelligent security method based on machine vision provided by an embodiment of the present invention. In this embodiment, the intelligent security method based on machine vision includes:

[0061] S1. Acquire a target security video, extract a security video frame sequence corresponding to the target security video, and calculate a baseline enhancement parameter of the security video frame sequence.

[0062] In the embodiment of the present invention, the target security video is a video shot in an intelligent security scenario, for example, the time frequency of urban security monitoring, the video shot by home security equipment, and the intelligent security scenarios of intelligent security applications such as commercial security and traffic management. Then, a security video frame sequence is extracted from the target security video at a preset time interval, for example, video frames are extracted from the target security video every second to form a security video frame sequence, video frames are extracted from the target security video every half second to form a security video frame sequence, or other preset time intervals are used to extract video frames to form a security video frame sequence.

[0063] In an embodiment of the present invention, the baseline enhancement parameter is an enhancement parameter corresponding to the first security video frame in the security video frame sequence, wherein the enhancement parameter can adjust the illumination component of the first security video frame, thereby effectively suppressing noise near low illumination, removing the influence of light source illumination, and restoring the original color of the target in the security video frame.

[0064] In the embodiment of the present invention, refer to Figure 2 As shown, the calculation of the baseline enhancement parameters of the security video frame sequence includes:

[0065] S21, converting the first security video frame in the security video frame sequence into a preset color space to obtain a spatial video frame;

[0066] S22, extracting the illumination component of the spatial video frame, and constructing an image evaluation function according to the illumination component;

[0067] S23: Optimize the image evaluation function to obtain a baseline enhancement parameter of the security video frame sequence.

[0068] In an embodiment of the present invention, the preset color space is the HSI (Hue-Saturation-Intensity) color space, which is a color model that uses hue (Hue), saturation (Saturation or Chroma), and brightness (Intensity or Brightness) to describe colors, and obtains a spatial video frame under the HSI color space.

[0069] Furthermore, the spatial video frame can be converted into the product of an illuminance component and a reflection component, wherein the illuminance component contains the approximate outline and brightness distribution of the image, while the reflection component contains all edge details and color information of the image. Specifically, the illuminance component can be estimated by performing Gaussian filtering on the spatial video frame to obtain the illuminance component of the spatial video frame.

[0070] Specifically, the image evaluation function is shown in the following formula:

[0071]

[0072] Among them, F represents the image evaluation function, α, β represent the preset initial enhancement parameters, and L in represents the illumination value in the illumination component, num(L in ) indicates that the illumination value in the spatial video frame is L in , where n represents the number of pixels in the spatial video frame, t represents the integral variable, and dt represents the differential element of the integral.

[0073] In the embodiment of the present invention, the image evaluation function includes two initial enhancement parameters α and β. The image evaluation function is optimized, and the values ​​of the initial enhancement parameters α and β when the image evaluation function is maximum are calculated to obtain the reference enhancement parameters.

[0074] Specifically, the image evaluation function can be used as a fitness function, and the genetic algorithm can be used to optimize the image evaluation function, and then the enhancement function corresponding to the maximum value of the image evaluation function can be solved to obtain a benchmark enhancement function.

[0075] In the embodiment of the present invention, the enhancement coefficient corresponding to the first security video frame in the security video frame sequence can be solved through the reference enhancement function, and then the security video frame sequence is enhanced with the first security video frame as the reference frame, thereby reducing the amount of calculation and improving the efficiency of generating the enhanced image frame sequence.

[0076] S2. Performing baseline enhancement on the security video frame sequence according to the baseline enhancement parameters to obtain baseline enhanced image frames, and performing inter-frame image enhancement on the security video frame sequence according to the baseline enhanced image frames to obtain an enhanced image frame sequence.

[0077] In the embodiment of the present invention, the reference enhancement is to perform image enhancement on the first security video frame in the security video frame sequence using the reference enhancement parameters to obtain a reference enhanced image frame corresponding to the first security video frame.

[0078] In the embodiment of the present invention, the step of performing baseline enhancement on the security video frame sequence according to the baseline enhancement parameter to obtain a baseline enhanced image frame includes:

[0079] Extracting a first security video frame from the security video frame sequence, and performing illumination component mapping on the first security video frame using the reference enhancement parameter to obtain a reference illumination component;

[0080] The first security video frame is reconstructed according to the reference illumination component to obtain a reference enhanced image frame.

[0081] In the embodiment of the present invention, the illumination component mapping is performed on the first security video frame using the following formula:

[0082]

[0083] Wherein, T represents the reference illumination component, u represents the normalized illumination value of the first security video frame, represents the reference enhancement parameter, D represents the illumination value range corresponding to the first security video frame, and dt represents the differential element of the integral.

[0084] In an embodiment of the present invention, the benchmark enhancement parameter is the value corresponding to the enhancement function when the image evaluation function is maximum, and u represents the normalized value of the illuminance value of the spatial video frame corresponding to the first security video frame. The illuminance component of the spatial video frame can be normalized to obtain a normalized illuminance value. Specifically, the ratio of the illuminance value to the illuminance value range can be used as the normalization value, and then the illuminance value can be mapped to the range of 0 to 1, thereby improving the efficiency of the calculation.

[0085] In detail, the reference illumination component is multiplied by the reflection component corresponding to the illumination component to obtain the reconstructed first security video frame, and the reconstructed first security video frame is converted into the RGB color space to obtain the reference enhanced image frame.

[0086] In the embodiment of the present invention, the reference illumination component can be evenly distributed to the maximum extent through the reference enhancement parameter, so that the illumination value is stretched within a moderate illumination range, the noise near the minimum illumination is effectively suppressed, and a more accurate reference enhanced image frame is obtained.

[0087] In the embodiment of the present invention, since adjacent security video frames in the security video frame sequence have a strong correlation and their histogram information is relatively similar, the enhancement parameters of adjacent security video frames are also similar. Therefore, by taking the reference enhanced image frame as the basis, image enhancement is performed on the security video frames after the first frame of the security video frame sequence, which can reduce the computational complexity of image enhancement of the entire security video frame sequence.

[0088] Specifically, performing inter-frame image enhancement on the security video frame sequence according to the reference enhanced image frame to obtain an enhanced image frame sequence includes:

[0089] Taking the reference enhanced image frame as a reference frame, calculating a histogram of a security video frame next to the reference frame and the reference enhanced image frame, and calculating a Euclidean distance between the histograms;

[0090] When the Euclidean distance is less than a preset threshold, performing image enhancement on the next security video frame using the benchmark enhancement parameters of the benchmark enhanced image frame to obtain a second enhanced image frame, and using the second enhanced image frame as a benchmark frame;

[0091] When the Euclidean distance is greater than or equal to the threshold, calculating an updated enhanced image frame corresponding to the next security video frame, and using the updated enhanced image frame as a reference frame;

[0092] Return to the above step of calculating the histogram of the next security video frame of the reference frame and the reference enhanced image frame, and calculating the Euclidean distance between the histograms, until the next security video frame is the last security video frame in the security video frame sequence, to obtain an enhanced image frame sequence.

[0093] In the embodiment of the present invention, the reference enhanced image frame is the first security video frame, then the Euclidean distance of the histogram between the second security image frame and the reference enhanced image frame in the security video frame sequence is calculated, and when the Euclidean distance is less than a preset threshold, the reference enhancement parameters of the reference enhanced image frame are used as the enhancement parameters of the security image frame, thereby performing image enhancement on the second security image frame to obtain an updated enhanced image frame; when the Euclidean distance is greater than or equal to the preset threshold, the enhancement parameters of the second security image frame are recalculated according to the above-mentioned step S1, and the second security image frame is enhanced using the enhancement parameters of the second security image frame to obtain an updated enhanced image frame.

[0094] Furthermore, according to the Euclidean distance, the updated enhanced image frame or the second enhanced image frame is used again as a reference frame to perform image enhancement of the next security video frame, until the next security video frame is the last video frame in the security video frame sequence, completing the image enhancement of the entire security video frame sequence to obtain an enhanced image frame sequence.

[0095] In the embodiment of the present invention, by performing inter-frame image enhancement on the security video frame sequence, the computational complexity can be reduced while performing image enhancement on the security video frame sequence, more image information can be retained, and the efficiency and accuracy of image enhancement can be improved.

[0096] S3. Performing interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence, and calculating multi-scale full-dimensional features of each target image frame in the target image frame sequence.

[0097] In an embodiment of the present invention, when a moving target appears in an enhanced image frame sequence, a more obvious pixel value change will occur between the pixel points corresponding to two adjacent frames, thereby realizing the detection of the moving target area. Therefore, through differential optimization, the enhanced image frames in which the moving target appears in the enhanced image frame sequence can be detected, and the enhanced image frame sequence is optimized to obtain a target image frame sequence with a moving target.

[0098] In the embodiment of the present invention, refer to Figure 3 As shown, performing interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence includes:

[0099] S31, converting the enhanced image frame sequence into a grayscale image to obtain a grayscale image frame sequence;

[0100] S32, extracting a continuous differential image frame of each grayscale image frame in the grayscale image frame sequence, and calculating a median grayscale value of the grayscale image frame;

[0101] S33, calculating an interval differential image between the differential image frames, and converting the interval differential image into a binary image according to the grayscale value median;

[0102] S34, performing image detection on the binary image to obtain a target image frame, and collecting the target image frames to obtain a target image frame sequence.

[0103] In an embodiment of the present invention, each enhanced image frame in the enhanced image frame sequence is grayed to obtain a grayscale image frame sequence, and then the grayscale image frames adjacent to and continuous with each grayscale image frame are extracted as differential image frames. For example, if the grayscale image frame is the kth frame, the grayscale image frames of the k-1th frame, the k-2th frame, and the k-3th frame are extracted to form a differential image frame with the kth frame, where k is a positive integer greater than 3.

[0104] In detail, the median grayscale value can be processed by median filtering to obtain the median of the grayscale level of each grayscale image frame. For example, each pixel of the grayscale image frame and the neighborhood composed of the surrounding 8 pixels are used as a filtering template to calculate the median of all the pixels in the template, and then the median grayscale value of the grayscale image frame is calculated using the median of all the pixels.

[0105] Furthermore, the interval differential image can be obtained by differentiating the k-3 frame image from the k-1 frame image, and differentiating the k-2 frame image from the k frame image in the differential image frame, and obtaining a differential image. The median grayscale value is used as a threshold to convert the differential image into a binary image. A logical AND operation is performed on the binary image to eliminate errors caused by environmental changes (such as lighting changes, shadows, etc.) to obtain a logically processed image. Then, the significant area in the logically processed image is detected, which indicates that there is a moving target in the grayscale image frame, and the grayscale image frame is used as the target image frame.

[0106] Specifically, morphological operations such as opening and closing operations can be performed on the logical processing image to eliminate noise and small holes, and then a minimum area threshold can be set to only consider connected areas with an area larger than the minimum area threshold as significant areas; or the shape features of the area such as aspect ratio and compactness can be analyzed to identify significant areas in the roadbed logical processing image after morphological operations, and significant areas where moving targets exist can be detected by edge detection methods to determine whether there are moving targets.

[0107] In an embodiment of the present invention, grayscale image frames containing moving targets can be identified through interval differential optimization. At the same time, through four-frame interval differential optimization, the loss of edge information and the formation of holes in the overlapping parts of the moving targets, which in serious cases causes the problem of disconnected segmentation results, can be avoided, and the target image frame sequence can be obtained more accurately.

[0108] In an embodiment of the present invention, the multi-scale full-dimensional feature uses full-dimensional dynamic convolution to replace ordinary convolution, wherein the full-dimensional dynamic convolution calculates four types of attention values ​​(spatial dimension, input channel dimension, output channel dimension and convolution kernel dimension) in parallel, and assigns different attention weights to each convolution kernel, so as to better capture rich contextual information.

[0109] Furthermore, the calculating of the multi-scale full-dimensional features of each target image frame in the target image frame sequence includes:

[0110] Convert the target image frame into a feature vector, perform vector mapping and full connection on the feature vector, and obtain a target feature vector;

[0111] Calculating the convolution attention of the preset multi-layer convolution kernel using a pre-built full-dimensional dynamic convolution module according to the target feature vector;

[0112] Adjusting the attention of the multiple convolution kernels according to the convolution attention to obtain a target multi-layer convolution kernel;

[0113] The target multi-layer convolution kernel is used to perform layered convolution on the feature vector to obtain multi-scale full-dimensional features of each target image frame.

[0114] In an embodiment of the present invention, the feature vector can be vector-mapped through a global average pooling operation, the feature vector can be mapped to a low-dimensional space, and then the nonlinear expression ability of the feature vector can be enhanced through a fully connected layer to obtain a more accurate target feature vector.

[0115] Furthermore, the attention module in the pre-built full-dimensional dynamic convolution (ODConv) can be used to calculate the attention of the feature vector in the spatial dimension of each convolution kernel, the input channel dimension, the output channel dimension and the convolution kernel dimension, wherein the full-dimensional dynamic convolution module includes multiple attention heads for calculating convolution attention of different dimensions, for example, having multiple attention heads to calculate multiple types of attention.

[0116] In detail, each convolution attention is used to adjust the weight of each convolution kernel in different dimensions. For example, the spatial dimension attention is used to adjust the weight of the convolution kernel in the spatial dimension (ie, the height and width of the convolution kernel), the input channel dimension attention is used to adjust the weight of the convolution kernel in the input channel dimension, and the output channel dimension attention is used to adjust the weight of the convolution kernel in the output channel dimension. The convolution kernel dimension attention is used to assign different values ​​to the overall convolution kernel.

[0117] In an embodiment of the present invention, the preset multi-layer convolution kernel can be composed of convolution kernels of multiple different convolution scales. The output of the previous layer is the input of the next layer. The output of each layer of convolution kernel is used as a feature of different scales to obtain multi-scale full-dimensional features.

[0118] In the embodiment of the present invention, the rich context information can be better captured through full-dimensional convolution, and the feature information of different scales can be extracted by using convolution kernels of different scales, thereby further improving the richness and comprehensiveness of feature extraction.

[0119] S4. Calculate the scale weight of the multi-scale full-dimensional features, and perform normalization fusion on the multi-scale full-dimensional features according to the scale weight to obtain the target features of the target image frame.

[0120] In the embodiment of the present invention, the scale weight is the importance of each scale feature in the full-scale full-dimensional features. The larger the scale weight is, the more feature information is retained in the subsequent normalization fusion, thereby realizing the screening of feature information and improving the accuracy of normalization fusion.

[0121] In the embodiment of the present invention, the calculating the scale weight of the multi-scale full-dimensional feature includes:

[0122] The scale weight of the multi-scale full-dimensional feature is calculated using the following formula:

[0123]

[0124] Where W represents the scale weight, w represents the preset initial weight, Resize represents the feature map size adjustment parameter, R represents the feature in the multi-scale full-dimensional feature, ε represents the preset minimum value, δ represents the activation function, and f 7×7 represents a 7×7 convolution operation, G max represents the maximum pooling operation, G ave Represents an average pooling operation.

[0125] In the embodiment of the present invention, the multi-scale full-dimensional features are normalized and fused through scale weights, and the feature weights of different levels can be adaptively adjusted to ensure effective information transmission, which can effectively alleviate the problem of excessive or insufficient information and optimize the feature fusion process.

[0126] Specifically, the normalizing and fusing the multi-scale full-dimensional features according to the scale weights to obtain the target features of the target image frame includes:

[0127] The multi-scale full-dimensional features are normalized and fused using the following formula:

[0128]

[0129] Among them, O represents the target feature, w j Represents the jth feature R in the multi-scale full-dimensional feature j The corresponding scale weight, ε represents the preset minimum value.

[0130] In the embodiment of the present invention, the multi-scale full-dimensional features are normalized and fused through scale weights, and the feature weights of different levels can be automatically adjusted to ensure effective information transmission and obtain more accurate target features, which is conducive to improving the accuracy of subsequent security warnings.

[0131] S5. Identify the security target state in the target image frame according to the target feature, and perform a security warning on the target security video according to the security target state.

[0132] In the embodiment of the present invention, the security target state is the behavior state of the moving target, for example, the movement state of a vehicle, wandering pedestrians, illegal intrusion of pedestrians, intrusion of external objects, normal driving, normal walking, etc., and then the unsafe behavior of the moving target is identified and an early warning is issued for the unsafe behavior in the target security video.

[0133] Specifically, a classification module can be set after the target features are calculated. For example, a fully connected layer can be set to map the target features to the feature space corresponding to the security target state, and the security target state corresponding to each moving target is selected, such as pedestrians walking on the street, cars driving, etc.

[0134] In the embodiment of the present invention, the step of performing a security warning on the target security video according to the security target state includes:

[0135] Tracking the security target state to obtain a moving target state set;

[0136] Detecting the motion state behavior of the motion target in the motion target state set;

[0137] A security warning is performed on the target security video according to the motion state behavior.

[0138] Specifically, the security target state of the same moving target in all target image frames is used to track the moving target state to obtain a moving target state set for each moving target. The motion trajectory and continuous motion behavior of the moving target can be determined according to the continuous security target state, and then the state behavior of each moving target can be obtained, such as abnormal motion trajectory of personnel, abnormal events, or reverse driving of vehicles, abnormal behavior state of personnel, etc. Specifically, the motion state behavior can be performed by setting motion trajectory and motion behavior detection rules. For example, the corresponding motion trajectory and motion behavior detection rules are different in different intelligent security scenarios to improve the flexibility and accuracy of security warning.

[0139] In detail, the rules can be matched according to the motion state behavior and the preset warning to obtain the corresponding security warning, and the security warning can be sent to the corresponding security processing end to quickly respond to and process the security processing. Among them, the security warning can include the warning level, the motion target of the warning, and the motion state behavior that needs to be warned, thereby improving the accuracy of the security warning.

[0140] In the embodiment of the present invention, by performing security warning on the target security video according to the security target state, the unsafe behavior of the moving target in the target security video can be quickly and accurately identified, thereby realizing precise intelligent security.

[0141] like Figure 4 , which is a functional module diagram of an intelligent security system based on machine vision provided by one embodiment of the present invention.

[0142] The intelligent security system 400 based on machine vision of the present invention can be installed in an electronic device. According to the functions implemented, the intelligent security system 400 based on machine vision can include a reference enhancement parameter calculation module 401, an image enhancement module 402, a feature extraction module 403, a feature fusion module 404 and a security warning module 405. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0143] In this embodiment, the functions of each module / unit are as follows:

[0144] The reference enhancement parameter calculation module 101 is used to obtain a target security video, extract a security video frame sequence corresponding to the target security video, and calculate a reference enhancement parameter of the security video frame sequence;

[0145] The image enhancement module 102 is used to perform baseline enhancement on the security video frame sequence according to the baseline enhancement parameters to obtain a baseline enhanced image frame, and perform inter-frame image enhancement on the security video frame sequence according to the baseline enhanced image frame to obtain an enhanced image frame sequence;

[0146] The feature extraction module 103 is used to perform interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence, and calculate the multi-scale full-dimensional features of each target image frame in the target image frame sequence;

[0147] The feature fusion module 104 is used to calculate the scale weight of the multi-scale full-dimensional feature, and perform normalization fusion on the multi-scale full-dimensional feature according to the scale weight to obtain the target feature of the target image frame;

[0148] The security warning module 105 is used to identify the security target state in the target image frame according to the target feature, and perform a security warning on the target security video according to the security target state.

[0149] In detail, each module described in the intelligent security system 400 based on machine vision in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 3 The same technical means as the intelligent security method based on machine vision described in the text can produce the same technical effects, so I will not go into details here.

[0150] The present invention also provides an electronic device, which may include a processor, a memory, a communication bus and a communication interface, and may also include a computer program stored in the memory and executable on the processor, such as an intelligent security method program based on machine vision.

[0151] In some embodiments, the processor may be composed of an integrated circuit, for example, it may be composed of a single packaged integrated circuit, or it may be composed of multiple packaged integrated circuits with the same or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors and a combination of various control chips.

[0152] The memory includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device.

[0153] The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory and at least one processor, etc.

[0154] The communication interface is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface or a wireless interface.

[0155] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0156] Specifically, the specific implementation method of the processor for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0157] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0158] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0160] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0161] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0162] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0163] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent security method based on machine vision, characterized in that: The method comprises: Acquire a target security video, extract a security video frame sequence corresponding to the target security video, and calculate a baseline enhancement parameter of the security video frame sequence; Performing baseline enhancement on the security video frame sequence according to the baseline enhancement parameters to obtain baseline enhanced image frames, and performing inter-frame image enhancement on the security video frame sequence according to the baseline enhanced image frames to obtain an enhanced image frame sequence; Performing interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence, and calculating multi-scale full-dimensional features of each target image frame in the target image frame sequence; Calculating the scale weight of the multi-scale full-dimensional features, and normalizing and fusing the multi-scale full-dimensional features according to the scale weight to obtain the target features of the target image frame; The security target state in the target image frame is identified according to the target feature, and a security warning is performed on the target security video according to the security target state.

2. The intelligent security method based on machine vision according to claim 1, characterized in that: The calculating of the baseline enhancement parameter of the security video frame sequence includes: Converting the first security video frame in the security video frame sequence into a preset color space to obtain a spatial video frame; Extracting the illumination component of the spatial video frame, and constructing an image evaluation function according to the illumination component; The image evaluation function is optimized to obtain a baseline enhancement parameter of the security video frame sequence.

3. The intelligent security method based on machine vision as claimed in claim 2, characterized in that: Optimizing the image evaluation function to obtain the baseline enhancement parameters of the security video frame sequence includes: The image evaluation function is shown in the following formula: Among them, F represents the image evaluation function, α, β represent the preset initial enhancement parameters, and L in represents the illumination value in the illumination component, num(L in ) indicates that the illumination value in the spatial video frame is L in , where n represents the number of pixels in the spatial video frame, t represents the integral variable, and dt represents the differential element of the integral.

4. The intelligent security method based on machine vision according to claim 1, characterized in that: The step of performing baseline enhancement on the security video frame sequence according to the baseline enhancement parameter to obtain a baseline enhanced image frame includes: Extracting a first security video frame from the security video frame sequence, and performing illumination component mapping on the first security video frame using the reference enhancement parameter to obtain a reference illumination component; The illumination component of the first security video frame is mapped using the following formula: Wherein, T represents the reference illumination component, u represents the normalized illumination value of the first security video frame, represents the reference enhancement parameter, D represents the illumination value range corresponding to the first security video frame, and dt represents the differential element of the integral; The first security video frame is reconstructed according to the reference illumination component to obtain a reference enhanced image frame.

5. The intelligent security method based on machine vision according to claim 1, characterized in that: The step of performing inter-frame image enhancement on the security video frame sequence according to the reference enhanced image frame to obtain an enhanced image frame sequence includes: Taking the reference enhanced image frame as a reference frame, calculating a histogram of a security video frame next to the reference frame and the reference enhanced image frame, and calculating a Euclidean distance between the histograms; When the Euclidean distance is less than a preset threshold, performing image enhancement on the next security video frame using the benchmark enhancement parameters of the benchmark enhanced image frame to obtain a second enhanced image frame, and using the second enhanced image frame as a benchmark frame; When the Euclidean distance is greater than or equal to the threshold, calculating an updated enhanced image frame corresponding to the next security video frame, and using the updated enhanced image frame as a reference frame; Return to the above step of calculating the histogram of the next security video frame of the reference frame and the reference enhanced image frame, and calculating the Euclidean distance between the histograms, until the next security video frame is the last security video frame in the security video frame sequence, to obtain an enhanced image frame sequence.

6. The intelligent security method based on machine vision according to claim 1, characterized in that: The step of performing interval difference optimization on the enhanced image frame sequence to obtain a target image frame sequence includes: Converting the enhanced image frame sequence into a grayscale image to obtain a grayscale image frame sequence; Extracting continuous differential image frames of each grayscale image frame in the grayscale image frame sequence, and calculating the median grayscale value of the grayscale image frame; Calculating an interval differential image between the differential image frames, and converting the interval differential image into a binary image according to the median of the grayscale value; Perform image detection on the binary image to obtain a target image frame, and collect the target image frames to obtain a target image frame sequence.

7. The intelligent security method based on machine vision according to claim 1, characterized in that: The calculating of the multi-scale full-dimensional features of each target image frame in the target image frame sequence comprises: Convert the target image frame into a feature vector, perform vector mapping and full connection on the feature vector, and obtain a target feature vector; Calculating the convolution attention of the preset multi-layer convolution kernel using a pre-built full-dimensional dynamic convolution module according to the target feature vector; Adjusting the attention of the multiple convolution kernels according to the convolution attention to obtain a target multi-layer convolution kernel; The target multi-layer convolution kernel is used to perform layered convolution on the feature vector to obtain multi-scale full-dimensional features of each target image frame.

8. The intelligent security method based on machine vision according to claim 1, characterized in that: The calculating the scale weight of the multi-scale full-dimensional feature includes: The scale weight of the multi-scale full-dimensional feature is calculated using the following formula: Where W represents the scale weight, w represents the preset initial weight, Resize represents the feature map size adjustment parameter, R represents the feature in the multi-scale full-dimensional feature, ε represents the preset minimum value, δ represents the activation function, and f 7×7 represents a 7×7 convolution operation, G max represents the maximum pooling operation, G ave Represents an average pooling operation.

9. The intelligent security method based on machine vision as claimed in claim 1, characterized in that: The step of normalizing and fusing the multi-scale full-dimensional features according to the scale weights to obtain target features of the target image frame includes: The multi-scale full-dimensional features are normalized and fused using the following formula: Among them, O represents the target feature, w j Represents the jth feature R in the multi-scale full-dimensional feature j The corresponding scale weight, ε represents the preset minimum value.

10. The intelligent security method based on machine vision according to claim 1, characterized in that: The performing a security warning on the target security video according to the security target state includes: Tracking the security target state to obtain a moving target state set; Detecting the motion state behavior of the motion target in the motion target state set; A security warning is performed on the target security video according to the motion state behavior.

Images