A method and system for depth detection with improved inter-frame feature extraction

Abstract

The application provides a deep detection method and system for improving inter-frame feature extraction, adopts a double-network scheme, namely an intra-frame sub-network and an inter-frame sub-network, and uses the difference between intra-frame features and inter-frame features to detect network videos, wherein the intra-frame sub-network uses multi-stage hollow convolution layers with different expansion rates to extract intra-frame features, the inter-frame sub-network uses a three-layer structure and point convolution to extract inter-frame features, and finally, the Euclidean distance and the contrast loss function between the two features are calculated to complete detection and recognition, and the problem of incomplete extraction and insufficient precision of inter-frame features in the prior art is overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a deep detection method and system for improved inter-frame feature extraction. BACKGROUND

[0002] With the rapid development of computer vision technology, image depth information detection is of great significance in many application fields, such as automatic driving, robot navigation, virtual reality, etc. However, the existing technology often has the problems of incomplete extraction and insufficient precision when extracting inter-frame features of images, which affects the accuracy of depth information detection. Therefore, it is necessary to develop a new deep detection method and system for improved inter-frame feature extraction to improve the precision of feature extraction and depth detection.

[0003] Therefore, there is an urgent need for a targeted deep detection method and system for improved inter-frame feature extraction. SUMMARY

[0004] The purpose of the present application is to provide a deep detection method and system for improved inter-frame feature extraction to solve the above problems.

[0005] In a first aspect, the present application provides a deep detection method for improved inter-frame feature extraction, comprising:

[0006] Collecting current raw data in the network and inputting the raw data into an intra-frame subnetwork and an inter-frame subnetwork respectively;

[0007] The intra-frame subnetwork comprises three convolution layers, all of which are dilated convolution layers with different expansion rates, wherein the expansion rates of the first, second and third convolution layers are obtained by sequentially stacking on the basis of the initial expansion rate;

[0008] After the intra-frame subnetwork receives the raw data, it performs dilated convolution operation according to the level of the convolution layer to obtain different convolution values, and the input of the next level convolution layer is the output of the previous level convolution layer. Finally, the intra-frame features of the raw data are extracted from the convolution values output by the third level convolution layer;

[0009] The inter-frame subnetwork is composed of a three-layer structure built for the data frames of the raw data and a point convolution; the size of each layer in the three-layer structure is half of the size of the previous layer, and each layer is used to calculate the least square polynomial expansion of all point positions within the layer size to obtain the fitting point position map of the layer, which is passed to the next layer; the above operation is repeated until the final fitting point position map is output by the third layer to obtain the intermediate feature map;

[0010] The intermediate feature map is fused by point convolution to obtain the final output feature map, and the inter-frame features of the raw data are extracted;

[0011] calculating the Euclidean distance between the intra-frame feature and the inter-frame feature, and then calculating a contrast loss function between the intra-frame feature and the inter-frame feature according to the Euclidean distance;

[0012] when the value of the contrast loss is less than or equal to a threshold value, inputting the original data into a recognition model to obtain a classification result;

[0013] when the value of the contrast loss is greater than the threshold value, issuing a prompt information;

[0014] controlling according to the classification result or the prompt information.

[0015] In a second aspect, the application provides a deep detection system for improving inter-frame feature extraction, which comprises:

[0016] a collection unit configured to collect current original data in a network and input the original data into an intra-frame sub-network and an inter-frame sub-network respectively;

[0017] the intra-frame sub-network unit comprises three convolution layers, each of which is a dilated convolution layer with different expansion rates, wherein the expansion rates of the first, second and third convolution layers are obtained by sequentially stacking on the basis of an initial expansion rate; after receiving the original data, the intra-frame sub-network performs dilated convolution operation according to the levels of the convolution layers to obtain different convolution values, the input of the next level convolution layer is the output of the previous level convolution layer, and finally the intra-frame feature of the original data is extracted from the convolution value output by the third level convolution layer;

[0018] the inter-frame sub-network unit comprises a three-layer structure built for data frames of the original data and a point convolution; the size of each layer in the three-layer structure is half of the size of the previous layer, and each layer is used to calculate the least square polynomial expansion of all point positions in the size range of the layer to obtain a fitting point position map of the layer, which is passed to the next layer; the above operation is repeated until the final fitting point position map is output by the third layer to obtain an intermediate feature map; the intermediate feature map is fused by point convolution to obtain a final output feature map, and the inter-frame feature of the original data is extracted;

[0019] a calculation unit configured to calculate the Euclidean distance between the intra-frame feature and the inter-frame feature, and then calculate a contrast loss function between the intra-frame feature and the inter-frame feature according to the Euclidean distance;

[0020] a determination unit configured to determine that the detection is qualified when the value of the contrast loss is less than or equal to a threshold value, input the original data into a recognition model to obtain a classification result; and determine that the detection is abnormal when the value of the contrast loss is greater than the threshold value, and issue a prompt information;

[0021] an execution unit configured to control according to the classification result or the prompt information.

[0022] In a third aspect, the present application provides a depth detection system for improving inter-frame feature extraction, comprising a processor and a memory:

[0023] The memory is configured to store program code and transmit the program code to the processor.

[0024] The processor is configured to execute the method according to any one of the four possibilities of the first aspect according to the instructions in the program code.

[0025] In a fourth aspect, the present application provides a computer readable storage medium for storing program code, wherein the program code is configured to execute the method according to any one of the four possibilities of the first aspect.

[0026] Advantages

[0027] The present application provides a depth detection method and system for improving inter-frame feature extraction, which adopts a dual-network scheme, i.e., an intra-frame sub-network and an inter-frame sub-network, and uses the difference between intra-frame features and inter-frame features to detect network videos, wherein the intra-frame sub-network adopts multi-level dilated convolution layers with different expansion rates to extract intra-frame features, the inter-frame sub-network adopts a three-layer structure and point convolution to extract inter-frame features, and finally the Euclidean distance and the contrast loss function between the two features are calculated to complete detection and recognition, thereby overcoming the problem of incomplete extraction and insufficient precision of inter-frame features in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0029] Figure 1 A general flowchart of the depth detection method for improving inter-frame feature extraction of the present application;

[0030] Figure 2 An architecture diagram of the depth detection system for improving inter-frame feature extraction of the present application. DETAILED DESCRIPTION

[0031] The preferred embodiments of the present application will be described in detail below with reference to the drawings, so that the advantages and features of the present application can be more easily understood by those skilled in the art, and the protection scope of the present application can be more clearly defined.

[0032] Figure 1 A general flowchart of the depth detection method for improving inter-frame feature extraction provided by the present application, the method comprising:

[0033] The current raw data in the collection network is input into the intra-frame sub-network and the inter-frame sub-network respectively;

[0034] The intra-frame sub-network comprises three convolution layers, and each of the three convolution layers is a dilated convolution layer with different dilated rates, wherein the dilated rates of the first, second and third convolution layers are obtained by sequentially stacking on the basis of an initial dilated rate;

[0035] After the intra-frame sub-network receives the raw data, the dilated convolution operation is performed according to the levels of the convolution layers to obtain different convolution values, the input of the next level convolution layer is the output of the previous level convolution layer, and finally the intra-frame features of the raw data are extracted from the convolution values output by the third level convolution layer;

[0036] Directly using a common convolution layer has good detection capability for small targets, but the effect is not ideal for medium and large targets. At the same time, the limited receptive range cannot capture larger area semantic information, so a convolution structure of multiple dilated convolution layers is used here.

[0037] However, directly using a dilated convolution layer will also bring many problems, and the same dilated convolution layer stacking will produce a grid effect, resulting in a segmentation result containing too much irrelevant semantic information.

[0038] Therefore, a convolution layer structure composed of three dilated convolution layers with different dilated rates is used, and the dilated rates of the three dilated convolution layers are sequentially stacked, for example, the dilated rate of the first dilated convolution layer is an initial dilated rate N, the dilated rate of the second dilated convolution layer is obtained by adding 1 to N, and the dilated rate of the third dilated convolution layer is obtained by adding 2 to N.

[0039] The combination of different dilated rates can make the information range captured by each convolution different, ensuring the expansion of the receptive field while avoiding the loss of local information.

[0040] The inter-frame sub-network is composed of a three-layer structure built for the data frames of the raw data and a point convolution; the size of each layer in the three-layer structure is half of the size of the previous layer, and each layer is used to calculate the least square polynomial expansion of all point positions in the size range of the layer to obtain a fitting point position map of the layer, which is transmitted to the next layer; the above operation is repeated until the final fitting point position map is output by the third layer to obtain an intermediate feature map;

[0041] The intermediate feature map is fused by point convolution to obtain a final output feature map, and the inter-frame features of the raw data are extracted;

[0042] The three-layer structure can quickly capture objects with large movement amplitudes. The additional point convolution can be used to extract the features and accurate position information of small targets, so as to fuse the subsequent features.

[0043] calculating the Euclidean distance between the intra-frame features and the inter-frame features, and calculating a contrast loss function between the intra-frame features and the inter-frame features according to the Euclidean distance;

[0044] When the value of the contrast loss is less than or equal to a threshold value, it is determined to be qualified for detection, and the original data is input into the recognition model to obtain a classification result;

[0045] When the value of the contrast loss is greater than the threshold value, it is determined to be abnormal for detection, and a prompt information is sent out;

[0046] According to the classification result or the prompt information, management and control are performed.

[0047] When the network video is qualified, the contrast loss is small, and when the network video is abnormal, the contrast loss is large.

[0048] It is worth noting that the loss function needs to be optimized in some application scenarios. Since the multi-level dilated convolution layer is used in the process of extracting the intra-frame features, if the loss function is not properly optimized, the expansion of the receptive field to capture spatial information may not fully play a role.

[0049] A plurality of samples can be introduced into the loss function, and the number of intersections W of elements in the plurality of samples is calculated. The first coefficient is obtained by multiplying W by 2 and then dividing by the sum A of the number of elements in each sample.

[0050] The first coefficient and the result of the loss function are normalized and accumulated to obtain an optimized contrast loss value, which is compared with the threshold value.

[0051] In some preferred embodiments, the fusion of the intermediate feature map through point convolution includes:

[0052] The channel of the data frame of the original data is M, and after processing by the three-layer structure, an intermediate feature map with a size of D*D*M is obtained, wherein M and D are positive integers, M is the number of channels, and D is the size of the single dimension of the data frame of the original data;

[0053] The intermediate feature map is further processed by N point convolutions with a size of 1*1 to obtain an output feature map with a size of D*D*N, wherein N is a positive integer, and N is the number of point convolutions.

[0054] When training the identification model, an entropy loss function is minimized by a back propagation method to avoid over saturation, and when the accuracy of the identification model meets the threshold requirement, it indicates that the training of the identification model is completed. Then it can be used for data verification.

[0055] In some preferred embodiments, the least square polynomial expansion of all point positions in the layer size range comprises: determining the weight of each point position according to the value and coordinates of other point positions in the vicinity of each point position, and then expanding the coordinates of the point position by a polynomial.

[0056] In some preferred embodiments, the vicinity refers to setting a (2n+1)*(2n+1) field in the layer size range, and using (2n+1) square point positions in the field as sample points for fitting by the least square method, where n is an integer.

[0057] After fitting, the flow capture between the three layers of structures can be further strengthened by parameter vector calculation and local fuzzing processing, so that the data structure of the captured inter-frame vector corresponds to the data structure of the intra-frame vector one by one.

[0058] Figure 2 The architecture diagram of the deep detection system for improved inter-frame feature extraction provided in the present application comprises:

[0059] The acquisition unit is configured to acquire the original data in the network and input the original data into the intra-frame sub-network and the inter-frame sub-network respectively;

[0060] The intra-frame sub-network unit comprises three convolution layers, and each of the three convolution layers is a dilated convolution layer with different expansion rates, wherein the expansion rates of the first, second and third convolution layers are obtained by sequentially stacking on the basis of the initial expansion rate; after receiving the original data, the intra-frame sub-network performs dilated convolution operation according to the levels of the convolution layers to obtain different convolution values, the input of the next level convolution layer is the output of the previous level convolution layer, and finally the intra-frame features of the original data are extracted from the convolution values output by the third level convolution layer;

[0061] The inter-frame sub-network unit comprises a three-layer structure built for the data frames of the original data and a point convolution; the size of each layer in the three-layer structure is half of the size of the previous layer, each layer is configured to calculate the least square polynomial expansion of all point positions in the layer size range to obtain a fitting point position map of the layer, and the fitting point position map is transmitted to the next layer; the above operation is repeated until the final fitting point position map is output by the third layer to obtain an intermediate feature map; the intermediate feature map is fused by point convolution to obtain a final output feature map, and the inter-frame features of the original data are extracted.

[0062] a calculation unit configured to calculate an Euclidean distance between the intra-frame feature and the inter-frame feature, and calculate a contrast loss function between the intra-frame feature and the inter-frame feature according to the Euclidean distance;

[0063] a determination unit configured to determine that the detection is qualified when the value of the contrast loss is less than or equal to a threshold value, input the original data into the recognition model to obtain a classification result; and determine that the detection is abnormal when the value of the contrast loss is greater than the threshold value, and send a prompt information;

[0064] an execution unit configured to perform management and control according to the classification result or the prompt information.

[0065] The present application provides a deep detection system for improving inter-frame feature extraction, the system comprising: the system comprises a processor and a memory:

[0066] The memory is configured to store program code and transmit the program code to the processor.

[0067] The processor is configured to execute the method according to the instructions in the program code.

[0068] The present application provides a computer readable storage medium for storing program code, the program code being used to execute the method according to any one of all embodiments of the first aspect.

[0069] In specific implementations, the present application further provides a computer storage medium, wherein the computer storage medium can store a program, and the program can include some or all steps in various embodiments of the present application when executed. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM) and the like.

[0070] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be realized by means of software and necessary general hardware platforms. Based on such understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk and the like, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device and the like) to execute the method described in various embodiments or some parts of the embodiments of the present application.

[0071] The same or similar parts among various embodiments of the present application can be referred to each other. Especially, for the embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0072] The above-described embodiments of the present application are not intended to define the scope of the present application.

Claims

1. A deep detection method of improving inter-frame feature extraction, characterized in that, The method comprises: Collecting the current original data in the network, and inputting the original data into the intra-frame sub-network and the inter-frame sub-network respectively; The intra-frame sub-network comprises three convolution layers, and each of the three convolution layers is a dilated convolution layer with different expansion rates, wherein the expansion rates of the first, second and third convolution layers are obtained by sequentially stacking on the basis of the initial expansion rate; After the intra-frame sub-network receives the original data, the intra-frame sub-network performs dilated convolution operation according to the levels of the convolution layers to obtain different convolution values, the input of the next level convolution layer is the output of the previous level convolution layer, and finally the intra-frame features of the original data are extracted from the convolution values output by the third level convolution layer; The inter-frame sub-network comprises a three-layer structure built for the data frames of the original data and a point convolution; each layer of the three-layer structure has a size of half of the size of the previous layer, each layer is used to calculate the least square polynomial expansion of all point positions in the size range of the layer to obtain a fitting point position map of the layer, and the fitting point position map is transmitted to the next layer; the above operation is repeated until the final fitting point position map is output by the third layer to obtain an intermediate feature map; The intermediate feature map is fused by point convolution to obtain a final output feature map, and the inter-frame features of the original data are extracted; The Euclidean distance between the intra-frame features and the inter-frame features is calculated, and then a contrast loss function between the intra-frame features and the inter-frame features is calculated according to the Euclidean distance; When the value of the contrast loss is less than or equal to a threshold value, the original data is input into a recognition model to obtain a classification result; When the value of the contrast loss is greater than the threshold value, a prompt information is sent out; The classification result or the prompt information is used for management and control.

2. The method of claim 1, wherein: The fusion of the intermediate feature map by point convolution comprises: The channels of the data frames of the original data are M, and after the processing of the three-layer structure, an intermediate feature map with a size of D*D*M is obtained, wherein M and D are positive integers, M is the number of channels, and D is the size of a single dimension of the data frames of the original data; The intermediate feature map is further processed by N point convolutions with a size of 1*1 to obtain an output feature map with a size of D*D*N, wherein N is a positive integer, and N is the number of point convolutions.

3. The method of claim 1, wherein: The least square polynomial expansion of all point positions in the layer size range comprises: determining the weight of each point position according to the values and coordinates of other point positions in the vicinity of each point position, and expanding the coordinates of the point position by a polynomial.

4. The method of claim 3, wherein: The vicinity refers to a (2n+1)*(2n+1) field set in the layer size range, and (2n+1) square point positions in the field are used as sample points for fitting by the least square method, and n is an integer.

5. A depth detection system that improves inter-frame feature extraction, characterized in that, The system comprises: A collection unit configured to collect the current original data in the network, and input the original data into the intra-frame sub-network and the inter-frame sub-network respectively; The intra-frame sub-network unit includes three convolution layers, each being a hollow convolution layer with different expansion rates, wherein the expansion rates of the first, second and third convolution layers are sequentially superimposed on the basis of the initial expansion rate; after receiving the original data, the intra-frame sub-network unit performs a hollow convolution operation according to the levels of the convolution layers to obtain different convolution values, the input of the next-level convolution layer is the output of the previous-level convolution layer, and finally the intra-frame feature of the original data is extracted from the convolution value output by the third-level convolution layer; The inter-frame sub-network unit is composed of a three-layer structure built for the data frames of the original data and a point convolution; the size of each layer in the three-layer structure is half of the size of the previous layer, each layer is used to calculate the least square polynomial expansion of all point positions in the size range of the layer to obtain a fitting point position map of the layer, which is passed to the next layer; the above operation is repeated until the final fitting point position map is output by the third layer to obtain an intermediate feature map; the intermediate feature map is fused by point convolution to obtain a final output feature map, and the inter-frame feature of the original data is extracted; The calculation unit is configured to calculate the Euclidean distance between the intra-frame feature and the inter-frame feature, and then calculate a contrast loss function between the intra-frame feature and the inter-frame feature according to the Euclidean distance; The determination unit is configured to determine that the detection is qualified when the value of the contrast loss is less than or equal to a threshold value, input the original data into a recognition model to obtain a classification result; and determine that the detection is abnormal when the value of the contrast loss is greater than the threshold value, and send a prompt information; The execution unit is configured to perform management and control according to the classification result or the prompt information.

6. A depth detection system that improves inter-frame feature extraction, characterized in that, The system includes a processor and a memory: The memory is configured to store program code and transmit the program code to the processor; The processor is configured to execute instructions in the program code to implement the method of any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store program code, and the program code is configured to implement the method of any one of claims 1-4. The computer-readable storage medium is configured to store program code, and the program code is configured to implement the method of any one of claims 1-4.

Images