Target detection method, system and device based on full inference neural network

Through the inter-layer difference attention extraction method and capsule inference subnetwork, the problem of poor detection performance of DETR in small objects is solved, the detection accuracy and performance are improved, and the target information is matched and updated with multi-head attention and Hungarian algorithm to achieve Effective detection of target categories and locations.

CN114067125BActive Publication Date: 2024-11-26HANGZHOU XINHE SHENGSHI TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111357503.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-16
Publication Date
2024-11-26
Estimated Expiration
2041-11-16

AI Technical Summary

Technical Problem

DETR has poor performance in small objects detection, and since there is no multi-scale feature map, the calculation complexity increases, affecting the detection effect.

Method used

The inter-layer difference value attention extraction method is used to enhance the difference information of large-scale feature maps and small-scale feature maps, and a capsule inference subnet is constructed to mine target position and category information, and the multi-head attention and Hungarian algorithm are used to match and Parameter update.

Benefits of technology

It improves the accuracy of small object detection of DETR, makes full use of large-scale feature map information, avoids the burden of model training, and enhances the target detection performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114067125B_ABST
    Figure CN114067125B_ABST
Patent Text Reader

Abstract

The present invention discloses a target detection method based on a full-reasoning neural network, comprising the following steps: obtaining a target image database; training and verifying a full-reasoning target detection network model based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model; mining target position and category information based on a capsule reasoning sub-network model; inputting a target image to be detected into the trained full-reasoning target detection network model to obtain target information to be detected. A difference attention extraction method is adopted to enhance feature map information, by extracting a large-resolution feature map and comparing it with the information of a small-resolution map, extracting the information filtered out between them, and merging the large-resolution feature map and the small-resolution feature map in the large-resolution feature map for subsequent target detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method, system and device based on a full inference neural network. Background Art

[0002] In the prior art, the goal of object detection is to predict a set of boxes and their corresponding category labels for each object of interest. Current object detection algorithms mainly complete the task in an indirect way by defining regression and classification problems on a large number of proposals, anchors, or window centers. These algorithms remove duplicate prediction boxes by adopting post-processing methods, and their performance is easily affected by the post-processing steps, the design of anchors, and the design of the matching algorithm between the target box and the anchor.

[0003] The newly proposed DETR based on Transformer regards detection as a set prediction problem, simplifying the overall process of target detection. It does not have the operations such as anchor, label assign, nms, etc. in the current mainstream target detectors, and is truly end-to-end. It mainly adopts the encoder-decoder structure of Transformer, which has been very successful in sequence prediction tasks. The self-attention mechanism in Transformer explicitly defines the interaction between elements in the sequence, so it can be used to constrain the prediction results of the set. DETR can predict multiple targets at one time and uniquely match the prediction results and annotation information through the loss function. The entire structure consists only of Transformer and ResNet. At the same time, in the final binary matching process, it can be seen that when the generated prediction result does not contain the target category of interest, it matches the empty.

[0004] DETR is a common technology for image detection. It regards detection as a set prediction problem, which simplifies the overall process of target detection. It does not have operations such as anchor, label assign, and nms in the current mainstream target detectors, and achieves true end-to-end. Although DETR has an interesting design and good performance, it also has its own problems:

[0005] In the final classification and positioning process of the network, the stacking of fully connected layers is used. However, the use of full connection for classification and positioning destroys the relative position relationship of the image, which has a great impact on the target detection method.

[0006] DETR performs poorly in small object detection. Existing detectors usually have multi-scale features. Small object targets are usually detected on high-resolution feature maps, while DETR does not use multi-scale features for detection. The main reason is that high-resolution feature maps will increase unacceptable computational complexity for DETR. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a target detection method, system and device based on a full inference neural network.

[0008] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0009] A target detection method based on a full inference neural network comprises the following steps:

[0010] Obtain target image database;

[0011] The full-reasoning target detection network model is trained and verified based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model;

[0012] The target image to be detected is input into the trained full-inference target detection network model to obtain the target information to be detected.

[0013] As an implementable method, the initial improvement of the initial target detection network model based on the difference attention algorithm includes the following steps:

[0014] Based on the initial target detection network model, the feature maps of the last two layers, the first feature map and the second feature map, are obtained, wherein the first feature map is the feature map of the second-to-last layer, and the second feature map is the feature map of the first-to-last layer;

[0015] Perform upsampling and convolution processing on the second feature map to obtain a third feature map;

[0016] Compare the third feature map with the first feature map to obtain a corresponding difference, and perform sigmoid processing on the difference as a mask result;

[0017] Based on the mask result, the fourth feature map is obtained by adding the product to the first feature map;

[0018] The fourth feature map and the second feature map are summed to obtain a fifth feature map.

[0019] As an implementable method, the target location and category information mining based on the capsule reasoning sub-network model includes the following steps:

[0020] The fifth feature map is used as the feature map of the last layer and converted into a capsule, which is used as the input capsule. The input capsule is expanded to obtain the output capsule.

[0021] Based on multi-head attention, the correlation between the output capsule and the input capsule after affine transformation is obtained;

[0022] The Hungarian algorithm is used for binary matching and matching results are obtained, and loss results are calculated respectively. Gradient calculation and parameter update are performed according to the loss results, wherein the loss results include GIOU loss, cross entropy loss and matching loss.

[0023] As an implementable method, the method of obtaining the correlation between the output capsule after affine transformation and the input capsule based on multi-head attention includes the following steps:

[0024] The number of capsules corresponding to the input capsule is used as the head of the multi-head attention mechanism. Along the dimension where the number of capsules is located, multi-head attention is used to calculate the correlation between the output capsule after affine transformation and the input capsule, and obtain the probability value of the output capsule being assigned to the input capsule, which is achieved through the following formula:

[0025]

[0026] Among them, S is the query vector, set to DCaps, Z is the key and value vector, set to Pcaps, and the scale factor is the inner product value, d represents the dimension of the key value vector;

[0027] The output capsule is used as the lower-level feature map, and the semantic information contained in the image is inferred based on the attention routing transfer method, where the semantic information at least includes the number of objects in the image, the category information of each object, and the position of each object in the image.

[0028] As an implementable method, the input capsule is expanded to obtain the output capsule, specifically as follows:

[0029] The dimensional shape of the input capsule is expressed as [bs, 100, head, dim], and the dimensional shape of the output capsule is expressed as [bs, 100, head, dim, classes + coord], where 100 represents the number of output capsules, classes + coord represents the dimension of the output capsule, head represents the number of input capsules, and dim represents the dimension of the input capsule.

[0030] A target detection system based on a full inference neural network, comprising a data acquisition module, a model acquisition module and a result detection module;

[0031] The data acquisition module is used to acquire a target image database;

[0032] The model acquisition module is used to train and verify the full-reasoning target detection network model based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model;

[0033] The result detection module is used to input the target image to be detected into the trained full-inference target detection network model to obtain the target information to be detected.

[0034] As an implementable embodiment, the model acquisition module is configured as follows:

[0035] Based on the initial target detection network model, the feature maps of the last two layers, the first feature map and the second feature map, are obtained, wherein the first feature map is the feature map of the second-to-last layer, and the second feature map is the feature map of the first-to-last layer;

[0036] Perform upsampling and convolution processing on the second feature map to obtain a third feature map;

[0037] Compare the third feature map with the first feature map to obtain a corresponding difference, and perform sigmoid processing on the difference as a mask result;

[0038] Based on the mask result, the fourth feature map is obtained by adding the product to the first feature map;

[0039] The fourth feature map and the second feature map are summed to obtain a fifth feature map.

[0040] As an implementable embodiment, the model acquisition module is configured as follows:

[0041] The fifth feature map is used as the feature map of the last layer and converted into a capsule, which is used as the input capsule. The input capsule is expanded to obtain the output capsule.

[0042] Based on multi-head attention, the correlation between the output capsule and the input capsule after affine transformation is obtained;

[0043] The Hungarian algorithm is used for binary matching and matching results are obtained, and loss results are calculated respectively. Gradient calculation and parameter update are performed according to the loss results, wherein the loss results include GIOU loss, cross entropy loss and matching loss.

[0044] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0045] Obtain target image database;

[0046] The full-reasoning target detection network model is trained and verified based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model;

[0047] The target image to be detected is input into the trained full-inference target detection network model to obtain the target information to be detected.

[0048] A target detection device based on a full inference neural network comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following method steps when executing the computer program: acquiring a target image database;

[0049] The full-reasoning target detection network model is trained and verified based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model;

[0050] The target image to be detected is input into the trained full-inference target detection network model to obtain the target information to be detected.

[0051] The present invention has significant technical effects due to the adoption of the above technical solution:

[0052] The present invention adopts a difference attention extraction method to realize the enhancement of feature map information, mainly by extracting large-scale feature maps in the backbone network, extracting large-resolution feature maps, and comparing them with the information of small-resolution maps, extracting the filtered information between them, and using it as a mask to further enhance the information of small targets contained in the feature map in the large-resolution feature map. Finally, the large and small resolution feature maps are merged for subsequent target detection.

[0053] Inspired by the capsule network, the present invention constructs a capsule inference subnetwork to extract the category and location information of the target based on capsule construction and attention information transmission. The capsule is used to mine the target components, and the target category and location are predicted by using attention information transmission, and finally the category and location information of the target are obtained;

[0054] The present invention proposes a target detection network framework that can realize the full reasoning mode. Based on the DETR target detection network, the framework first proposes an inter-layer difference attention method to enhance feature information, thereby improving the small target detection accuracy of the DETR method; then, in view of the special requirements of the target detection task for the relative position relationship between targets, a capsule reasoning sub-network is constructed to realize target position and category information mining, further improving the detection performance;

[0055] To solve the problem of poor small target detection performance, a layer-by-layer difference attention extraction method is proposed to fully utilize large-scale feature map information while avoiding increasing the burden of model training. This method is used to enhance feature map information and improve the detection performance of the model.

[0056] In order to improve the detection performance, the relative position relationship and category relationship of the target are fully utilized to build an inference capsule inference sub-network, and the attention routing is used to extract global information to realize the inference of target category and position information. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 It is a schematic flow diagram of the method of the present invention;

[0059] Figure 2 It is a structural schematic diagram of the system of the present invention;

[0060] Figure 3 This is a schematic diagram of the inter-layer difference attention implementation principle of the present invention. DETAILED DESCRIPTION

[0061] The present invention is further described in detail below in conjunction with embodiments. The following embodiments are for explanation of the present invention but the present invention is not limited to the following embodiments.

[0062] Embodiment 1:

[0063] A target detection method based on full inference neural network, such as Figure 1 As shown, the following steps are included:

[0064] S100, acquiring a target image database;

[0065] S200, training and verifying the full-reasoning target detection network model based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model;

[0066] S300: Input the target image to be detected into the trained full-inference target detection network model to obtain the target information to be detected.

[0067] In one embodiment, the initial improvement of the initial target detection network model based on the difference attention algorithm includes the following steps:

[0068] Based on the initial target detection network model, the feature maps of the last two layers, the first feature map and the second feature map, are obtained, wherein the first feature map is the feature map of the second-to-last layer, and the second feature map is the feature map of the first-to-last layer;

[0069] Perform upsampling and convolution processing on the second feature map to obtain a third feature map;

[0070] Compare the third feature map with the first feature map to obtain a corresponding difference, and perform sigmoid processing on the difference as a mask result;

[0071] Based on the mask result, the fourth feature map is obtained by adding the product to the first feature map;

[0072] The fourth feature map and the second feature map are summed to obtain a fifth feature map.

[0073] The above steps can be found in the attached Figure 3As shown, in order to make full use of large-scale feature map information and avoid increasing the burden of model training, the present invention proposes an inter-layer difference attention extraction method for realizing the enhancement of feature map information, mainly by mining the difference between large-scale feature maps and small-scale feature maps, and using this difference to realize the enhancement of feature map information, and finally improve the detection performance of the model. The specific implementation details are as follows: the input image passes through the CNN convolutional neural network to obtain the feature maps of the last two layers of different networks, recorded as: the feature F1 of the l-1 layer, the feature F2 of the l layer; then, after upsampling and convolution processing, F2 is compared with F1 to obtain the difference, then the difference represents the information lost in the optimization process from the previous layer to the next layer. The deeper the network, the more obvious the loss of small target information, so the difference information can be considered to contain more small target information; then, we use the difference result as the mask result of the small target after sigmoid activation, and add it to F1 as the product to enhance the small target information in F1; finally, we sum F4 and F2 obtained after processing F1 to realize the application of difference attention in network structure.

[0074] In one embodiment, the target location and category information mining based on the capsule reasoning sub-network model includes the following steps:

[0075] The fifth feature map is used as the feature map of the last layer and converted into a capsule. This is used as the input capsule. The input capsule is expanded to obtain the output capsule. Specifically, the dimensional shape of the input capsule is expressed as [bs, 100, head, dim], and the dimensional shape of the output capsule is expressed as [bs, 100, head, dim, classes + coord], where 100 represents the number of output capsules, classes + coord represents the dimension of the output capsule, head represents the number of input capsules, and dim represents the dimension of the input capsule;

[0076] Based on multi-head attention, the correlation between the output capsule and the input capsule after affine transformation is obtained;

[0077] The Hungarian algorithm is used for binary matching and matching results are obtained, and loss results are calculated respectively. Gradient calculation and parameter update are performed according to the loss results, wherein the loss results include GIOU loss, cross entropy loss and matching loss.

[0078] Specifically, the method of obtaining the correlation between the output capsule after affine transformation and the input capsule based on multi-head attention includes the following steps:

[0079] The number of capsules corresponding to the input capsule is used as the head of the multi-head attention mechanism. Along the dimension where the number of capsules is located, multi-head attention is used to calculate the correlation between the output capsule after affine transformation and the input capsule, and obtain the probability value of the output capsule being assigned to the input capsule, which is achieved through the following formula:

[0080]

[0081] Among them, S is the query vector, set to DCaps, Z is the key and value vector, set to Pcaps, and the scale factor is the inner product value, d represents the dimension of the key value vector;

[0082] The output capsule is used as the lower-level feature map, and the semantic information contained in the image is inferred based on the attention routing transfer method, where the semantic information at least includes the number of objects in the image, the category information of each object, and the position of each object in the image.

[0083] That is to say, in this embodiment, the present invention actually transforms the target detection problem into a semantic analysis problem of the image, that is: how many targets are there in the image, which category they belong to, and where are they located. To realize this idea, we construct an inference capsule inference subnetwork inspired by the capsule network, in which the number of targets can be realized through the classic Transformer codec, and the extraction of the target category and position information is realized by constructing a capsule structure and transferring attention information. The specific details are as follows: In the target detection method, Transformer is used to realize the reasoning process from a single image to the number of targets, but the subsequent acquisition of the target category and position adopts a multi-layer fully connected form. In order to introduce the relative position information and category relationship of the target, global information is captured during the reasoning process to enhance the performance of target detection. We borrowed the construction idea of ​​the capsule network and adopted the information reasoning method of the capsule to realize the full reasoning construction method of target detection, that is, in the form of set prediction, the description of the image is converted into: how many targets are there in the image, what category each target belongs to, and what is the position of each target in the image. Here, the initial target detection network model can choose Transformer, and the specific implementation is as follows:

[0084] The feature representation obtained by the Transformer encoder is capsule-converted to obtain PCaps representation. Then the Feat feature can be converted into capsule representation PCaps with a size of [bs, 100, head, dim], where head represents the number of capsules and dim represents the vector representation of each capsule, which is used to represent the posture, texture, direction, etc. of the capsule. At this point, the PCaps representation is obtained.

[0085] With PCaps feature map as the lower layer, the semantic information contained in the image is inferred through attention routing, including: how many targets are there in the image, what category each target belongs to, and what is the position of each target in the image, represented as digital capsules DCaps, with a size of [bs, 100, classes + coord], where classes represents the number of target categories, and coord represents the number of coordinates of the target box, with a value of 4. The implementation method of attention routing is as follows: First, the dimension shape of PCaps is [bs, 100, head, dim], and it is expanded to [bs, 100, head, dim, classes + coord], where the number and dimension of the output capsules are 100 and classes + coord respectively, and the number and dimension of the input capsules are head and dim respectively. In order to obtain the probability value of the lower-layer capsule being assigned to the upper-layer capsule, the number of capsules corresponding to the input capsule is used as the head of the multi-head attention mechanism. Along the dimension where the head is located, multi-head attention is used to calculate the correlation between the output capsule and the input capsule after affine transformation, so as to realize the information transmission from input to output. The calculation is shown in formula (1).

[0086]

[0087] Among them, S is the query vector, which is set to DCaps here, Z is the key and value vector, which is set to Pcaps here, and the scale factor It is for adjustment to avoid too large inner product value, and d is the dimension of the key value vector.

[0088] For the above output results, the Hungarian algorithm is used for binary matching to obtain the matching results, and the GIOU loss, cross entropy loss and matching loss are calculated respectively. Gradient calculation and parameter update are performed according to the loss results. Finally, the full reasoning target detection network model is obtained and this model can be updated in subsequent applications. After obtaining the full reasoning target detection network model, it can be used to detect objects in the image.

[0089] Embodiment 2:

[0090] A target detection system based on a full inference neural network includes a data acquisition module 100, a model acquisition module 200 and a result detection module 300;

[0091] The data acquisition module 100 is used to acquire a target image database;

[0092] The model acquisition module 200 is used to train and verify the full-reasoning target detection network model based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model;

[0093] The result detection module 300 is used to input the target image to be detected into the trained full-inference target detection network model to obtain the target information to be detected.

[0094] The model acquisition module 200 is configured as follows:

[0095] Based on the initial target detection network model, the feature maps of the last two layers, the first feature map and the second feature map, are obtained, wherein the first feature map is the feature map of the second-to-last layer, and the second feature map is the feature map of the first-to-last layer;

[0096] Perform upsampling and convolution processing on the second feature map to obtain a third feature map;

[0097] Compare the third feature map with the first feature map to obtain a corresponding difference, and perform sigmoid processing on the difference as a mask result;

[0098] Based on the mask result, the fourth feature map is obtained by adding the product to the first feature map;

[0099] The fourth feature map and the second feature map are summed to obtain a fifth feature map.

[0100] The model acquisition module 200 is configured as follows:

[0101] The fifth feature map is used as the feature map of the last layer and converted into a capsule, which is used as the input capsule. The input capsule is expanded to obtain the output capsule.

[0102] Based on multi-head attention, the correlation between the output capsule and the input capsule after affine transformation is obtained;

[0103] The Hungarian algorithm is used for binary matching and matching results are obtained, and loss results are calculated respectively. Gradient calculation and parameter update are performed according to the loss results, wherein the loss results include GIOU loss, cross entropy loss and matching loss.

[0104] Embodiment 3:

[0105] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0106] S100, acquiring a target image database;

[0107] S200, training and verifying the full-reasoning target detection network model based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model;

[0108] S300: Input the target image to be detected into the trained full-inference target detection network model to obtain the target information to be detected.

[0109] The present invention adopts a difference attention extraction method to realize the enhancement of feature map information, mainly by extracting large-scale feature maps in the backbone network, extracting large-resolution feature maps, and comparing them with the information of small-resolution maps, extracting the filtered information between them, and using it as a mask to further enhance the information of small targets contained in the feature map in the large-resolution feature map. Finally, the large and small resolution feature maps are merged for subsequent target detection.

[0110] Inspired by the capsule network, the present invention constructs a capsule inference subnetwork to extract the category and location information of the target based on capsule construction and attention information transmission. The capsule is used to mine the target components, and the target category and location are predicted by using attention information transmission, and finally the category and location information of the target are obtained;

[0111] The present invention proposes a target detection network framework that can realize the full reasoning mode. Based on the DETR target detection network, the framework first proposes an inter-layer difference attention method to enhance feature information, thereby improving the small target detection accuracy of the DETR method; then, in view of the special requirements of the target detection task for the relative position relationship between targets, a capsule reasoning sub-network is constructed to realize target position and category information mining, further improving the detection performance;

[0112] To solve the problem of poor small target detection performance, a layer-by-layer difference attention extraction method is proposed to fully utilize large-scale feature map information while avoiding increasing the burden of model training. This method is used to enhance feature map information and improve the detection performance of the model.

[0113] In order to improve the detection performance, the relative position relationship and category relationship of the target are fully utilized to build an inference capsule inference sub-network, and the attention routing is used to extract global information to realize the inference of target category and position information.

[0114] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0115] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0119] It should be noted that:

[0120] The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0121] In addition, it should be noted that the specific embodiments described in this specification may have different functional expressions and names in the programs. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. Technicians in the technical field to which the present invention belongs can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A target detection method based on a full inference neural network, characterized in that: The following steps are involved: Obtain target image database; The full-reasoning target detection network model is trained and verified based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model; Input the target image to be detected into the trained full-reasoning target detection network model to obtain the target information to be detected; The initial improvement of the initial target detection network model based on the difference attention algorithm includes the following steps: Based on the initial target detection network model, the feature maps of the last two layers, the first feature map and the second feature map, are obtained, wherein the first feature map is the feature map of the second-to-last layer, and the second feature map is the feature map of the first-to-last layer; Perform upsampling and convolution processing on the second feature map to obtain a third feature map; Compare the third feature map with the first feature map to obtain a corresponding difference, and perform sigmoid processing on the difference as a mask result; Based on the mask result, the fourth feature map is obtained by adding the product to the first feature map; The fourth feature map and the second feature map are summed to obtain a fifth feature map.

2. The target detection method based on full inference neural network according to claim 1 is characterized in that: The target location and category information mining based on the capsule reasoning sub-network model includes the following steps: The fifth feature map is used as the feature map of the last layer and converted into a capsule, which is used as the input capsule. The input capsule is expanded to obtain the output capsule. Based on multi-head attention, the correlation between the output capsule and the input capsule after affine transformation is obtained; The Hungarian algorithm is used for binary matching and matching results are obtained, and loss results are calculated respectively. Gradient calculation and parameter update are performed according to the loss results, wherein the loss results include GIOU loss, cross entropy loss and matching loss.

3. The target detection method based on full inference neural network according to claim 2 is characterized in that: The method of obtaining the correlation between the output capsule after affine transformation and the input capsule based on multi-head attention includes the following steps: The number of capsules corresponding to the input capsule is used as the head of the multi-head attention mechanism. Along the dimension where the number of capsules is located, multi-head attention is used to calculate the correlation between the output capsule after affine transformation and the input capsule, and obtain the probability value of the output capsule being assigned to the input capsule, which is achieved through the following formula: in, is the query vector, set to , For key and value vectors, set to , scale factor is the inner product value, Represents the dimension of the key-value vector; The output capsule is used as the lower-level feature map, and the semantic information contained in the image is inferred based on the attention routing transfer method, where the semantic information at least includes the number of objects in the image, the category information of each object, and the position of each object in the image.

4. The target detection method based on full inference neural network according to claim 3 is characterized in that: The input capsule is expanded to obtain the output capsule, as follows: The dimensional shape of the input capsule is represented as , then the dimensional shape of the output capsule is expressed as , where 100 represents the number of output capsules, represents the dimension of the output capsule, Indicates the number of input capsules, Represents the dimension of the input capsule.

5. A target detection system based on a full inference neural network, characterized in that: It includes a data acquisition module, a model acquisition module and a result detection module; The data acquisition module is used to acquire a target image database; The model acquisition module is used to train and verify the full-reasoning target detection network model based on the target image database to obtain a trained full-reasoning target detection network model, wherein the full-reasoning target detection network model is obtained by obtaining an initial target detection network model, performing initial improvement on the initial target detection network model based on a difference attention algorithm, and mining target position and category information based on a capsule reasoning sub-network model; The result detection module is used to input the target image to be detected into the trained full-reasoning target detection network model to obtain the target information to be detected; Wherein, the model acquisition module is set to: Based on the initial target detection network model, the feature maps of the last two layers, the first feature map and the second feature map, are obtained, wherein the first feature map is the feature map of the second-to-last layer, and the second feature map is the feature map of the first-to-last layer; Perform upsampling and convolution processing on the second feature map to obtain a third feature map; Compare the third feature map with the first feature map to obtain a corresponding difference, and perform sigmoid processing on the difference as a mask result; Based on the mask result, the fourth feature map is obtained by adding the product to the first feature map; The fourth feature map and the second feature map are summed to obtain a fifth feature map.

6. The target detection system based on full inference neural network according to claim 5 is characterized in that: The model acquisition module is configured as follows: The fifth feature map is used as the feature map of the last layer and converted into a capsule, which is used as the input capsule. The input capsule is expanded to obtain the output capsule. Based on multi-head attention, the correlation between the output capsule and the input capsule after affine transformation is obtained; The Hungarian algorithm is used for binary matching and matching results are obtained, and loss results are calculated respectively. Gradient calculation and parameter update are performed according to the loss results, wherein the loss results include GIOU loss, cross entropy loss and matching loss.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

8. A target detection device based on a full inference neural network, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.