AR intelligent detection system and method based on improved small target detection algorithm

Abstract

The invention relates to an AR intelligent detection system and method based on an improved small target detection algorithm. An acquisition module acquires image information; the acquired image information is transmitted to the image processing module through the information transmitting and receiving module; the image processing module processes the transmitted data and obtains a result; the processing result is input to the interactive display unit module, and the interactive display unit module displays the processing result. According to the method, the context feature information and the residual attention mechanism module are integrated in the SSD model, compared with an original network, the new network model can utilize more shallow features and global information, a small target area can be better concerned in a detection task, and the benefit of improving the accuracy of an AR intelligent detection system is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of AR detection technology, and in particular to an AR intelligent detection system and method based on an improved small target detection algorithm. Background Art

[0002] With the development of augmented reality technology, the research on AR intelligent detection system is becoming more and more popular, and has been gradually applied to military, transportation, industry and other fields. The key to high-precision detection of various objects lies in the use of highly robust target detection algorithms. For example, in the secondary equipment inspection project of substations, manual offline operations are still the main method. Trained and experienced technicians can generally achieve a high accuracy rate when checking, but after working for a long time, due to factors such as fatigue, the detection accuracy rate decreases to a certain extent. In addition to the manpower time cost and accident probability, the detection accuracy rate is a very important reference indicator. The accuracy rate of the image processing detection based on AR intelligent detection system should be maintained at or above the accuracy rate of manual detection. The existing AR intelligent detection system can achieve high-precision detection of general target images, but in the complex scene of substation secondary circuit verification, there are a large number of small targets such as terminal blocks. Small targets are targets with a small proportion of pixels in the image, but their accuracy rate has an extremely important impact on the detection results of the entire project. The existing AR-based intelligent detection system generally adopts a single-stage small target detection algorithm SSD model.

[0003] like Figure 1 As shown in the figure, the steps of using the SSD model for target detection are: first, the original input image needs to be normalized and scaled to a fixed size as the input of the model. Secondly, a convolutional neural network (CNN) is used in the feature extraction module to extract the features of the input image. Six multi-scale feature layers can be obtained, with different sizes: 38×38×512, 19×19×1,024, 10×10×512, 5×5×256, 3×3×256 and 1×1×256. Each feature layer focuses on extracting feature information of objects of a specific scale. Deeper feature maps focus on the features of larger targets, while shallower feature maps focus on the features of small targets. For feature maps of different sizes, SSD sets prior boxes with different aspect ratios and numbers. These prior boxes can be used to predict the location and category of the target. Through two 3×3 convolutional layers, SSD predicts the class score and position offset of the default bounding box. Finally, the post-processing strategy of maximum suppression decomposition (NMS) is used to obtain the final positioning bounding box. SSD can complete target detection simultaneously in a single forward propagation. Its advantage is that it can combine detection rate and accuracy. However, in many engineering scenarios, the accuracy of small target detection does not meet the performance index requirements. Therefore, the model needs to be improved to better adapt to most working conditions.

[0004] In real scenes, small targets are difficult to identify due to different shooting angles, occlusion by non-target objects, and the influence of weather and lighting conditions. At the same time, small-sized targets lack appearance information to distinguish themselves from the background or similarities, and feature information is easily lost in shallow and deep networks, resulting in missed detections and false detections during detection. At present, the accuracy of the SSD algorithm for small target detection cannot meet engineering requirements. Therefore, how to improve the existing SSD model and improve its recognition accuracy for small target objects in complex scenes is a technical problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and propose an AR intelligent detection system and method based on an improved small target detection algorithm. By integrating contextual feature information and a residual attention mechanism module into the SSD model, the new network model can utilize more shallow features and global information compared to the original network, and can better pay attention to the small target area in the detection task, thereby achieving the benefit of improving the accuracy of the AR intelligent detection system.

[0006] The present invention solves the technical problem by adopting the following technical solutions:

[0007] The AR intelligent detection system based on the improved small target detection algorithm includes an acquisition module, an information transmission module, an image processing module and an interactive display unit module, wherein the acquisition module, the information transmission module, the image processing module and the interactive display unit module are connected in sequence, the acquisition module acquires image data and transmits the data to the image processing module through the information transmission module, the image processing module processes the transmitted information and inputs the processing result to the interactive display unit module, and the interactive display unit module displays the detection result.

[0008] Moreover, the acquisition module includes a camera, a sensor and a tracking unit. The camera is mainly used to capture the image information of the detection object. By acquiring data through the camera, the system can identify and locate small targets, and superimpose virtual information into the real scene to achieve an augmented reality effect. Sensors such as accelerometers and gyroscopes are used to detect the motion state and environmental information of the equipment, provide position, posture and motion data, and help the system to perform accurate spatial positioning and posture estimation. The tracking unit can track the position and posture of the device in real time. By combining the data from the camera and the sensor, the tracking unit can achieve high-precision tracking to ensure that the inspection object can be accurately positioned and stably displayed.

[0009] Moreover, the image processing module performs data preprocessing, feature extraction, classification and positioning on the collected information by fusing the SSD network model with context feature information and the residual attention mechanism module.

[0010] A detection method of an AR intelligent detection system based on an improved small target detection algorithm comprises the following steps:

[0011] Step 1: The acquisition module acquires image information;

[0012] Step 2: The image information is transmitted to the image processing module through the information transmission and receiving module;

[0013] Step 3: The image processing module processes the transmitted data and obtains the result;

[0014] Step 4: Input the processing result to the interactive display unit module, and the interactive display unit module displays the processing result.

[0015] Moreover, the specific implementation method of step 3 is: the image processing module performs data preprocessing, feature extraction, classification and positioning on the collected information by fusing the SSD network model with the context feature information and the residual attention mechanism module.

[0016] Moreover, the context information feature fusion module is:

[0017]

[0018] Among them, Conv represents the convolution operation, F(Conv) represents the output of the convolution, and Up represents the upsampling operation.

[0019] Moreover, the feature map output by the residual attention mechanism module is:

[0020] H i (x) = M i (x)*T i (x)

[0021] Among them, M i (x) represents a mask of the same size as the backbone output, T i (x) represents the output feature map of the trunk branch; during forward propagation, the attention mask serves as a feature selector and during backpropagation, it serves as a gradient update filter:

[0022]

[0023] Among them, θ represents the mask branch parameter, φ represents the trunk branch parameter, and the attention module is made more robust to noise through feature selection and filtering operations.

[0024] The advantages and positive effects of the present invention are:

[0025] The present invention collects image information through an acquisition module; the image information is transmitted to an image processing module through an information transmission and receiving module; the image processing module processes the transmitted acquisition and obtains a result; the processing result is input into an interactive display unit module, and the interactive display unit module displays the processing result. The present invention integrates context feature information and residual attention mechanism modules into the SSD model. Compared with the original network, the new network model can utilize more shallow features and global information, and can better focus on small target areas in detection tasks, thereby achieving the benefit of improving the accuracy of the AR intelligent detection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of an SSD network in the prior art;

[0027] Figure 2 is a system structure diagram of the present invention;

[0028] Figure 3 Schematic diagram of context information feature fusion module of the present invention;

[0029] Figure 4 Schematic diagram of the residual attention mechanism module of the present invention;

[0030] Figure 5 This is a schematic diagram of the new network model architecture after fusion of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below with reference to the accompanying drawings.

[0032] The construction idea of ​​the present invention is: since the SSD network loses some information when extracting shallow and deep features, it leads to the situation of misdetection when detecting small targets. It is proposed to fuse the SSD network model with context feature information and residual attention mechanism module, use VGG16 as the backbone network, use SPP instead of global pooling in the image output layer, and add a propagation path from deep features to shallow features. This path forwards the deep feature information, adaptively fuses the shallow feature map and the upsampled deep feature map in a recursive manner, and further focuses on the small target object in the image through the residual attention mechanism module. In this way, the shallow feature map can better extract the semantic information of the context, thereby enhancing the model's perception ability of small targets, helping to more accurately determine the position and category of small targets, and improving the accuracy of small target detection. This path mainly includes two modules: the first is the context feature fusion module, which enables the network to better use global features to identify small targets by extracting target context information. The second is the residual attention mechanism module. After extracting the context features, the attention mechanism is used to extract the global information between different channels, increase the weight of small target objects, and further strengthen the features of small targets.

[0033] AR intelligent detection system based on improved small target detection algorithm, such as Figure 2 As shown, it includes an acquisition module, an information transmission and receiving module, an image processing module and an interactive display unit module, wherein the acquisition module, the information transmission and receiving module, the image processing module and the interactive display unit module are connected in sequence, the acquisition module acquires image information and transmits the acquired information to the image processing module through the information transmission and receiving module, the image processing module processes the transmitted acquisition and inputs the processing result to the interactive display unit module, and the interactive display unit module displays the detection result.

[0034] Among them, the acquisition module includes cameras, sensors and tracking units. The camera is mainly used to capture the image information of the detection object. By acquiring data through the camera, the system can identify and locate small targets, and superimpose virtual information into the real scene to achieve augmented reality effects. Sensors such as accelerometers and gyroscopes are used to detect the motion state and environmental information of the equipment, provide position, posture and motion data, and help the system to perform accurate spatial positioning and posture estimation. The tracking unit can track the position and posture of the device in real time. By combining the data from the camera and the sensor, the tracking unit can achieve high-precision tracking to ensure that the inspection object can be accurately positioned and stably displayed.

[0035] Among them, the image processing module performs data preprocessing, feature extraction, classification and positioning on the collected information by fusing the SSD network model with contextual feature information and the residual attention mechanism module.

[0036] A detection method of an AR intelligent detection system based on an improved small target detection algorithm comprises the following steps:

[0037] Step 1: The acquisition module acquires image information.

[0038] Step 2: The image information is transmitted to the image processing module through the information transmission and receiving module.

[0039] Step 3: The image processing module processes the transmitted data and obtains the result.

[0040] like Figure 5 As shown in the figure, the image processing module performs data preprocessing, feature extraction, classification and positioning on the collected information by fusing the SSD network model with the context feature information and the residual attention mechanism module.

[0041] Among them, context feature information fusion:

[0042] (1) Extract features from feature maps at different levels. (2) Upsampling and alignment. (3) Feature fusion.

[0043] Residual Attention Mechanism:

[0044] (1) Apply residual module on the feature map. (2) Embed the attention mechanism in the residual module. (3) Weight the feature map according to the attention weight.

[0045] Combination of feature fusion and attention mechanism: The feature map after feature fusion is input into the residual attention module to further enhance the feature representation capability.

[0046] Multi-level application: Figure 5 The above fusion strategy is applied in multiple layers of the SSD network to improve the detection performance of objects of different scales.

[0047] The context information feature fusion module is as follows Figure 3 As shown, Conv9_m contains context feature information extracted using a 3*3 convolution kernel. Through the full-play average pooling step, Conv9_n and Conv10_n can obtain two different global context information extracted from Conv9 and Conv10, so that the network can use more complete global information; Conv11 is a prediction layer containing deeper semantic information. Conv9_z is generated by fusion of Conv9_m and upsampled Conv9_n, Conv10_n and Conv11, which contains deep and shallow semantic information and global context information. The upsampling operation is based on the feature map of 1*1 size through the three convolutional networks Conv9-n, Conv10-n and Conv11, and then forms a feature map with the same dimension as Conv3_1 according to the ratio of channels of different sizes:

[0048]

[0049] Among them, Conv represents the convolution operation, F(Conv) represents the output of the convolution, and Up represents the upsampling operation.

[0050] The original network only uses global average pooling to extract features, which easily leads to the weights of important features and edge features in the same channel being evenly shared, thus weakening the features of small objects. Figure 4 As shown in Figure 1, the residual attention mechanism allows the network to focus on specific parts of the image rather than the entire area. The attention mechanism is added to the SSD model, specifically by introducing a two-stage residual attention module, which is placed after conv4_3 and conv7 respectively. Each residual attention stage includes a trunk branch and a mask branch. The trunk branch consists of two residual blocks, each of which contains three convolutional layers. The mask branch generates an attention map by downsampling and upsampling with the residual connection, and then nonlinearized by the sigmoid activation function. These attention maps are multiplied with the output of the trunk branch to form the small target features that are focused on.

[0051] If the input is x, the output feature map of the main branch is T i (x); The mask branch uses a combination of bottom-up and top-down attention to obtain a mask M with the same size as the trunk output i (x).

[0052] M i (x) can be regarded as the weight of the feature map output by the main branch, and the feature map output by the attention module can be obtained as follows:

[0053] H i (x) = M i (x)*T i (x)

[0054] During forward propagation, the attention mask acts as a feature selector and during backpropagation it acts as a gradient update filter, which can be expressed as follows:

[0055]

[0056] Among them, θ represents the mask branch parameter, φ represents the trunk branch parameter, and the robustness of the attention module to noise is enhanced through feature selection and filtering operations. Finally, these focused small target features are activated by another residual block, L2 normalization, and ReLU function. In this way, the network can better focus on the small target area in the detection task and improve the detection accuracy.

[0057] Step 4: Input the processing result to the interactive display unit module, and the interactive display unit module displays the processing result.

[0058] like Figure 5 As shown in the figure, compared with the original SSD model, the improved new network model has significant advantages in the detection of small target areas and has significant innovations in algorithm technology. In the context feature fusion module, spatial pyramid pooling (SPP) is used to replace global flat pooling in the input stage, so that the network can adapt to image inputs of various sizes, better retain global information, and improve the generation speed of candidate boxes. SPP can better capture the characteristics of small targets, upsample the deep features and align them with the shallow feature maps, so that the network can obtain richer context information, while retaining the details of the target features, providing sufficient feature information for small targets, and further improving the network's recognition ability of small target features; at the same time, the residual attention mechanism module is introduced. The residual structure can avoid the gradient disappearance problem of the deep network. The attention mechanism can enable SSD to focus on the recognition of small target areas and reduce the interference of background noise, so that the improved model can adapt to complex on-site detection in engineering. In addition, the improved new model only improves the detection accuracy of small targets from the retention, extraction and utilization of the global information of the input image, and does not bring a large increase in the number of parameters. Therefore, it can also maintain a high detection speed and is suitable for real-time applications.

[0059] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. AR intelligent detection system based on improved small target detection algorithm, characterized by: It includes an acquisition module, an information transmission module, an image processing module and an interactive display unit module, wherein the acquisition module, the information transmission module, the image processing module and the interactive display unit module are connected in sequence, the acquisition module acquires image data and transmits the data to the image processing module through the information transmission module, the image processing module processes the transmitted information and inputs the processing result to the interactive display unit module, and the interactive display unit module displays the detection result.

2. The AR intelligent detection system based on the improved small target detection algorithm according to claim 1 is characterized in that: The acquisition module includes a camera, a sensor and a tracking unit. The camera is mainly used to capture image information of the detection object. By acquiring data through the camera, the system identifies and locates small targets and superimposes virtual information on the real scene to achieve an augmented reality effect. Sensors such as accelerometers and gyroscopes are used to detect the motion state and environmental information of the equipment, provide position, posture and motion data, and help the system to perform accurate spatial positioning and posture estimation. The tracking unit tracks the position and posture of the equipment in real time. By combining the data from the camera and the sensor, the tracking unit achieves high-precision tracking to ensure that the inspection object can be accurately positioned and stably displayed.

3. The AR intelligent detection system based on the improved small target detection algorithm according to claim 1 is characterized in that: The image processing module performs data preprocessing, feature extraction, classification and positioning on the collected information by fusing the SSD network model with context feature information and the residual attention mechanism module.

4. A detection method of an AR intelligent detection system based on an improved small target detection algorithm according to any one of claims 1 to 3, characterized in that: The following steps are involved: Step 1: The acquisition module acquires image information; Step 2: The image information is transmitted to the image processing module through the information transmission module; Step 3: The image processing module processes the transmitted data and obtains the result; Step 4: Input the processing result to the interactive display unit module, and the interactive display unit module displays the processing result.

5. The detection method of the AR intelligent detection system based on the improved small target detection algorithm according to claim 4 is characterized in that: The specific implementation method of step 3 is: the image processing module performs data preprocessing, feature extraction, classification and positioning on the collected information by fusing the SSD network model with the context feature information and the residual attention mechanism module.

6. The detection method of the AR intelligent detection system based on the improved small target detection algorithm according to claim 5 is characterized in that: The context information feature fusion module is: Among them, Conv represents the convolution operation, F(Conv) represents the output of the convolution, and Up represents the upsampling operation.

7. The detection method of the AR intelligent detection system based on the improved small target detection algorithm according to claim 5 is characterized in that: The feature map output by the residual attention mechanism module is: H i (x)=M i (x)*T i (x) Among them, M i (x) represents a mask of the same size as the backbone output, T i (x) represents the output feature map of the trunk branch; during forward propagation, the attention mask serves as a feature selector and during backpropagation, it serves as a gradient update filter: Among them, θ represents the mask branch parameter, φ represents the trunk branch parameter, and the attention module is made more robust to noise through feature selection and filtering operations.

Images