A target detection system based on super-resolution technology
By introducing super-resolution technology and object detection subnetwork into the object detection system, the problem of insufficient detection capabilities of small and low-resolution object is solved, and accurate detection and diversified detection modes of these targets are achieved, and remote support is supported. Intelligent management improves the overall performance of the detection system.
Patent Information
- Application Number
- CN202111089433.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-16
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-09-16
AI Technical Summary
The existing object detection system lacks the capability of small target and low resolution object detection, it is difficult to distinguish detailed features, is easy to be confused with the background, and lacks remote intelligent management functions.
A target detection system based on super-resolution technology is designed, including a target image acquisition unit, a super-resolution reconstruction unit, a target detection unit and an information supervision platform. By combining super-resolution reconstruction model and a target detection subnet, image details are improved. and local features to achieve accurate detection of small and low-resolution targets, and support remote intelligent management.
It realizes accurate detection of small targets and low-resolution targets, with fast detection rates and diverse detection modes. Users can conduct real-time remote management, improving detection capabilities in special application scenarios.
Smart Images

Figure CN113888407B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a target detection system based on super-resolution technology. Background Art
[0002] With the increasing maturity of monitoring technology, target detection systems are widely used in various industries, such as security monitoring, traffic violation detection, etc. Small targets lack discriminative details due to low resolution and are easily confused with the background, making them difficult to detect correctly. Current target detection systems often directly extract features and detect and identify the target images to be detected. Although these algorithms have excellent detection and recognition capabilities for general target detection, they are still slightly insufficient for special cases such as low-definition targets and small target detection. However, if super-resolution algorithms are combined with target detection algorithms, the detection capabilities in special application scenarios can be improved. Super-resolution technology refers to the process of using algorithms to restore images from low resolution to high resolution, which can enhance the details and local features of the image, making the recognition more accurate.
[0003] In summary, it is an urgent problem for technicians in this field to provide a target detection system based on super-resolution technology that can accurately detect small targets and low-resolution targets, has diverse detection modes, and can be remotely intelligently managed by users. Summary of the invention
[0004] In response to the above-mentioned problems and needs, this solution proposes a target detection system based on super-resolution technology, which can solve the above-mentioned technical problems by adopting the following technical solutions.
[0005] To achieve the above-mentioned object, the present invention provides the following technical solutions: A target detection system based on super-resolution technology, comprising: a target image acquisition unit, a super-resolution reconstruction unit, a target detection unit and an information supervision platform;
[0006] The target image acquisition unit is used to collect image information of the target to be detected, cache and preprocess it, and send the preprocessed image information to the super-resolution reconstruction unit;
[0007] The super-resolution reconstruction unit is used to perform super-resolution reconstruction on the input image and output the super-resolution reconstruction result to the target detection unit;
[0008] The target detection unit detects the target to be detected by using a super-resolution target detection model, and determines whether there is a cross-border or intrusion behavior by using a tripwire detection method, and issues an alarm prompt when there is a cross-border or intrusion behavior;
[0009] The information supervision platform is used to receive the target detection information and alarm information sent by the target detection unit, and to visualize the detected target and the alarm information, and to send image acquisition control information to the target image acquisition unit.
[0010] Furthermore, the target image acquisition unit includes a visual acquisition device, an electric control module and an image preprocessing module;
[0011] The visual acquisition device includes a receiving module, a camera module and an image data storage module, wherein the receiving module is used to receive an external target image to be detected through an image transmission interface, the camera module is used to collect images according to the control instructions of the electronic control module, and the image data storage module is used to classify and store the received external target image to be detected and the collected images;
[0012] The electric control module includes a zoom drive motor, a fill light group, an electronic relay, a light sensor and an intelligent pan-tilt platform. The zoom drive motor, the fill light group, the electronic relay and the light sensor are all electrically connected to the intelligent pan-tilt platform. The intelligent pan-tilt platform is used to receive image acquisition control information sent by the information supervision platform, obtain parameter information based on the image acquisition control information, and control the camera module to work according to the parameter information. The zoom drive motor is used to drive the camera module to zoom, the light sensor is used to detect ambient light information, the fill light group is used to automatically fill light when the light information is lower than a set threshold, and the electronic relay is used to control the working switch of the camera module;
[0013] The image preprocessing module comprises an image processing module to be tested, and the image processing module to be tested is used to perform size standardization, filtering processing and image noise removal through smoothing operation on the image to be tested.
[0014] Furthermore, the image preprocessing module also includes a training data processing module, which is used to divide the data set into a training set, a test set and a verification set according to a certain ratio, and randomly rotate and mirror the training set, and then downsample all images in the training set to one quarter of the original image, and restore the downsampled images to the original image size through an interpolation algorithm, so that low-definition versions of all original images in the training set can be obtained. After the preprocessed training data, each image will have two images, a low-definition image and an original image, and the image will be annotated to mark the category and coordinates of the target object in the image.
[0015] Furthermore, the super-resolution reconstruction unit includes an input module and a super-resolution reconstruction module. The input module obtains a low-resolution target image to be detected and transmits it to the super-resolution reconstruction module. The super-resolution reconstruction module includes a three-layer convolutional neural network. The first layer of convolution extracts features of the input image through a 9*9*64 convolution kernel, the second layer of convolution performs nonlinear mapping on the feature map generated by the first layer of convolution through a 1*1*32 convolution kernel, and the third layer of convolution reconstructs the mapped feature map through a 5*5*1 convolution kernel to obtain a reconstructed high-resolution image.
[0016] Furthermore, the super-resolution reconstruction network is trained and optimized by using a first loss function and the training set, wherein the first loss function is a mean square error function. Among them, N represents the number of pixels, a i represents the i-th pixel of the predicted high-resolution image, b i Represents the i-th pixel of the standard high-resolution image.
[0017] Furthermore, the super-resolution target detection model includes an image input layer and a target detection sub-network layer. The reconstructed image output by the super-resolution reconstruction unit is input into the image input layer. The target detection sub-network layer adopts a FSSD-based target detection network for target detection classification and target positioning. The target detection sub-network layer includes an SSD network with the last three convolutional layers removed. The SSD network with the last three convolutional layers removed is used to extract features. The extracted feature maps of different sizes are fused through a feature fusion module to obtain feature maps of the same size. Then, each feature map is merged, and the merged feature map is input into an inverted pyramid network to obtain a feature pyramid. Feature maps of different sizes are generated through the inverted pyramid module for target detection. Softmax classification and position regression can be performed simultaneously using multiple feature maps, and a non-maximum suppression algorithm is used to obtain the best target detection frame.
[0018] Furthermore, the feature fusion module adopts a sub-pixel rearrangement method to convert a small-size feature map into a large-size feature map. The sub-pixel rearrangement method includes inputting an image of size p*q into a convolutional network, performing multi-layer convolution operations, obtaining n*n feature maps of size p*q, and rearranging the feature maps by pixels to synthesize them into a feature map of size np*nq.
[0019] Furthermore, the target detection subnetwork is trained and optimized by using a second loss function and the training set, wherein the second loss function is Among them, K is the number of default boxes that match the real box, L x is the classification loss of the SSD network, Ly is the positioning loss of the SSD network, and α is the proportional coefficient used to adjust the classification loss and positioning loss.
[0020] Furthermore, the information supervision platform includes an information storage module, a visual display module and an interactive module. The information storage module is used to receive the target detection information and alarm information sent by the target detection unit, and classify and store the information according to a storage list. The information storage module includes a storage list and an alarm data management module. The alarm data management module is used to record and periodically count the historical data of alarm events. The interactive module is used to input remote information image acquisition control instructions, parameters or external target image data to be detected; the visual display module is used to display the target detection location, time, target name, and coordinate position information. The visual display module includes an information processing server and an intelligent display screen. The interactive module and the intelligent display screen are both connected to the information processing server.
[0021] It can be seen from the above technical solutions that the beneficial effects of the present invention are: the present invention can accurately detect small targets and low-resolution targets, has a fast detection rate, and has various detection modes. Users can perform real-time target detection or remote image data transmission detection, and users can remotely intelligently manage the system.
[0022] In addition to the objects, features and advantages described above, the best embodiments for implementing the present invention will be described in more detail below with reference to the accompanying drawings so that the features and advantages of the present invention can be easily understood. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present invention or the description of the prior art, wherein the drawings are only used to show some embodiments of the present invention, rather than limiting all embodiments of the present invention thereto.
[0024] Figure 1 The figure is a schematic diagram of the composition structure of a target detection system based on super-resolution technology of the present invention.
[0025] Figure 2 It is a schematic diagram of the composition structure of the target image acquisition unit in the present invention.
[0026] Figure 3 It is a schematic diagram of the composition structure of the information supervision platform in the present invention.
[0027] Figure 4 Schematic diagram of the target detection process in this embodiment. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the technical solution of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described in conjunction with the drawings of specific embodiments of the present invention. The same figure marks in the drawings represent the same parts. It should be noted that the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] The present invention provides a target detection system based on super-resolution technology that can accurately detect small targets and low-resolution targets, has a high detection rate, has various detection modes, and can be remotely managed intelligently by users. Figures 1 to 4 As shown, the system includes: a target image acquisition unit, a super-resolution reconstruction unit, a target detection unit and an information supervision platform.
[0030] The target image acquisition unit is used to collect image information of the target to be detected, cache and preprocess it, and send the preprocessed image information to the super-resolution reconstruction unit.
[0031] The target image acquisition unit includes a visual acquisition device, an electric control module and an image preprocessing module; the visual acquisition device includes a receiving module, a camera module and an image data storage module, the receiving module is used to receive an external target image to be detected through an image transmission interface, the camera module is used to collect images according to the control instructions of the electric control module, and the image data storage module is used to classify and store the received external target image to be detected and the collected image; the electric control module includes a zoom drive motor, a fill light group, an electronic relay, a light sensor and an intelligent pan-tilt platform, the zoom drive motor, the fill light group, the electronic relay and the light sensor are all connected to the intelligent pan-tilt platform The intelligent pan-tilt head is electrically connected to receive image acquisition control information sent by the information supervision platform, and parameter information is obtained based on the image acquisition control information. The camera module is controlled to work according to the parameter information. The zoom drive motor is used to drive the camera module to zoom. The light sensor is used to detect ambient light information. The fill light group is used to automatically fill light when the light information is lower than a set threshold. The electronic relay is used to control the working switch of the camera module. The image preprocessing module includes an image processing module to be tested, and the image processing module to be tested is used to perform size standardization, filtering and smoothing operations on the image to be detected. Remove image noise.
[0032] Specifically, the image preprocessing module also includes a training data processing module, which is used to divide the data set into a training set, a test set and a validation set according to a certain ratio, and perform random angle rotation and mirroring on the training set, and then downsample all images in the training set to one-fourth of the original image, and restore the downsampled image to the original image size through an interpolation algorithm, so that a low-definition version of all the original images in the training set can be obtained. After the preprocessed training data, each image will have two images, a low-definition image and an original image, and the image will be annotated to mark the category and coordinates of the target object in the image. The data set uses the PASCAL VOC data set.
[0033] Directly extracting features and detecting and identifying small targets or low-resolution target images will cause misidentification and reduce the accuracy of detection. Before the input image enters the target detection network, the application first passes the input image through a super-resolution sub-network to enhance the local details and clarity of the input image, and then inputs the feature-enhanced image into the target detection sub-network for target detection. In this system, the super-resolution reconstruction unit is used to perform super-resolution reconstruction on the input image and output the super-resolution reconstruction result to the target detection unit.
[0034] The super-resolution reconstruction unit includes an input module and a super-resolution reconstruction module. The input module obtains a low-resolution target image to be detected and transmits it to the super-resolution reconstruction module. The super-resolution reconstruction module includes a three-layer convolutional neural network. The first layer of convolution extracts features of the input image through a 9*9*64 convolution kernel, the second layer of convolution performs nonlinear mapping on the feature map generated by the first layer of convolution through a 1*1*32 convolution kernel, and the third layer of convolution reconstructs the mapped feature map through a 5*5*1 convolution kernel to obtain a reconstructed high-resolution image.
[0035] In this embodiment, the super-resolution reconstruction network is trained and optimized by using a first loss function and the training set, wherein the first loss function is a mean square error function. Among them, N represents the number of pixels, a i represents the i-th pixel of the predicted high-resolution image, b i Represents the i-th pixel of the standard high-resolution image.
[0036] The target detection unit detects the target to be detected through the super-resolution target detection model, and determines whether there is a cross-border or intrusion behavior through the tripwire detection method, and issues an alarm prompt when there is a cross-border or intrusion behavior. After the target is detected, the target is identified and tracked, and a cordon is set in a specific monitoring area to determine whether the target has crossed the boundary. When there is a cross-border behavior, the management personnel are reminded by voice reminder or sound and light alarm.
[0037] Specifically, the super-resolution target detection model includes an image input layer and a target detection sub-network layer. The reconstructed image output by the super-resolution reconstruction unit is input into the image input layer. The target detection sub-network layer adopts a target detection network based on FSSD to perform target detection classification and target positioning. The target detection sub-network layer includes an SSD network with the last three convolutional layers removed. The SSD network with the last three convolutional layers removed is used to extract features. The extracted feature maps of different sizes are fused through a feature fusion module to obtain feature maps of the same size. Then, each feature map is merged, and the merged feature map is input into an inverted pyramid network to obtain a feature pyramid. Feature maps of different sizes are generated through the inverted pyramid module for target detection. Softmax classification and position regression can be performed simultaneously using multiple feature maps, and a non-maximum suppression algorithm is used to obtain the best target detection frame. Among them, the feature fusion module adopts a sub-pixel rearrangement method to convert a small-size feature map into a large-size feature map. The sub-pixel rearrangement method includes inputting an image of size p*q into a convolutional network, performing multi-layer convolution operations, obtaining n*n feature maps of size p*q, and rearranging the feature maps by pixels to synthesize them into a feature map of size np*nq. Sub-pixels are information parts that cannot be captured by the camera. In theory, the pixels should be continuous, but due to the limitation of sensor accuracy, the pixels that the camera can capture will have a minimum size, and the part smaller than this size cannot be captured. In this embodiment, a sub-pixel rearrangement scheme is used to optimize the upsampling operation, and the feature fusion module of the FSSD network is designed so that the model can more fully learn the feature information of feature maps of each size and different input images.
[0038] In this embodiment, the target detection subnetwork is trained and optimized by using the second loss function and the training set, and the second loss function is Among them, K is the number of default boxes that match the real box, L x is the classification loss of the SSD network, L y is the positioning loss of the SSD network, and α is the proportional coefficient used to adjust the classification loss and positioning loss.
[0039] The information supervision platform is used to receive the target detection information and alarm information sent by the target detection unit, and to visualize the detected target and alarm information, and to send image acquisition control information to the target image acquisition unit. The information supervision platform includes an information storage module, a visualization display module and an interaction module. The information storage module is used to receive the target detection information and alarm information sent by the target detection unit, and to classify and store the information according to a storage list. The information storage module includes a storage list and an alarm data management module. The alarm data management module is used to record and periodically count the historical data of alarm events. The interaction module is used to input remote information image acquisition control instructions, parameters or external target image data to be detected; the visualization display module is used to display the target detection location, time, target name, and coordinate position information. The visualization display module includes an information processing server and an intelligent display screen. The interaction module and the intelligent display screen are both connected to the information processing server.
[0040] In this embodiment, the user can control the target detection process of the target detection system through remote management, which can not only realize real-time detection of on-site targets, but also remotely collect image data and then send the image data to the target detection system for target detection, so as to better realize multi-scene target recognition.
[0041] It should be noted that the embodiments described in the present invention are only preferred ways to implement the present invention, and any obvious modifications that belong to the overall concept of the present invention should fall within the protection scope of the present invention.
Claims
1. A target detection system based on super-resolution technology, characterized in that: include: Target image acquisition unit, super-resolution reconstruction unit, target detection unit and information supervision platform; The target image acquisition unit is used to collect image information of the target to be detected, cache and preprocess it, and send the preprocessed image information to the super-resolution reconstruction unit; The super-resolution reconstruction unit is used to perform super-resolution reconstruction on the input image and output the super-resolution reconstruction result to the target detection unit; The target detection unit detects the target to be detected by using a super-resolution target detection model, and determines whether there is a cross-border or intrusion behavior by using a tripwire detection method, and issues an alarm prompt when there is a cross-border or intrusion behavior; The information monitoring platform is used to receive the target detection information and alarm information sent by the target detection unit, and to visualize the detected target and the alarm information, and to send image acquisition control information to the target image acquisition unit; The super-resolution reconstruction unit includes an input module and a super-resolution reconstruction module. The input module obtains a low-resolution target image to be detected and transmits it to the super-resolution reconstruction module. The super-resolution reconstruction module includes a three-layer convolutional neural network. The first layer of convolution extracts features of the input image through a 9*9*64 convolution kernel. The second layer of convolution performs nonlinear mapping on the feature map generated by the first layer of convolution through a 1*1*32 convolution kernel. The third layer of convolution reconstructs the mapped feature map through a 5*5*1 convolution kernel to obtain a reconstructed high-resolution image. The super-resolution reconstruction network is trained and optimized by using a first loss function and the training set, wherein the first loss function is a mean square error function , where N represents the number of pixels. represents the i-th pixel of the predicted high-resolution image, Represents the i-th pixel of the standard high-resolution image; The super-resolution target detection model includes an image input layer and a target detection sub-network layer. The reconstructed image output by the super-resolution reconstruction unit is input into the image input layer. The target detection sub-network layer uses a target detection network based on FSSD to perform target detection classification and target positioning. The target detection sub-network layer includes an SSD network with the last three convolutional layers removed. The SSD network with the last three convolutional layers removed is used to extract features. The extracted feature maps of different sizes are fused through a feature fusion module to obtain feature maps of the same size. Then, each feature map is merged, and the merged feature map is input into an inverted pyramid network to obtain a feature pyramid. Feature maps of different sizes are generated through the inverted pyramid for target detection. Softmax classification and position regression can be performed simultaneously using multiple feature maps, and a non-maximum suppression algorithm is used to obtain the best target detection frame. The feature fusion module converts a small-size feature map into a large-size feature map by using a sub-pixel rearrangement method, wherein the sub-pixel rearrangement method includes inputting an image of size p*q into a convolutional network, performing multi-layer convolution operations, obtaining n*n feature maps of size p*q, and rearranging the feature maps by pixels to synthesize them into a feature map of size np*nq; The target detection subnetwork is trained and optimized by using a second loss function and the training set, where the second loss function is: , where K is the number of default boxes that match the real box, is the classification loss of the SSD network, is the positioning loss of the SSD network, is the proportional coefficient used to adjust the classification loss and positioning loss.
2. The target detection system based on super-resolution technology according to claim 1, characterized in that: The target image acquisition unit includes a visual acquisition device, an electric control module and an image preprocessing module; The visual acquisition device includes a receiving module, a camera module and an image data storage module, wherein the receiving module is used to receive an external target image to be detected through an image transmission interface, the camera module is used to collect images according to the control instructions of the electronic control module, and the image data storage module is used to classify and store the received external target image to be detected and the collected images; The electric control module includes a zoom drive motor, a fill light group, an electronic relay, a light sensor and an intelligent pan-tilt platform. The zoom drive motor, the fill light group, the electronic relay and the light sensor are all electrically connected to the intelligent pan-tilt platform. The intelligent pan-tilt platform is used to receive image acquisition control information sent by the information supervision platform, obtain parameter information based on the image acquisition control information, and control the camera module to work according to the parameter information. The zoom drive motor is used to drive the camera module to zoom, the light sensor is used to detect ambient light information, the fill light group is used to automatically fill light when the light information is lower than a set threshold, and the electronic relay is used to control the working switch of the camera module; The image preprocessing module comprises an image processing module to be tested, and the image processing module to be tested is used to perform size standardization, filtering processing and image noise removal through smoothing operation on the image to be tested.
3. The target detection system based on super-resolution technology according to claim 2, characterized in that: The image preprocessing module also includes a training data processing module, which is used to divide the data set into a training set, a test set and a verification set according to a certain ratio, and randomly rotate and mirror the training set, and then downsample all images in the training set to one-fourth of the original image, and restore the downsampled images to the original image size through an interpolation algorithm, so that low-definition versions of all original images in the training set can be obtained. After the preprocessed training data, each image will have two images, a low-definition image and an original image, and the image will be annotated to mark the category and coordinates of the target object in the image.
4. The target detection system based on super-resolution technology according to claim 1, characterized in that: The information supervision platform includes an information storage module, a visual display module and an interactive module. The information storage module is used to receive the target detection information and alarm information sent by the target detection unit, and classify and store the information according to a storage list. The information storage module includes a storage list and an alarm data management module. The alarm data management module is used to record and periodically count the historical data of alarm events. The interactive module is used to input remote information image acquisition control instructions, parameters or external target image data to be detected; the visual display module is used to display the target detection location, time, target name, and coordinate position information. The visual display module includes an information processing server and an intelligent display screen. The interactive module and the intelligent display screen are both connected to the information processing server.
Citation Information
Patent Citations
Image enhancement-based power transmission line fine hardware defect detection method and system
CN111524135A
Target detection system and method based on super-resolution reconstruction
CN113139896A
Target detection method and device based on multi-scale image
CN113221925A
Image super-resolution and model training method and apparatus, electronic device, and medium
WO2021121108A1
