Small target detection method and system based on multiple focusing and magnification of camera

Abstract

The invention is suitable for the technical field of video target detection, and provides a small target detection method and system based on multiple focusing and magnification of a camera, and the technical idea of the invention is as follows: an original scene image is subjected to primary recognition processing through a target detection model, a suspected target region is recognized, and the region with the highest confidence coefficient is subjected to secondary recognition processing; image details are enhanced through primary focusing amplification, then secondary recognition processing is performed through a target detection model, the confidence coefficient is calculated again, whether secondary focusing amplification is needed or not is judged through area screening, third recognition processing is performed through the target detection model after secondary focusing amplification, and then area screening is performed. The accurate detection and filtering of the small target are realized through multiple times of focusing amplification in combination with a preset area threshold value filtering condition. The method improves the detection precision and efficiency of the small target, reduces the false alarm rate, and has a wide application prospect and an important practical value.

Description

Technical Field

[0001] The present invention belongs to the technical field of video target detection, and in particular relates to a small target detection method and system based on multiple focusing and magnification of a camera. Background Art

[0002] Video object detection technology is an important branch of artificial intelligence and is widely used. In a surveillance system, cameras can be used to identify moving objects to trigger intrusion alarms. However, not all moving objects mean an intrusion. In many cases, small objects such as small animals may trigger an alarm, which can lead to a large number of false alarms and increase the workload of security personnel.

[0003] However, no algorithm can detect targets with 100% accuracy, especially small targets identified by algorithms, which are often difficult for even humans to distinguish. Further detection of such small targets is not very valuable and will only increase the amount of calculation and human workload.

[0004] Therefore, the current image recognition method is used to further distinguish what kind of object the recognition result is, and then determine whether it is a small target. This is inefficient, so a more efficient method is needed to eliminate small targets in the recognition results. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a small target detection method and system based on multiple focusing and magnification of a camera, aiming to solve the technical problems of low accuracy and efficiency and high false alarm rate in identifying small targets in existing direct image recognition methods.

[0006] The present invention adopts the following technical solution: On the one hand, the small target detection method based on multiple focusing and magnification of the camera comprises the following steps: Step S1, capturing a scene image by a camera with an automatic zoom function; Step S2: Processing the scene image with a pre-trained target detection model for preliminary screening to identify a series of suspected target areas; Step S3, sorting all suspected target areas according to their confidence levels, selecting the suspected target area with the highest confidence level, and setting it as the highest area; Step S4, controlling the camera to zoom in and focus on the highest area to obtain an image of the highest area; Step S5: Use the target detection model to process the highest region image. If the suspected target region can still be identified, select the suspected target region with the highest confidence as the highest region and calculate its area. Step S6: If the area of ​​the highest region is greater than or equal to the area threshold, the current highest region image is used as the detection result; if the area of ​​the highest region is less than the area threshold, the camera is controlled to zoom in and focus on the highest region again, and the target detection model is used for processing; Step S7: If the suspected target area can be identified, filter the area whose area is smaller than the area threshold, and use the detected image obtained after filtering as the detection result.

[0007] Furthermore, in step S2, after processing the image, the target detection model outputs a series of bounding boxes of suspected target areas and the confidence of each bounding box. The bounding box is a rectangular box surrounding the edge of the detected suspected target, and the confidence indicates the probability that the target detection model believes that the bounding box does contain the target.

[0008] Furthermore, in step S4 and step S6, the method of controlling the camera to zoom in and focus on the highest area is to use the bounding box of the highest area as the focusing center, send the bounding box information to the camera recognition control unit, and focus the camera to the focusing center through the relationship between the image and the coordinates to obtain the highest area image.

[0009] Furthermore, in step S5 and step S6, when the target detection model processes the highest area image, if there is no target area in the image, the current focusing is canceled and no alarm is generated; if the target type of the target identified by the focus center position changes, or the target cannot be identified by the focus center position, the current focusing is canceled and no alarm is generated; if the confidence level of the target identified by the focus center position drops below the threshold, it is also regarded as having no target area, and the current focusing is canceled and no alarm is generated.

[0010] Furthermore, in step S6 and step S7, after the detection results are obtained, the target is continuously monitored according to the current camera parameters, and an alarm is generated after the set number of image evidence is reached, and the original scene image before focusing and magnification is pushed to the background.

[0011] Furthermore, if there are multiple suspected target areas with the highest confidence after the target detection model processes the image, the multiple suspected target areas are magnified and focused in turn.

[0012] Furthermore, if the target detection model identifies multiple types of targets after processing the image, for each type of target, the suspected target area with the highest confidence is selected and processed in accordance with steps S2-S7.

[0013] On the other hand, the small target detection system based on multiple focusing and magnification of the camera includes a camera and a recognition control unit. The camera has an automatic zoom function and is used to capture scene images; the recognition control unit is used to execute the small target detection method.

[0014] The beneficial effects of the present invention are as follows: the present invention performs image processing through a target detection model, identifies the suspected target area therein, enhances the image details through focusing and magnification for the area with the highest confidence, and then performs secondary processing through the target detection model again, calculates the confidence and focusing and magnification again, and achieves accurate detection and filtering of small targets through such multiple focusing and magnification, combined with a preset area threshold filtering condition. The present invention improves the detection accuracy and efficiency of small targets, reduces the false alarm rate, and has broad application prospects and important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of a small target detection method based on multiple focusing and magnification of a camera provided by an embodiment of the present invention; Figure 2 This is an example image of the original scene image after being processed by the model; Figure 3 This is an example of an image after being processed by the model after focusing and magnifying once; Figure 4 This is an example of an image after secondary focusing and magnification and model processing. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0017] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0018] Embodiment 1: like Figure 1 As shown, the small target detection method based on multiple focusing and magnification of the camera provided in this embodiment includes the following steps: Step S1: Capturing a scene image by using a camera with an automatic zoom function.

[0019] The camera described in this embodiment is a monitoring camera with a network function, that is, a network camera, which can monitor outdoor images in real time and capture video frames in real time.

[0020] Step S2: Processing the scene image with a pre-trained target detection model for preliminary screening to identify a series of suspected target areas; After acquiring the scene image, use the pre-trained object detection model to process this scene image. Object detection models are usually based on deep learning architectures, such as convolutional neural networks (CNNs), which can learn and recognize multiple types of objects in images. The model object detection model scans the entire image and looks for areas that match the features learned in the training data. For example, the object detection model samples Faster-RCNN (Faster Region Proposal Network, which is the evolution and improvement of Region Proposal Network (R-CNN), Fast Region Proposal Network (Fast R-CNN) and Mask Region-based Convolutional Neural Network (Mask R-CNN)), YOLO or any other effective object detection algorithm.

[0021] In this step, after processing the image, the target detection model outputs a series of bounding boxes of suspected target areas and the confidence scores of each bounding box. The bounding box is a rectangular box around the edge of the suspected target detected, and the confidence score indicates the probability that the target detection model believes that the bounding box does contain the target.

[0022] for example Figure 2 The specific image shown is a real-life photo. After being processed by the model, a suspected target area can be identified in the upper right part and framed by a bounding box.

[0023] Step S3: sort all suspected target areas according to their confidence levels, select the suspected target area with the highest confidence level, and set it as the highest area.

[0024] In order to determine the most noteworthy area in the image, since the model outputs multiple bounding boxes of suspected target areas, this step selects the central area that needs to be initially focused on based on the confidence of the bounding box, that is, selects the suspected target area for initial focus. When implemented, the process is as follows: S31. Select the highest confidence: sort the confidences of all suspected target areas from high to low, and select the bounding box corresponding to the highest confidence, such as determining the bounding box with the highest confidence by the following calculation formula: box = boxes[argmax(conf)]; The argmax(conf) function returns the index of the maximum value in the confidence array, while the boxes array contains the bounding boxes of all suspected targets. Therefore, box ultimately points to the bounding box with the highest confidence, which is the area that is most likely to be the real target.

[0025] S32, determine the focus center area: Based on the selected highest confidence bounding box, this area can be regarded as the preliminary focus center in the current image. This step provides key information for subsequent detailed analysis or processing, such as further target classification, attribute recognition or target tracking.

[0026] In addition, this embodiment needs to consider two situations: If there are multiple suspected target areas with the highest confidence after the target detection model processes the image, the multiple suspected target areas are enlarged and focused in turn.

[0027] There are multiple suspected target areas with the highest confidence here, which means that there are multiple bounding boxes with the same or very close confidence (very close can be approximately regarded as the same), for example, two, and the two bounding boxes need to be processed successively using the subsequent method.

[0028] Second, if the target detection model identifies multiple types of targets after processing the image, for each type of target, the suspected target area with the highest confidence is taken and processed in accordance with the subsequent steps S2-S7.

[0029] There may be multiple types of targets that need to be monitored. Therefore, if the model scans the image and identifies two or more targets that need to be monitored, each type of target is processed in the order of the first step. That is, the suspected target area with the highest confidence level is selected for the first type of target, and then the subsequent focus and magnification are continued for re-identification. After the first type of target is identified, the second type of suspected target area with the highest confidence level is selected and identified and processed in the same way.

[0030] Step S4, controlling the camera to zoom in and focus on the highest area to obtain an image of the highest area.

[0031] After obtaining the bounding box with the highest confidence, the bounding box is used as the focus center and the bounding box information is sent to the recognition control unit of the camera. According to the image and ptz coordinate relationship in the camera manufacturer (sdkSoftware Development Kit, software development kit), the image is focused and enlarged to the bounding box with the highest confidence obtained in the previous step, and the image is acquired again, achieving the first focus enlargement. In addition, it should be noted that the focus enlargement does not mean enlarging the bounding box area to cover the entire lens, but only enlarging the bounding box area as the focus center of the image. The magnification can be fixed, or the focal length of the camera can be gradually increased to meet the clarity requirements and obtain image details that meet the resolution requirements.

[0032] for example Figure 3 The image shown is after a focus magnification. Figure 2The bounding box area in is focused to the center of the image.

[0033] Step S5: Use the target detection model to process the highest region image. If the suspected target region can still be identified, the suspected target region with the highest confidence is also selected as the highest region and its area is calculated.

[0034] The target detection model is used again to process the image after the aggregation and enlargement. If the suspected target area can be identified, the bounding boxes and confidence levels of all suspected target areas can also be obtained. Figure 3 The bounding box shown in the figure is the suspected target area identified. Similarly, the suspected target area with the highest confidence is selected again.

[0035] Of course, after model processing, if there is no target area in the image, the current focus is canceled and no alarm is generated; if the target type of the target identified at the focus center position changes, or the target cannot be identified at the focus center position, the current focus is canceled and no alarm is generated; if the confidence of the target identified at the focus center position drops below the threshold, it is also regarded as a target-free area, and the current focus is canceled and no alarm is generated. In this embodiment, since the initial target is small and easy to be misidentified during the first model processing, after focusing and amplifying, a secondary model processing feature comparison will be performed to determine whether it is a target that needs to be identified. If the type is identified to be changed and it is not a target that needs to be alarmed, for example, the target type is identified as smoke during the first model processing, and after focusing and amplifying, the target type is identified as cloud during the second model processing, it can be considered that the target type has changed, and it is directly filtered, and the current focus is canceled without alarming. If the target disappears after identification, it is basically because the initial target identification is inaccurate. After amplification, it is determined to be a wrong target and will not continue to be identified, and the current focus is directly canceled without alarming. If the confidence level of the identified target is too low after the secondary model processing, that is, less than the threshold, it can be determined that the suspected target area is not the target to be identified. It is also directly filtered and the current focus is canceled without alarming.

[0036] For a focused image, after selecting the bounding box with the maximum confidence, the area of ​​the bounding box needs to be calculated.

[0037] Step S6: If the area of ​​the highest region is greater than or equal to the area threshold, the current highest region image is used as the detection result; if the area of ​​the highest region is less than the area threshold, the camera is controlled to zoom in and focus on the highest region again, and the target detection model is used for processing.

[0038] If the area of ​​the maximum confidence bounding box is greater than or equal to the area threshold that the target detection model can stably identify, it means that the recognition result of the current focused and enlarged image after being processed by the model is credible, and the current image is used as the detection result. In addition, the current camera parameters can be further maintained to continue monitoring the target, and an alarm will be generated after the set number of image evidence is reached, and the detection results together with the original scene image before focusing and enlarging are pushed to the background.

[0039] If the area of ​​the maximum confidence bounding box is smaller than the area threshold, it means that the image after the first focus and magnification is still not very reliable after the model recognition. Therefore, this step further uses the current maximum confidence bounding box as the focus center to control the camera to focus and magnify again. Specifically, the maximum confidence bounding box information is sent to the camera recognition control unit, and the camera is focused to the focus center through the relationship between the image and the coordinates to obtain the highest area image. The following is an example of an image after the second focus and magnification: Figure 4 As shown, it then goes through target recognition model recognition processing, which is the third model processing.

[0040] Step S7: If the suspected target area can be identified, filter the area whose area is smaller than the area threshold, and use the detected image obtained after filtering as the detection result.

[0041] Similar to step S5, in this step, after the image after secondary focusing and magnification is processed by the model, if the suspected target area can still be identified and the target type remains unchanged, such as Figure 4 The identified bounding box of the central area is shown. The identified suspected target area is compared with the area threshold, and the suspected target area smaller than the area threshold is filtered out (at this time, since it has been magnified twice, there is no need to focus and magnify the small suspected target area again, and it has no identification value and can be directly filtered). For the suspected target area larger than the area threshold, it can be used as the detection result. Similarly, the current camera parameters are maintained to continue monitoring the target. When the set number of image evidence is reached, an alarm is generated, and the detection results are pushed to the background together with the original scene image before focusing and magnification.

[0042] Therefore, in the technical solution of the present invention, the image details are enhanced by multiple focusing and magnification technology, and the target detection model and preset filtering conditions are combined to achieve accurate detection and filtering of small targets. The present invention improves the detection accuracy and efficiency of small targets, reduces the false alarm rate, and has broad application prospects and important practical value.

[0043] Embodiment 2: This embodiment provides a small target detection system based on multiple focusing and magnification of a camera, including a camera and a recognition control unit. The camera has an automatic zoom function and is used to capture scene images; the recognition control unit is used to execute the small target detection method described in Example 1.

[0044] The hardware structure of this system is a camera, including a focusing module that can realize the automatic zoom function, and the software part is a recognition control unit. The recognition control unit can be used as a functional module of the camera and built into the camera; it can also be a functional module located in a remote server. The camera takes pictures by focusing and transmits the image back to the remote server, and the server performs model processing, focus magnification control, target screening, etc. The specific execution action content is shown in Example 1 and will not be repeated here.

[0045] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A small target detection method based on multiple focusing and magnification of a camera, characterized in that: The method comprises the following steps: Step S1, capturing a scene image by a camera with an automatic zoom function; Step S2: Processing the scene image with a pre-trained target detection model for preliminary screening to identify a series of suspected target areas; Step S3, sorting all suspected target areas according to their confidence levels, selecting the suspected target area with the highest confidence level, and setting it as the highest area; Step S4, controlling the camera to zoom in and focus on the highest area to obtain an image of the highest area; Step S5: Use the target detection model to process the highest region image. If the suspected target region can still be identified, select the suspected target region with the highest confidence as the highest region and calculate its area. Step S6: If the area of ​​the highest region is greater than or equal to the area threshold, the current highest region image is used as the detection result; if the area of ​​the highest region is less than the area threshold, the camera is controlled to zoom in and focus on the highest region again, and the target detection model is used for processing; Step S7: If the suspected target area can be identified, filter the area whose area is smaller than the area threshold, and use the detected image obtained after filtering as the detection result.

2. The small target detection method based on multiple focusing and magnification of a camera as claimed in claim 1, characterized in that: In step S2, after processing the image, the target detection model outputs a series of bounding boxes of suspected target areas and the confidence of each bounding box. The bounding box is a rectangular box surrounding the edge of the detected suspected target, and the confidence indicates the probability that the target detection model believes that the bounding box does contain the target.

3. The small target detection method based on multiple focusing and magnification of a camera as claimed in claim 2, characterized in that: In step S4 and step S6, the method of controlling the camera to zoom in and focus on the highest area is to use the bounding box of the highest area as the focusing center, send the bounding box information to the camera recognition control unit, focus the camera to the focusing center through the relationship between the image and the coordinates, and obtain the highest area image.

4. The small target detection method based on multiple focusing and magnification of a camera as claimed in claim 3, characterized in that: In step S5 and step S6, when the target detection model processes the highest area image, if there is no target area in the image, the current focusing is canceled and no alarm is generated; if the target type of the target identified by the focus center position changes, or the focus center position cannot identify the target, the current focusing is canceled and no alarm is generated; if the confidence of the target identified by the focus center position drops below the threshold, it is also regarded as having no target area, and the current focusing is canceled and no alarm is generated.

5. The small target detection method based on multiple focusing and magnification of a camera as claimed in claim 4, characterized in that: In step S6 and step S7, after obtaining the detection results, the target is continuously monitored according to the current camera parameters, and an alarm is generated after reaching the set number of image evidence collections, and the original scene image before focusing and magnification is pushed to the background.

6. The small target detection method based on multiple focusing and magnification of a camera as claimed in claim 4, characterized in that: If there are multiple suspected target areas with the highest confidence after the target detection model processes the image, the multiple suspected target areas are enlarged and focused in turn.

7. The small target detection method based on multiple focusing and magnification of a camera as claimed in claim 6, characterized in that: If the target detection model identifies multiple types of targets after processing the image, for each type of target, the suspected target area with the highest confidence level is selected and processed in accordance with steps S2-S7.

8. A small target detection system based on multiple focusing and magnification of a camera, characterized in that: The small target detection system includes a camera and a recognition control unit. The camera has an automatic zoom function and is used to capture scene images. The recognition control unit is used to execute the small target detection method according to any one of claims 1 to 7.

Images