Target object detection method, device and computer storage medium

Abstract

The present application provides a method, device and computer storage medium for detecting a target object, including performing feature extraction on an image to be detected based on a first dimension, a second dimension and a third dimension, obtaining first dimension features satisfying the first dimension, second dimension features satisfying the second dimension and third dimension features satisfying the third dimension; detecting background information of the target object in the image to be detected based on the first dimension features, the second dimension features and the target object, respectively, and obtaining background features of the image to be detected; performing fusion based on the background features and the third dimension features, and obtaining fusion features; segmenting the image to be detected based on the fusion features, and determining the target area containing the target object in the image to be detected. Accordingly, the present application can accurately locate the target area of ​​the target object in the image to be detected by fusing multi-level features, which is conducive to the timely investigation and treatment of safety hazards such as fire.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer testing technology, and in particular to a target object detection method, device and computer storage medium. Background Art

[0002] Due to natural and human factors, many fire incidents occur every year, causing great harm to people's lives and property, and causing extremely serious losses. There is an urgent need for fire prevention monitoring in forest fire prevention, traffic tunnel fire prevention, construction site fire prevention, and other scenarios. If all scenarios are monitored and reviewed manually, it will cost a huge amount of manpower and increase the possibility of underreporting.

[0003] Therefore, the research on intelligent fire alarm technology has very important application significance. Intelligent algorithms can perform real-time analysis of surveillance videos, issue alarms for images that are judged to be fires, and then manually review and take relevant emergency measures, which will greatly improve the efficiency of fire monitoring and the real-time nature of fire alarms.

[0004] Intelligent fire prevention technology mainly detects fire point objects and smoke objects to determine whether a fire has occurred. However, fire points and smoke are non-rigid objects with different shapes and scales, which makes the detection of flames and smoke difficult. In addition, both smoke and flames have obvious characteristics in the central area, and as the boundaries spread, the characteristics gradually weaken. At the same time, objects in real scenes are ever-changing, and many of them have visual expressions similar to smoke and fire points, which also increases the possibility of the model falsely reporting smoke and fire points. Summary of the invention

[0005] In view of the above problems, the present application provides a target object detection method, device and computer storage medium, which can improve the accuracy of smoke and fire detection results.

[0006] The first aspect of the present application provides a target object detection method, including performing feature extraction on an image to be detected based on a first dimension, a second dimension, and a third dimension to obtain first dimensional features that satisfy the first dimension, second dimensional features that satisfy the second dimension, and third dimensional features that satisfy the third dimension; based on the first dimensional features, the second dimensional features, and the target object, respectively detecting background information of the target object in the image to be detected to obtain background features of the image to be detected; performing fusion according to the background features and the third dimensional features to obtain fusion features; segmenting the image to be detected based on the fusion features to determine a target area in the image to be detected that contains the target object; wherein the first dimension is higher than the second dimension, and the second dimension is larger than the third dimension.

[0007] The second aspect of the present application provides a target object detection device, including: a feature extraction module, which is used to perform feature extraction on an image to be detected based on a first dimension, a second dimension, and a third dimension, to obtain a first dimension feature that satisfies the first dimension, a second dimension feature that satisfies the second dimension, and a third dimension feature that satisfies the third dimension; wherein the first dimension is higher than the second dimension, and the second dimension is larger than the third dimension; a feature processing module, which is used to detect background information of the target object in the image to be detected based on the first dimension feature, the second dimension feature, and the target object, respectively, to obtain background features of the image to be detected, and perform fusion according to the background features and the third dimension features to obtain fusion features; an object detection module, which is used to segment the image to be detected based on the fusion features, and determine a target area in the image to be detected that contains the target object.

[0008] A third aspect of the present application provides a computer storage medium, characterized in that the computer storage medium stores instructions for executing each step in the method described in the first aspect.

[0009] In summary, the target object detection method, device and computer storage medium provided in the present application can accurately locate the target area of ​​the target object in the image to be detected by determining the dimensional features of different levels of the image to be detected and fusing them. It is particularly suitable for the detection of fireworks scenes and is conducive to the timely elimination of fire hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0011] Figure 1 This is a processing flow chart of a target object detection method according to an exemplary embodiment of the present application.

[0012] Figure 2 This is a processing flow chart of a target object detection method according to another exemplary embodiment of the present application.

[0013] Figure 3 This is a processing flow chart of a target object detection method according to another exemplary embodiment of the present application.

[0014] Figure 4 It is a structural block diagram of a target object detection device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0016] The specific embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

[0017] Figure 1 This is a processing flow chart of a target object detection method of an exemplary embodiment of the present application. As shown in the figure, this embodiment mainly includes the following steps:

[0018] Step S102, based on the first dimension, the second dimension, and the third dimension, perform feature extraction on the image to be detected to obtain first dimension features that satisfy the first dimension, second dimension features that satisfy the second dimension, and third dimension features that satisfy the third dimension.

[0019] In this embodiment, the first dimension is higher than the second dimension, and the second dimension is larger than the third dimension, so the first dimension feature can be regarded as a high-dimensional feature, the second dimension feature can be regarded as a medium-dimensional feature, and the third dimension feature can be regarded as a low-dimensional feature.

[0020] Optionally, a feature extraction algorithm may be used to perform feature extraction of different dimensions on the image to be detected.

[0021] Optionally, the feature extraction algorithm may include but is not limited to a ResNet model and a Swin-Transformer model.

[0022] Step S104, based on the first dimensional features, the second dimensional features, and the target object, respectively detect the background information of the target object in the image to be detected to obtain the background features of the image to be detected.

[0023] Optionally, based on the first dimensional features (high dimensional features) and the target object, the background information of the target object in the image to be detected can be detected to obtain the first background in the image to be detected; and based on the second dimensional features (medium dimensional features) and the target object, the background information of the target object in the image to be detected can be detected to obtain the second background in the image to be detected, and the first background and the second background can be fused to determine the background features of the image to be detected.

[0024] Step S106, performing fusion according to the background features and the third dimensional features to obtain fused features.

[0025] Optionally, fusion may be performed based on background features and low-latitude features to obtain fusion features of the image to be detected.

[0026] Step S108, segmenting the image to be detected based on the fused features, and determining a target area in the image to be detected that contains the target object.

[0027] Optionally, a semantic segmentation model may be used to segment the image to be detected based on fusion features to determine an irregular target region containing a target object in the image to be detected.

[0028] Optionally, edge detection may be performed on the irregular target area to obtain a contour curve of the target object, and rectangle fitting may be performed based on the contour curve of the target object to obtain an object detection frame of the target object in the image to be detected.

[0029] In this embodiment, the target object in the image to be detected may include at least one of smoke and flame.

[0030] To sum up, the target object detection method of this embodiment can accurately locate the target area of ​​the target object to be detected by extracting dimensional features at different levels in the image to be detected and performing fusion analysis. It is particularly suitable for the prediction and processing of fire hazards by semantic segmentation models, and can enable management personnel to promptly check fire hazards.

[0031] Figure 2 The processing flow chart of the target object detection method of another exemplary embodiment of the present application is shown. This embodiment is a specific implementation of the above step S104. As shown in the figure, this embodiment mainly includes the following steps:

[0032] Step S202: obtaining a predicted category of each pixel in the image to be detected based on the first dimensional features.

[0033] Specifically, prediction may be performed based on the first dimensional features to obtain a predicted category (eg, one of category A, category B, and category C) for each pixel in the image to be detected.

[0034] Step S204: Based on the object category of the target object, each pixel point whose predicted category matches the object category in the image to be detected is determined as a candidate pixel point.

[0035] For example, if the object category of the target object is category B, all pixels in the image to be detected whose predicted category is category B are determined as candidate pixels.

[0036] Step S206: obtaining a first background of the image to be detected according to each candidate pixel point in the image to be detected.

[0037] Optionally, a background algorithm may be used to detect each candidate pixel point in the image to be detected to obtain a first background of the image to be detected.

[0038] Optionally, the background algorithm may include but is not limited to: one of an OCRNet algorithm and an ISNet algorithm.

[0039] Step S208: Utilize the background detection model to detect each pixel in the image to be detected based on the second dimensional feature to obtain a second background of the image to be detected.

[0040] Optionally, the background detection model may include, but is not limited to: a convolutional layer, a global average pooling, and a linear layer connected in sequence.

[0041] Step S210: using a feature fusion algorithm to fuse the first background and the second background to obtain background features of the image to be detected.

[0042] Optionally, the feature fusion algorithm may include at least one of an additive fusion algorithm, a serial fusion algorithm and an iAFF fusion algorithm.

[0043] To sum up, this embodiment obtains the first background and the second background of the image to be detected and fuses them based on the dimensional features of different levels extracted from the image to be detected, which can improve the accuracy of the background feature extraction results of the image to be detected and is conducive to improving the accuracy of the subsequent determination of the target area of ​​the target object.

[0044] Figure 3 The processing flow chart of the target object detection method of another exemplary embodiment of the present application is shown. This embodiment is a specific implementation of the above step S106. As shown in the figure, this embodiment mainly includes the following steps:

[0045] Step S302: performing a first upsampling operation on the background feature to make the feature resolution of the background feature consistent with the feature resolution of the third dimensional feature.

[0046] Optionally, the first upsampling operation may include at least one of bilinear interpolation and deconvolution.

[0047] Step S304: fuse the background features and the third dimensional features with the same feature resolution to obtain a fused feature.

[0048] Step S306: performing a second upsampling operation on the fused feature to make the feature resolution of the fused feature consistent with the feature resolution of the image to be detected.

[0049] Optionally, the second upsampling operation may include at least one of bilinear interpolation and deconvolution.

[0050] In summary, this embodiment performs multiple upsampling operations to obtain fused features with the same feature resolution as the image to be detected, which is beneficial to improving the accuracy of subsequent feature segmentation processing performed on the image to be detected.

[0051] Figure 4 As shown in the figure, the target object detection device 400 of this embodiment mainly includes: a feature extraction module 402 , a feature processing module 404 , and an object detection module 406 .

[0052] The feature extraction module 402 is used to perform feature extraction on the image to be detected based on the first dimension, the second dimension, and the third dimension to obtain the first dimension feature that satisfies the first dimension, the second dimension feature that satisfies the second dimension, and the third dimension feature that satisfies the third dimension; wherein the first dimension is higher than the second dimension, and the second dimension is larger than the third dimension.

[0053] The feature processing module 404 is used to detect the background information of the target object in the image to be detected based on the first dimensional features, the second dimensional features, and the target object, obtain the background features of the image to be detected, and perform fusion according to the background features and the third dimensional features to obtain fusion features.

[0054] The object detection module 406 is used to segment the image to be detected based on the fused features, and determine a target area in the image to be detected that contains the target object.

[0055] Optionally, the feature extraction module 402 is further used to: perform feature extraction on the image to be detected using a feature extraction algorithm; wherein the feature extraction algorithm includes at least one of a ResNet algorithm and a Swin-Transformer algorithm.

[0056] Optionally, the feature processing module 404 is also used to: based on the first dimensional feature and the target object, detect each pixel point of the image to be detected and associated with the target object to obtain a first background of the image to be detected; based on the second dimensional feature, detect each pixel point in the image to be detected to obtain a second background of the image to be detected; and fuse the first background and the second background to obtain background features of the image to be detected.

[0057] Optionally, the feature processing module 404 is also used to: obtain the predicted category of each pixel in the image to be detected based on the first dimensional feature; based on the object category of the target object, determine each pixel in the image to be detected whose predicted category is consistent with the object category as a candidate pixel; and obtain the first background of the image to be detected based on each candidate pixel in the image to be detected.

[0058] Optionally, the feature processing module 404 is further used to: use a background algorithm to detect each candidate pixel point in the image to be detected to obtain the first background of the image to be detected; wherein the background algorithm includes at least one of: an OCRNet algorithm and an ISNet algorithm.

[0059] Optionally, the feature processing module 404 is also used to: use a background detection model to detect each pixel in the image to be detected based on the second dimensional features to obtain a second background of the image to be detected; wherein the background detection model includes at least a convolutional layer, a global average pooling, and a linear layer connected in sequence.

[0060] Optionally, the feature processing module 404 is also used to: use a feature fusion algorithm to fuse the first background and the second background to obtain the background features of the image to be detected; wherein the feature fusion algorithm includes at least one of: an additive fusion algorithm, a concatenated fusion algorithm and an iAFF fusion algorithm.

[0061] Optionally, the feature processing module 404 is also used to: perform a first upsampling operation on the background feature to make the feature resolution of the background feature match the feature resolution of the third dimensional feature; fuse the background feature and the third dimensional feature with the same feature resolution to obtain the fused feature; perform a second upsampling operation on the fused feature to make the feature resolution of the fused feature match the feature resolution of the image to be detected; wherein the first upsampling operation and the second upsampling operation include at least one of bilinear interpolation and deconvolution.

[0062] Optionally, the object detection module 406 is further used to: use a semantic segmentation model to segment the image to be detected based on the fusion features, and determine an irregular target area in the image to be detected that contains the target object.

[0063] Optionally, the object detection module 406 is further used to: perform edge detection on the irregular target area to obtain a contour curve of the target object; perform rectangle fitting based on the contour curve of the target object to obtain an object detection frame of the target object in the image to be detected.

[0064] Optionally, the target object includes at least one of smoke and flame.

[0065] Another embodiment of the present application further provides a computer storage medium, in which instructions for executing the steps in the above target object detection method embodiments are stored.

[0066] In summary, the present application provides a target object detection scheme that integrates multi-level features, uses high-level features to learn the background information of the target object, uses mid-level features to learn the global background information, and uses low-level features for decoding guidance. By integrating dimensional features at different levels as the expression of the final feature information, the robustness of the model detection results can be improved, and the detection accuracy of the semantic segmentation model can be significantly improved, which is especially suitable for detection needs in scenarios with fire hazards.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A target object detection method, comprising: Based on the first dimension, the second dimension, and the third dimension, feature extraction is performed on the image to be detected to obtain first dimension features that satisfy the first dimension, second dimension features that satisfy the second dimension, and third dimension features that satisfy the third dimension; Based on the first dimensional features, the second dimensional features, and the target object, respectively detect background information of the target object in the image to be detected to obtain background features of the image to be detected; Perform fusion according to the background features and the third dimension features to obtain fusion features; Segment the image to be detected based on the fusion features, and determine a target area in the image to be detected that contains the target object; Wherein, the first dimension is higher than the second dimension, and the second dimension is greater than the third dimension; The detecting the background information of the target object in the image to be detected based on the first dimensional feature, the second dimensional feature, and the target object respectively, and obtaining the background feature of the image to be detected includes: Based on the first dimensional feature and the target object, detect each pixel point of the image to be detected that is associated with the target object to obtain a first background of the image to be detected; Based on the second dimensional feature, detect each pixel in the image to be detected to obtain a second background of the image to be detected; The first background and the second background are fused to obtain background features of the image to be detected.

2. The method according to claim 1, wherein: The performing feature extraction on the image to be detected comprises: Using a feature extraction algorithm, perform feature extraction on the image to be detected; The feature extraction algorithm includes at least one of a ResNet algorithm and a Swin-Transformer algorithm.

3. The method according to claim 1, wherein: The detecting each pixel point associated with the target object and the image to be detected based on the first dimensional feature and the target object to obtain a first background of the image to be detected includes: Based on the first dimensional features, obtaining a predicted category for each pixel in the image to be detected; Based on the object category of the target object, each pixel point in the image to be detected whose predicted category matches the object category is determined as a candidate pixel point; The first background of the image to be detected is obtained according to each candidate pixel point in the image to be detected.

4. The method according to claim 3, wherein: The step of obtaining the first background of the image to be detected according to each candidate pixel point in the image to be detected includes: Using a background algorithm, detect each candidate pixel point in the image to be detected to obtain the first background of the image to be detected; The background algorithm includes at least one of an OCRNet algorithm and an ISNet algorithm.

5. The method according to claim 1, wherein: The detecting each pixel point in the image to be detected based on the second dimensional feature to obtain a second background of the image to be detected includes: Using a background detection model to detect each pixel in the image to be detected based on the second dimensional feature, to obtain a second background of the image to be detected; The background detection model at least includes a convolutional layer, a global average pooling layer, and a linear layer connected in sequence.

6. The method according to claim 1, wherein: The fusing the first background and the second background to obtain the background feature of the image to be detected includes: Using a feature fusion algorithm, the first background and the second background are fused to obtain background features of the image to be detected; The feature fusion algorithm includes at least one of an additive fusion algorithm, a serial fusion algorithm and an iAFF fusion algorithm.

7. The method according to claim 1, wherein: The performing fusion according to the background feature and the third dimension feature to obtain a fusion feature includes: Performing a first upsampling operation on the background feature so that a feature resolution of the background feature matches a feature resolution of the third dimensional feature; Fusing the background feature and the third dimension feature having the same feature resolution to obtain the fused feature; Performing a second upsampling operation on the fused feature so that the feature resolution of the fused feature matches the feature resolution of the image to be detected; The first upsampling operation and the second upsampling operation include at least one of bilinear interpolation and deconvolution.

8. The method according to claim 1 or 7, wherein: The step of segmenting the image to be detected based on the fusion features and determining a target area in the image to be detected that contains the target object comprises: The image to be detected is segmented based on the fusion features using a semantic segmentation model to determine an irregular target area in the image to be detected that contains the target object.

9. The method according to claim 8, wherein: The method further comprises: Performing edge detection on the irregular target area to obtain a contour curve of the target object; Rectangular fitting is performed based on the contour curve of the target object to obtain an object detection frame of the target object in the image to be detected.

10. The method according to claim 1, wherein: The target object includes at least one of smoke and flame.

11. A target object detection device, comprising: A feature extraction module, configured to perform feature extraction on the image to be detected based on the first dimension, the second dimension, and the third dimension, to obtain a first dimension feature satisfying the first dimension, a second dimension feature satisfying the second dimension, and a third dimension feature satisfying the third dimension; wherein the first dimension is higher than the second dimension, and the second dimension is greater than the third dimension; a feature processing module, configured to detect background information of the target object in the image to be detected based on the first dimensional features, the second dimensional features, and the target object, respectively, to obtain background features of the image to be detected, and to perform fusion according to the background features and the third dimensional features to obtain fusion features; An object detection module, used for segmenting the image to be detected based on the fusion features, and determining a target area in the image to be detected that contains the target object; The feature processing module is also used to: based on the first dimensional feature and the target object, detect each pixel point of the image to be detected and associated with the target object to obtain a first background of the image to be detected; based on the second dimensional feature, detect each pixel point in the image to be detected to obtain a second background of the image to be detected; and fuse the first background and the second background to obtain background features of the image to be detected.

12. A computer storage medium, characterized in that: The computer storage medium stores instructions for executing the steps in the method according to any one of claims 1 to 10.

Images