Target detection method and device, equipment, readable storage medium and program product
Through the cascading feature extraction network and the target perception network, multi-level feature fusion and target detection are solved, and the problem of poor sign detection effect in the existing technology is achieved, more efficient and accurate target detection is achieved, and safety hazards are reduced.
Patent Information
- Application Number
- CN202411452460.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-02-21
AI Technical Summary
The existing target detection algorithms are not effective in detecting important signs in power construction and daily maintenance, especially in harsh environments, signs are prone to breakage, fading or falling off, resulting in increased safety hazards.
The cascading feature extraction network and target perception network are adopted to improve the accuracy of target detection through multi-level methods of feature extraction, feature fusion and target perception. The specific steps include obtaining the image to be detected and the object detection model, performing feature extraction, feature fusion, and performing final object detection through the target perception network.
It improves the accuracy and efficiency of sign detection, can more effectively monitor and correct the abnormal status of signs, and reduce safety hazards.
Smart Images
Figure CN119495021A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of target detection, and in particular to a target detection method, device, equipment, readable storage medium and program product. Background Art
[0002] In the field of power construction and daily maintenance, warning signs are key safety facilities and are essential to ensure the safety of the power grid. They effectively prevent accidents such as electric shock and illegal operations by providing clear power safety information. However, in actual applications, these signs are exposed to harsh environments for a long time and face challenges such as damage, color fading or falling off, resulting in the loss of warning information and increasing safety hazards.
[0003] To solve this problem, it is particularly important to monitor and correct the abnormal status of the sign in a timely manner. The real-time status of the sign is captured through monitoring and patrol photography. In order to achieve efficient and accurate anomaly detection, target detection through target detection algorithms has emerged. However, the existing target detection algorithms cannot achieve good target detection results for signboards. Summary of the invention
[0004] Based on this, it is necessary to provide a target detection method, device, equipment, readable storage medium and program product that can improve the target detection effect in response to the above technical problems.
[0005] In a first aspect, the present application provides a target detection method, comprising:
[0006] Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0007] For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0008] The feature extraction result of the feature extraction network is fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0009] Through the target perception network, the feature fusion result output by the tail feature extraction network is subjected to target perception to obtain the target detection result of the target to be detected in the image to be detected.
[0010] In one embodiment, the target detection model includes a feature fusion network, and the feature fusion network includes a splicing layer and a feature fusion layer; accordingly, the feature extraction result of the feature extraction network is feature fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network, including:
[0011] Input the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain the splicing result;
[0012] The concatenated result is input into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
[0013] In one embodiment, the feature extraction result of the feature extraction network and the context feature of the image to be processed are input into the splicing layer to obtain the splicing result, including:
[0014] The feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain the splicing result.
[0015] In one embodiment, the splicing layer includes a first feature correction layer and a second feature correction layer; accordingly, the feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain a splicing result, including:
[0016] Inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain the first feature correction result corresponding to the feature extraction network;
[0017] Inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network;
[0018] The first feature correction result and the second feature correction result are spliced to obtain a splicing result.
[0019] In one embodiment, the second feature correction layer includes at least one deconvolution layer; accordingly, the feature extraction result of the feature extraction network and the context feature of the image to be processed are input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network, including:
[0020] The feature fusion result output by the previous feature extraction network cascaded by the feature extraction network is input into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network;
[0021] The transposed features are input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network.
[0022] In one of the embodiments, the first weight corresponding to the context feature of the image to be processed is smaller than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is smaller than the third weight corresponding to the feature extraction result of the feature extraction network.
[0023] In a second aspect, the present application also provides a target detection device, comprising:
[0024] An acquisition module is used to acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0025] A feature extraction module is used to extract features of an image to be processed by a feature extraction network for any feature extraction network, and obtain a feature extraction result of the feature extraction network;
[0026] A feature fusion module is used to perform feature fusion on the feature extraction result of the feature extraction network and the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0027] The target detection module is used to perform target perception on the feature fusion result output by the tail feature extraction network through the target perception network, and obtain the target detection result of the target to be detected in the image to be detected.
[0028] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0029] Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0030] For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0031] The feature extraction result of the feature extraction network is fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0032] Through the target perception network, the feature fusion result output by the tail feature extraction network is subjected to target perception to obtain the target detection result of the target to be detected in the image to be detected.
[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:
[0034] Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0035] For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0036] The feature extraction result of the feature extraction network is fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0037] Through the target perception network, the feature fusion result output by the tail feature extraction network is subjected to target perception to obtain the target detection result of the target to be detected in the image to be detected.
[0038] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0039] Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0040] For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0041] The feature extraction result of the feature extraction network is fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0042] Through the target perception network, the feature fusion result output by the tail feature extraction network is subjected to target perception to obtain the target detection result of the target to be detected in the image to be detected.
[0043] The above-mentioned target detection method, device, equipment, readable storage medium and program product, for an image to be detected that carries a target to be detected, performs target detection on the image to be detected through a pre-trained target detection model. Specifically, for any feature extraction network, feature extraction is performed on the image to be processed of the feature extraction network to obtain the feature extraction result of the feature extraction network; the feature extraction result of the feature extraction network is feature fused with the contextual features of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein, the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade; through the target perception network, the feature fusion result output by the tail feature extraction network is target perceived to obtain the target detection result of the target to be detected in the image to be detected. Compared with the prior art, in which the feature extraction result of the previous feature extraction network is directly input into the next feature extraction network, this method fuses the feature extraction result of the previous feature extraction network with the corresponding contextual features, and uses the fusion result as the input data of the next feature extraction network. Throughout the entire process, the method of multi-level feature fusion is chosen, and contextual features are introduced in the process of feature fusion, which can improve the detection accuracy of the target detection model. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 Schematic diagram of a target detection method in one embodiment;
[0046] Figure 2A is a flowchart of a feature fusion step in an embodiment;
[0047] Figure 2B A schematic diagram of a model structure corresponding to feature fusion in an embodiment;
[0048] Figure 3 A schematic diagram of a process for determining a splicing result in one embodiment;
[0049] Figure 4A A schematic flow chart of a step for determining a splicing result in another embodiment;
[0050] Figure 4B A schematic diagram of a model structure corresponding to feature fusion in another embodiment;
[0051] Figure 4C A schematic diagram of a model structure corresponding to feature fusion in another embodiment;
[0052] Figure 5 is a structural block diagram of a target detection device in one embodiment;
[0053] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0055] Before introducing the target detection method provided by this application, it should be noted that safety signs and warning signs are an intuitive information communication tool, which is essential for guiding people to pay attention to potential dangers and take corresponding measures. The existence of safety signs and warning signs can help prevent accidents and reduce potential dangers.
[0056] In the scenarios of power construction and daily maintenance, relevant warning signs are also very important safety facilities. Standardized establishment of eye-catching signs is one of the important guarantees for power grid safety. Key components must use standardized signs to remind personnel of power safety information to prevent accidents such as electric shock and illegal operations. However, in actual scenarios, exposed signs are affected by harsh environments for a long time, and often have problems such as damage, lack of eye-catching safety colors or falling off, and missing warning information. Therefore, during monitoring and patrol photography, it is necessary to use images to promptly detect abnormal states of signs and make timely corrections. In order to facilitate the timely detection of abnormal states of signs, target detection algorithms are currently usually used to detect abnormal signs.
[0057] Object detection is a basic task in computer vision, which aims to accurately identify and locate objects of a specific category from images or videos. Traditional object detection algorithms, such as those based on artificially designed features (such as Haar features and HOG features) and traditional machine learning algorithms (such as SVM and Adaboost). Generally speaking, signboard targets are relatively small in the overall image, especially in 4k high-definition images of drone patrols. Traditional object detection algorithms find it difficult to achieve refined typical feature recognition of small targets. Due to the limited resolution and information in the image, reliably detecting small objects is a very challenging task.
[0058] In the above context, Single Shot Multibox detector (SSD) is used as a fast detector, which has greatly improved the speed compared with previous studies. SSD uses multi-scale feature maps to predict objects, using high-level feature information with a larger receptive field to predict large objects, and low-level feature information with a smaller receptive field to predict small objects. However, when the feature information of the low-level network used by SSD predicts small objects, due to the lack of high-level semantic features, there will be a problem of poor detection effect. Therefore, the target detection algorithm provided in this embodiment is proposed to solve this problem.
[0059] In one embodiment, Figure 1 As shown, a target detection method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:
[0060] S110, obtaining an image to be detected carrying a target to be detected and a target detection model.
[0061] The image to be detected may be an image that needs to be subjected to target detection. Usually, the image to be detected may be a two-dimensional image. The target to be detected may be a target object to be subjected to target detection in the image to be detected. For example, taking the image to be detected as a two-dimensional image containing a target sign as an example, the target to be detected is the target sign in the image to be detected. The purpose of target detection on the image to be detected is to detect whether the state of the target sign is abnormal, and the specific abnormal situation.
[0062] Exemplarily, the image to be detected may be acquired by an image acquisition device, such as a camera, and transmitted to a server. The target detection model may be a pre-trained model capable of performing target detection on an image. The target detection model includes at least one cascaded feature extraction network and a target perception network.
[0063] S120, for any feature extraction network, performing feature extraction on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network.
[0064] For any feature extraction network, the corresponding image to be processed is the input data of the feature extraction network. The feature extraction network can be a common convolutional neural network, and this application does not impose any limitation on the internal structure of the feature extraction network.
[0065] Specifically, for any feature extraction network, the image to be processed corresponding to the feature extraction network is input into the feature extraction network to obtain the feature extraction result of the feature extraction network.
[0066] S130, performing feature fusion on the feature extraction result of the feature extraction network and the context feature of the image to be processed to obtain a feature fusion result of the feature extraction network.
[0067] Among them, the image to be processed by the first feature extraction network is the image to be detected; the image to be processed by the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade.
[0068] The context feature may be a feature obtained by extracting the context information of the image to be processed. Exemplarily, the context information of the image to be processed may be input into a feature extraction network to obtain the context feature of the image to be processed. It should be noted that the context information of the image to be processed may be acquired by an image acquisition device.
[0069] Exemplarily, in this embodiment, the feature extraction result of the feature extraction network can be simply combined with the context feature of the image to be processed, and the combined result can be used as the feature fusion result of the feature extraction network.
[0070] S140, performing target perception on the feature fusion result output by the tail feature extraction network through the target perception network to obtain the target detection result of the target to be detected in the image to be detected.
[0071] Among them, the target perception network can also be understood as a target detection network, which is used to detect the received feature data and obtain the target detection result of the target to be detected corresponding to the feature data.
[0072] Specifically, for the output result of the last feature extraction network in the target detection model, the feature extraction result of the feature extraction network is fused with the contextual features of the image to be processed corresponding to the feature extraction network to obtain the feature fusion result output by the feature extraction network, and the feature fusion result is input into the target perception network for target detection to obtain the target detection result of the target to be detected in the image to be detected.
[0073] In the above target detection method, for an image to be detected carrying a target to be detected, a pre-trained target detection model is used to perform target detection on the image to be detected. Specifically, for any feature extraction network, feature extraction is performed on the image to be processed of the feature extraction network to obtain the feature extraction result of the feature extraction network; the feature extraction result of the feature extraction network is feature fused with the contextual features of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein, the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network cascaded; through the target perception network, the feature fusion result output by the tail feature extraction network is target perceived to obtain the target detection result of the target to be detected in the image to be detected. Compared with the prior art, in which the feature extraction result of the previous feature extraction network is directly input into the next feature extraction network, this method fuses the feature extraction result of the previous feature extraction network with the corresponding contextual features, and uses the fusion result as the input data of the next feature extraction network. In the whole process, the multi-level feature fusion method is selected, and the contextual features are introduced in the process of feature fusion, which can improve the detection accuracy of the target detection model.
[0074] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the target detection model includes a feature fusion network, and the feature fusion network includes a splicing layer and a feature fusion layer; in this case, the feature extraction result of the feature extraction network is fused with the contextual features of the image to be processed to obtain the feature fusion result of the feature extraction network, which is introduced in detail.
[0075] See also Figure 2A The feature fusion steps shown include:
[0076] S210, inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain a splicing result.
[0077] S220, input the splicing result into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
[0078] See also Figure 2B The model structure corresponding to the feature fusion shown inputs the feature extraction result and the contextual features of the processed image into the splicing layer of the feature fusion network to obtain the splicing result; further, the splicing result is input into the feature fusion layer of the feature fusion network to obtain the feature fusion result of the feature extraction network.
[0079] It can be understood that the feature fusion result of the feature extraction network is used as the image to be processed corresponding to the next feature extraction network cascaded by the feature extraction network, and is input into the next feature extraction network.
[0080] In the above embodiment, when the target detection model includes a feature fusion network, and the feature fusion network includes a splicing layer and a feature fusion layer, the feature extraction result of the feature extraction network is fused with the context features of the image to be processed to obtain the feature fusion result of the feature extraction network, which is introduced in detail. Specifically, the feature extraction result and the context features of the image to be processed are first feature spliced, so that the feature mapping sizes of the two neural network layers to be spliced are the same. Furthermore, the splicing result is feature fused, which can make the feature fusion result more accurate and convenient for subsequent target detection.
[0081] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the process of inputting the feature extraction results of the feature extraction network and the context features of the image to be processed into the splicing layer to obtain the splicing result is refined.
[0082] See also Figure 3 The steps for determining the splicing result shown include:
[0083] S310, inputting the feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed into the splicing layer to obtain a splicing result.
[0084] It should be noted that the traditional SSD model does not use the last feature map of the convolutional neural network, but uses a multi-layer pyramidal feature hierarchy within the convolutional neural network to predict objects of different scales. That is, shallower layers are used to predict smaller objects, while deeper layers are used to predict larger objects, thereby reducing the prediction burden of the entire model. However, shallower layers often lack semantic information, which is an important supplement for small object detection. Therefore, passing the semantic information captured in the convolution forward calculation (i.e., contextual features) back to the shallower neural network will be able to improve the performance of small object detection.
[0085] Since context features may introduce useless background noise, they are not always useful for small target detection. Therefore, the connection module uses a 1×1 convolutional layer to learn the weight of the fusion of target information and context features to reduce the interference of useless background noise. Specifically, the first weight corresponding to the context feature of the image to be processed is less than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is less than the third weight corresponding to the feature extraction result of the feature extraction network.
[0086] Among them, the first weight, the second weight and the third weight can all be determined based on manual experience or through a large number of experiments, and this application does not impose any limitation on this.
[0087] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the splicing layer includes a first feature correction layer and a second feature correction layer; in this case, the feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the contextual features of the image to be processed are input into the splicing layer to obtain the splicing result.
[0088] See also Figure 4A The steps for determining the splicing result shown include:
[0089] S410, inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain the first feature correction result corresponding to the feature extraction network.
[0090] Among them, the first feature correction layer and the second feature correction layer are both linear correction units, which can be understood as activation functions.
[0091] S420, inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network.
[0092] It should be noted that since too large a receptive field usually introduces a lot of useless background noise, it is necessary to select an appropriate conversion layer to provide useful contextual information. Since SSD predicts small objects in the shallower layers, in order to reduce the speed, the feature fusion module is not used for large objects in the deeper layers. In order to select a suitable feature fusion layer, in this embodiment, the deconvolution method is used to explore the effective receptive fields of different layers. Therefore, in this embodiment, the second feature correction layer includes at least one deconvolution layer.
[0093] Correspondingly, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network is input into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network; the transposed feature is input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network.
[0094] S430, splicing the first feature correction result and the second feature correction result to obtain a splicing result.
[0095] In an optional embodiment, the target detection model may include a first feature fusion network and a second feature fusion network. The first feature fusion network and the second feature fusion network have the same function, except that the size of the convolutional layer is different. For example, the size of the first feature fusion network may be twice that of the second feature fusion network. In addition, the target detection model may also include a third feature correction layer for further correction of the fused features to improve the accuracy of the feature fusion results.
[0096] For example, see Figure 4B The model structure corresponding to the feature fusion shown in the figure inputs the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the deconvolution layer, and inputs the deconvolution result into the feature extraction layer for feature extraction; inputs the feature extraction result output by the feature extraction layer into the normalization layer, and inputs the normalization result into the second feature correction layer for feature correction to obtain the first feature correction result. At the same time, the feature extraction result and the context feature of the image to be processed are input into the feature extraction layer for feature extraction; inputs the feature extraction result output by the feature extraction layer into the normalization layer, and inputs the normalization result into the first feature correction layer for feature correction to obtain the second feature correction result. The first feature correction result and the second feature correction result are input into the splicing network for splicing to obtain the splicing result.
[0097] Furthermore, the splicing result is input into the first feature fusion network to obtain a first feature fusion result, and the first feature fusion result is input into the feature extraction layer for feature extraction; and the feature extraction result output by the feature extraction layer is input into the third feature correction layer for further correction, and the corrected feature is input into the second feature fusion network to obtain a feature fusion result.
[0098] In the above optional implementation, a deconvolution layer is added after a neural network layer, and the deconvolution layer is initialized by bilinear upsampling. Two feature extraction layers are used after two network layers of uniform size to learn better features for fusion. Then the normalization layers are tracked at different scales respectively, and then they are connected along the channel axis. The final fused feature map is generated by dimensionality reduction and feature reorganization by a 1×1 convolution layer.
[0099] It should be noted that the above optional implementation is only the implementation corresponding to S410-S430. In order to increase the diversity of this solution, this embodiment also provides another optional implementation. In another optional implementation, see Figure 4CThe model structure corresponding to the feature fusion shown in the figure inputs the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the deconvolution layer, and inputs the deconvolution result into the feature extraction layer for feature extraction; the feature extraction result output by the feature extraction layer is input into the normalization layer to obtain the first processing result. At the same time, the feature extraction result and the contextual features of the image to be processed are input into the feature extraction layer for feature extraction; the feature extraction result output by the feature extraction layer is input into the normalization layer to obtain the second processing result. The first processing result and the second processing result are input into the first feature fusion network to obtain the first feature fusion result, and the first feature fusion result is input into the feature correction layer to obtain the feature correction result; and the feature correction result is input into the second feature fusion network to obtain the feature fusion result.
[0100] In the above optional implementation, two feature maps containing features of different levels are summarized point-to-point with equal weights. Since two feature extraction layers are used before, a better fusion effect can be obtained.
[0101] In the above embodiment, in the two cases where the splicing layer includes the first feature correction layer and the second feature correction layer, the feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain the splicing result. The above two model structures are provided, both of which can make the feature fusion result more accurate.
[0102] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the target detection method provided by the present application is introduced in detail, including:
[0103] Step 1, obtaining an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network, feature fusion network and target perception network;
[0104] Step 2, for any feature extraction network, extracting features from the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0105] Step 3, inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain the first feature correction result corresponding to the feature extraction network;
[0106] Step 4, inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network;
[0107] Step 5, inputting the transposed features into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network;
[0108] Step 6, splicing the first feature correction result and the second feature correction result to obtain a splicing result;
[0109] Among them, the first weight corresponding to the context feature of the image to be processed is smaller than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is smaller than the third weight corresponding to the feature extraction result of the feature extraction network.
[0110] Step 7: Input the splicing result into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
[0111] Among them, the image to be processed by the first feature extraction network is the image to be detected; the image to be processed by the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0112] Step 8: Perform target perception on the feature fusion result output by the tail feature extraction network through the target perception network to obtain the target detection result of the target to be detected in the image to be detected.
[0113] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0114] Based on the same inventive concept, the embodiment of the present application also provides a target detection device for implementing the target detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target detection device embodiments provided below can refer to the limitations on the target detection method above, and will not be repeated here.
[0115] In an exemplary embodiment, Figure 5 As shown, a target detection device is provided, including: an acquisition module 510, a feature extraction module 520, a feature fusion module 530 and a target detection module 540, wherein:
[0116] The acquisition module 510 is used to acquire an image to be detected that carries a target to be detected and a target detection model.
[0117] Wherein, the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0118] The feature extraction module 520 is used to perform feature extraction on the image to be processed by any feature extraction network to obtain the feature extraction result of the feature extraction network.
[0119] The feature fusion module 530 is used to perform feature fusion on the feature extraction result of the feature extraction network and the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network.
[0120] Among them, the image to be processed by the first feature extraction network is the image to be detected; the image to be processed by the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade.
[0121] The target detection module 540 is used to perform target perception on the feature fusion result output by the tail feature extraction network through the target perception network, so as to obtain the target detection result of the target to be detected in the image to be detected.
[0122] In one embodiment, the target detection model includes a feature fusion network, and the feature fusion network includes a splicing layer and a feature fusion layer; accordingly, the feature fusion module 530 includes a splicing unit, which is used to input the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain the splicing result; the feature fusion unit is used to input the splicing result into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
[0123] Among them, the first weight corresponding to the context feature of the image to be processed is smaller than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is smaller than the third weight corresponding to the feature extraction result of the feature extraction network.
[0124] In one embodiment, the splicing layer includes a first feature correction layer and a second feature correction layer; the splicing unit includes a first correction subunit, which is used to input the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain a first feature correction result corresponding to the feature extraction network; the second correction subunit is used to input the feature extraction result of the feature extraction network and the contextual features of the image to be processed into the second feature correction layer to obtain a second feature correction result corresponding to the feature extraction network; the splicing subunit is used to splice the first feature correction result and the second feature correction result to obtain a splicing result.
[0125] In one embodiment, the second feature correction layer includes at least one deconvolution layer; accordingly, the second correction subunit is specifically used to input the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network; the transposed feature is input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network.
[0126] Each module in the above target detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0127] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a target detection method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0128] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0129] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0130] Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0131] For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0132] The feature extraction result of the feature extraction network is fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0133] Through the target perception network, the feature fusion result output by the tail feature extraction network is subjected to target perception to obtain the target detection result of the target to be detected in the image to be detected.
[0134] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0135] Input the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain the splicing result;
[0136] The concatenated result is input into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
[0137] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0138] The feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain the splicing result.
[0139] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0140] Inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain the first feature correction result corresponding to the feature extraction network;
[0141] Inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network;
[0142] The first feature correction result and the second feature correction result are spliced to obtain a splicing result.
[0143] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0144] The feature fusion result output by the previous feature extraction network cascaded by the feature extraction network is input into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network;
[0145] The transposed features are input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network.
[0146] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0147] The first weight corresponding to the context feature of the image to be processed is controlled to be smaller than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is smaller than the third weight corresponding to the feature extraction result of the feature extraction network.
[0148] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0149] Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0150] For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0151] The feature extraction result of the feature extraction network is fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0152] Through the target perception network, the feature fusion result output by the tail feature extraction network is subjected to target perception to obtain the target detection result of the target to be detected in the image to be detected.
[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0154] Input the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain the splicing result;
[0155] The concatenated result is input into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
[0156] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0157] The feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain the splicing result.
[0158] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0159] Inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain the first feature correction result corresponding to the feature extraction network;
[0160] Inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network;
[0161] The first feature correction result and the second feature correction result are spliced to obtain a splicing result.
[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0163] The feature fusion result output by the previous feature extraction network cascaded by the feature extraction network is input into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network;
[0164] The transposed features are input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network.
[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0166] The first weight corresponding to the context feature of the image to be processed is controlled to be smaller than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is smaller than the third weight corresponding to the feature extraction result of the feature extraction network.
[0167] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:
[0168] Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network;
[0169] For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network;
[0170] The feature extraction result of the feature extraction network is fused with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade;
[0171] Through the target perception network, the feature fusion result output by the tail feature extraction network is subjected to target perception to obtain the target detection result of the target to be detected in the image to be detected.
[0172] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0173] Input the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain the splicing result;
[0174] The concatenated result is input into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
[0175] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0176] The feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain the splicing result.
[0177] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0178] Inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain the first feature correction result corresponding to the feature extraction network;
[0179] Inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network;
[0180] The first feature correction result and the second feature correction result are spliced to obtain a splicing result.
[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0182] The feature fusion result output by the previous feature extraction network cascaded by the feature extraction network is input into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network;
[0183] The transposed features are input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network.
[0184] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0185] The first weight corresponding to the context feature of the image to be processed is controlled to be smaller than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is smaller than the third weight corresponding to the feature extraction result of the feature extraction network.
[0186] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0187] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0188] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A target detection method, characterized in that: The method comprises: Acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network; For any feature extraction network, feature extraction is performed on the image to be processed by the feature extraction network to obtain a feature extraction result of the feature extraction network; The feature extraction result of the feature extraction network is subjected to feature fusion with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade; Through the target perception network, target perception is performed on the feature fusion result output by the tail feature extraction network to obtain the target detection result of the target to be detected in the image to be detected.
2. The method according to claim 1, characterized in that The target detection model includes a feature fusion network, and the feature fusion network includes a splicing layer and a feature fusion layer; accordingly, the feature extraction result of the feature extraction network is subjected to feature fusion with the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network, including: Inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain a splicing result; The splicing result is input into the feature fusion layer to obtain the feature fusion result of the feature extraction network.
3. The method according to claim 2, characterized in that The step of inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the splicing layer to obtain the splicing result includes: The feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain a splicing result.
4. The method according to claim 3, characterized in that The splicing layer includes a first feature correction layer and a second feature correction layer; accordingly, the feature extraction result of the feature extraction network, the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network, and the context feature of the image to be processed are input into the splicing layer to obtain a splicing result, including: Inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the first feature correction layer to obtain the first feature correction result corresponding to the feature extraction network; Inputting the feature extraction result of the feature extraction network and the context feature of the image to be processed into the second feature correction layer to obtain a second feature correction result corresponding to the feature extraction network; The first feature correction result and the second feature correction result are spliced to obtain a splicing result.
5. The method according to claim 4, characterized in that The second feature correction layer includes at least one deconvolution layer; accordingly, the feature extraction result of the feature extraction network and the context feature of the image to be processed are input into the second feature correction layer to obtain the second feature correction result corresponding to the feature extraction network, including: Inputting the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network into the deconvolution layer to obtain the transposed feature corresponding to the feature extraction network; The transposed features are input into the second feature correction layer to obtain a second feature correction result corresponding to the feature extraction network.
6. The method according to any one of claims 3 to 5, characterized in that: The first weight corresponding to the context feature of the image to be processed is smaller than the second weight corresponding to the feature fusion result output by the previous feature extraction network cascaded by the feature extraction network; the second weight is smaller than the third weight corresponding to the feature extraction result of the feature extraction network.
7. A target detection device, characterized in that: The device comprises: An acquisition module, used to acquire an image to be detected carrying a target to be detected and a target detection model; the target detection model includes at least one cascaded feature extraction network and a target perception network; A feature extraction module, used for performing feature extraction on an image to be processed by any feature extraction network to obtain a feature extraction result of the feature extraction network; A feature fusion module is used to perform feature fusion on the feature extraction result of the feature extraction network and the context feature of the image to be processed to obtain the feature fusion result of the feature extraction network; wherein the image to be processed of the first feature extraction network is the image to be detected; the image to be processed of the non-first feature extraction network is the feature fusion result output by the previous feature extraction network in cascade; The target detection module is used to perform target perception on the feature fusion result output by the tail feature extraction network through the target perception network to obtain the target detection result of the target to be detected in the image to be detected.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.