Image target detection method, system and device for cloud control platform and medium

Abstract

The embodiment of the invention relates to the technical field of image processing, and discloses an image target detection method, system and device for a cloud control platform and a medium. Preprocessing the input image and extracting image features to obtain a feature map; inputting the feature map into a decoupling head module, decomposing the feature map into a plurality of sub-feature maps by the decoupling head module, and respectively performing a bbox branch prediction task, a classification branch prediction task and an IoU confidence branch prediction task; performing target detection frame prediction on the sub-feature map through a bbox branch, and outputting position information of the target detection frame; fusing the position information output by the bbox branch with the sub-feature maps corresponding to the classification branch and the IoU confidence coefficient branch to obtain a fused feature map; and according to the fused feature map, outputting a classification result and IoU confidence information of the target through a classification branch and an IoU confidence branch respectively. The technical problem of insufficient image detection accuracy can be solved at least.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image target detection method, system, device and medium for cloud control platforms. Background Technology

[0002] In recent years, autonomous driving technology has made significant progress, with image object detection algorithms playing a crucial role in image processing on cloud control platforms. The core task of image object detection algorithms is to identify target objects (such as vehicles, pedestrians, traffic signs, etc.) from images or videos and determine their location, category, and size. Accurate object detection results are essential for autonomous driving systems to make correct decisions and control. Especially in complex urban environments, improving the accuracy and robustness of object detection remains a key focus of current research and applications.

[0003] To address these issues in object detection, recent research has begun to explore decoupling the detection head design. Traditional object detection methods typically couple classification and regression tasks together, using the same features for simultaneous prediction. While this approach is simple to implement, it is prone to interference between the classification and regression tasks. The core idea of ​​decoupled detection head algorithms is to break down the classification and regression tasks into independent subtasks, each handled by a different branch. This method improves the accuracy and robustness of object detection by decoupling the mutual interference between tasks.

[0004] The decoupling head algorithm works by first extracting high-dimensional features from the feature map using a convolutional neural network (CNN), and then passing these features to multiple independent branches. Each branch handles a specific task, such as class prediction, bounding box regression, or target confidence prediction. This design not only optimizes the performance of classification and regression tasks but also effectively reduces the mutual influence between different tasks, thereby improving the accuracy of the entire detection system.

[0005] While decoupled detection head methods have achieved significant improvements in accuracy, some issues remain in practical applications. For example, the classification and regression branches do not fully utilize each other's information when processing target locations. This misalignment can lead to biases in model detection accuracy, especially when the target is complex or heavily occluded. Furthermore, if target bounding box (Bbox) location information, classification results, and IoU (Intersection over Union) confidence information are not effectively fused, it can negatively impact the final detection performance. Summary of the Invention

[0006] One objective of this application is to provide an image target detection method, system, device, and medium for a cloud control platform, at least to solve the technical problem of insufficient image detection accuracy.

[0007] To achieve the above objectives, some embodiments of this application provide the following aspects:

[0008] Firstly, some embodiments of this application also provide an image target detection method for a cloud control platform, comprising: preprocessing an input image to extract image features to obtain a feature map; inputting the feature map into a decoupling head module, wherein the decoupling head module decomposes the feature map into multiple sub-feature maps, and performs bounding box branch prediction, classification branch prediction, and IoU confidence branch prediction tasks respectively; predicting target detection boxes on the sub-feature maps through the bounding box branch, and outputting the position information of the target detection boxes; fusing the position information output by the bounding box branch with the sub-feature maps corresponding to the classification branch and the IoU confidence branch to obtain a fused feature map; and outputting the target classification result and IoU confidence information by the classification branch and the IoU confidence branch respectively, based on the fused feature map.

[0009] Secondly, some embodiments of this application also provide an image target detection system for a cloud control platform, including a feature extraction module for preprocessing an input image and extracting image features to obtain a feature map; a decoupling head module for decomposing the feature map into multiple sub-feature maps and performing bounding box branch prediction, classification branch prediction, and IoU confidence branch prediction tasks respectively; a feature fusion module for predicting target detection boxes on the sub-feature maps through the bounding box branch and outputting the position information of the target detection boxes; fusing the position information output by the bounding box branch with the sub-feature maps corresponding to the classification branch and the IoU confidence branch to obtain a fused feature map; and a prediction module for outputting the target classification result and IoU confidence information by the classification branch and the IoU confidence branch respectively, based on the fused feature map.

[0010] Thirdly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0011] Fourthly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods described above.

[0012] Compared with related technologies, the solution provided in this application introduces a feature fusion mechanism in the decoupled head structure of the cloud control platform to effectively fuse the location information of the target detection box with the classification branch and the IoU confidence branch, thereby optimizing the accuracy of classification results and bounding box regression. Through feature fusion, the classification branch and the IoU confidence branch can obtain the location information of the target box, thereby reducing interference between tasks and improving the accuracy of target detection. Attached Figure Description

[0013] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0014] Figure 1 This is a flowchart illustrating an image target detection method for a cloud control platform according to an embodiment of this application;

[0015] Figure 2 This is a flowchart illustrating an image target detection method for a cloud control platform according to an embodiment of this application.

[0016] Figure 3 This is a structural diagram of an image target detection system provided according to an embodiment of this application;

[0017] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The following terms are used in this document.

[0020] Object detection bounding box (Bbox): A Bbox is a directed rectangular box used in object detection to represent the position and size of an object. In object detection tasks, for each object in an image, a corresponding Bbox is pre-defined to represent the object's position and size within the image. A Bbox typically contains four attributes: x, y, w, and h, where x and y represent the offset of the Bbox's center point relative to the top-left corner of the image, and w and h represent the Bbox's width and height, respectively. During model prediction, by detecting objects at multiple locations or scales within the image, target objects can be identified.

[0021] IOU (Intersection over Union): IOU, short for Intersection over Union, is an important concept used in object detection. It calculates the overlap ratio between the predicted bounding boxes and the ground truth bounding boxes, that is, the ratio of their intersection to their union. This ratio is used to measure the degree of overlap between the predicted and ground truth bounding boxes, thereby evaluating the accuracy of object detection.

[0022] Specifically, the formula for calculating IOU is: IOU = Area of ​​intersection between predicted and ground truth boxes / Area of ​​union between predicted and ground truth boxes. In object detection tasks, the IOU value between the bounding box (Bbox) and the ground truth box is typically used to judge the quality of the prediction result. The closer the IOU value is to 1, the higher the overlap between the predicted and ground truth boxes, and the better the prediction effect. Generally, an IOU greater than 0.5 is considered a good prediction result.

[0023] mAP: The higher the mAP value, the better the performance of the object detection model. mAP is one of the important metrics for measuring the performance of an object detection model. It evaluates the model's detection ability across all categories by calculating the average of the mean precision (AP).

[0024] First Embodiment

[0025] The first embodiment of this application relates to an image target detection method for a cloud control platform. For example... Figure 1 As shown, the method may include the following steps:

[0026] S101: After preprocessing the input image, extract the image features to obtain the feature map.

[0027] The cloud-based control platform can extract meaningful high-dimensional feature information from input images through preprocessing and feature extraction. This feature information can fully reflect the target objects in the image and provide basic data for subsequent object detection tasks. By using deep learning models (such as convolutional neural networks), this step can effectively capture key features in the image, enhancing the effectiveness and accuracy of subsequent model processing.

[0028] S102, the feature map is input into the decoupling head module, which decomposes the feature map into multiple sub-feature maps and performs bbox branch prediction task, classification branch prediction task and IoU confidence branch prediction task respectively.

[0029] By inputting the feature map into the decoupling head module, the feature map is decomposed into multiple sub-feature maps, thereby assigning independent processing paths to different tasks. This decoupling design separates the classification task, bounding box regression task, and IoU confidence task, reducing interference between tasks and improving the focus of each task. This allows each task to utilize specific features for more accurate predictions, improving the accuracy of object detection.

[0030] S103, predict the target detection box by using the bbox branch on the sub-feature map, and output the position information of the target detection box.

[0031] The bounding box (bbox) branch is used to predict the bounding boxes for object detection, directly outputting the location information of each object (such as the coordinates of the bounding box). Using a separate bbox branch ensures that the regression task can focus on accurate object location prediction without being affected by other tasks, thus reducing prediction errors. This step effectively improves the localization accuracy of the object detection boxes, providing precise location information support for subsequent classification and IoU confidence prediction.

[0032] S104, the position information output by the bbox branch is fused with the sub-feature maps corresponding to the classification branch and the IoU confidence branch to obtain a fused feature map.

[0033] The location information output by the bounding box (bbox) branch is effectively fused with the sub-feature maps of the classification branch and the IoU confidence branch. This fusion process allows the classification branch and the IoU confidence branch to fully utilize the location information of the target bounding box during prediction, thereby improving the accuracy of classification and the prediction quality of the IoU confidence. Through this feature fusion, classification results can be optimized, interference between tasks can be reduced, and the overall performance of object detection can be improved.

[0034] S105, Based on the fused feature map, the classification branch and IoU confidence branch respectively output the target classification result and IoU confidence information.

[0035] By fusing feature maps, the classification branch and the IoU confidence branch can output more accurate classification results and confidence information. The accuracy of the classification results is improved, and the prediction of IoU confidence becomes more reliable. Classification and regression tasks are no longer processed independently, but can work collaboratively based on shared fused features, thereby improving the performance of the entire object detection system, especially exhibiting higher robustness and accuracy in complex environments.

[0036] By fusing the location information obtained from the bounding box branch into the feature maps of other branches, the prediction accuracy of the classification branch and the IoU confidence branch is significantly improved. Through the effective fusion of bounding box information, the classification branch and the IoU confidence branch can utilize the location information of the target box to perform more accurate classification prediction and IoU confidence calculation, effectively reducing false positives and false negatives.

[0037] Furthermore, the technical solution of this application is flexible. Simple additions or deletions to other branch structures, such as adding branches to handle other tasks (e.g., target rotation angle prediction, target scale estimation, etc.) or reducing existing branches, and fusing the position information obtained from the bbox branches to improve prediction accuracy, still fall within the protection scope of this application. This flexible design can adapt to different task requirements and has strong scalability and adaptability, representing an improvement over existing decoupled detection head structures.

[0038] It is readily apparent that, compared to related technologies, the solution provided in this application employs a decoupled head design and feature fusion technology, which effectively improves the accuracy and efficiency of the object detection algorithm. This method reduces interference between classification, regression, and confidence prediction tasks, allowing each task to be accurately computed in an independent branch, and optimizes the final output through feature fusion. Experimental results demonstrate that this method significantly improves the accuracy and robustness of object detection, especially in complex application scenarios such as autonomous driving on cloud control platforms and roadside perception, exhibiting stronger adaptability and performance.

[0039] Second Embodiment

[0040] The second embodiment of this application relates to an image target detection method for a cloud control platform. The second embodiment is an improvement upon the first embodiment, specifically in that:

[0041] Furthermore, the process of extracting image features after preprocessing the input image includes: after preprocessing the input image, extracting image features through the body network and neck network in a convolutional neural network to obtain a feature map.

[0042] Image preprocessing is a crucial preliminary step in object detection. Preprocessing removes noise, standardizes image size, improves contrast, and enhances image features. These processes improve the quality of the input image, leading to more accurate feature extraction in subsequent steps. Common preprocessing techniques include normalization, grayscale conversion, and data augmentation, which enhance image usability and improve the generalization ability of detection models.

[0043] In Convolutional Neural Networks (CNNs), the body network is primarily responsible for extracting deep features from the input image. This network typically consists of multiple convolutional layers, pooling layers, and activation layers, progressively extracting both local and global features of the image. In deep networks, the first few layers mainly capture low-level features (such as edges, textures, and colors), while later layers gradually learn higher-level, more abstract semantic information. These features effectively represent the shape, color, and position of the target object, thus laying the foundation for accurate object detection.

[0044] By using a body network for feature extraction, object detection models can learn rich features from images. These features include edges, shapes, and textures, making them particularly suitable for object detection in complex scenes, such as detecting multiple types of objects like pedestrians, vehicles, and traffic signs. This process helps the network fully understand the image content, providing accurate information support for subsequent detection tasks.

[0045] In CNNs, the neck network connects the body network and the detection head, and is responsible for further fusing and multi-scale processing of the features extracted by the body network. Through the neck network, features in the image can be better aggregated, generating feature maps suitable for multi-scale object detection. In object detection, targets vary greatly in size and shape; therefore, multi-scale feature extraction can effectively solve the problem of detecting targets of different sizes, ensuring that the model has good detection capabilities for both small and large targets.

[0046] Neck networks typically include structures such as FPN (Feature Pyramid Network) and PANet (Path Aggregation Network). These structures can fuse feature maps of different scales, enhancing the representation of targets at different scales. For example, FPN transmits high-level features through a top-down path and then supplements details through a bottom-up path, effectively fusing low-level and high-level features and improving the detection capability of multi-scale targets.

[0047] Through the combined action of the body network and the neck network, a feature map rich in information is finally obtained. This feature map contains multi-level feature information of various targets in the image, including target information at different scales, positional information, and texture information. The quality of the feature map directly affects the performance of subsequent object detection. A refined feature map can provide more accurate input to the detection head, improving the accuracy of target bounding box localization, classification, and IoU confidence prediction.

[0048] Further, obtaining the fused feature map includes: concatenating the positional information output by the bbox branch along the channel direction at the feature level to the sub-feature maps corresponding to the classification branch and the IoU confidence branch, respectively, to obtain the fused feature map.

[0049] By concatenating the output location information of the bounding box branch (i.e., the location information of the target detection box) with the sub-feature maps corresponding to the classification branch and the IoU confidence branch, the location information can be effectively passed as an important feature to the classification branch and the IoU confidence branch. This fusion process enables the classification task and the IoU prediction task not only to make predictions based on their original features, but also to optimize them by utilizing the target's location information.

[0050] For example, when predicting the category of a target, the classification branch can refer to the target's spatial location information, thus avoiding classification errors caused by target location offset or ambiguity. Similarly, the IoU confidence branch can better predict the intersection-union ratio (IoU) between the target box and the ground truth box, improving prediction accuracy.

[0051] In object detection, the location information of the target (such as the bounding box position) is crucial for target identification and localization. By concatenating location information along the channel direction at the feature level, the classification branch and the IoU confidence branch can share the target's location information, allowing each branch to accurately analyze the target from different perspectives. The concatenation operation is not merely an overlay of information; it integrates location information into the feature map, enhancing the expressive power of the features. This feature concatenation method allows the classification branch and the IoU confidence branch to adjust their predictions using the target location information, thereby improving their prediction accuracy. For example, the location information of the object detection box directly affects the target's boundary localization. If the classification branch knows the target's location, it can more accurately determine the target's category; the IoU confidence branch can also use the location information to determine the degree of overlap between the predicted box and the ground truth box, thus outputting a more accurate Intersection over Union (IoU).

[0052] Furthermore, obtaining the fused feature map includes: fusing the location information with the sub-feature maps corresponding to the classification branch and the IoU confidence branch through an attention mechanism to obtain the fused feature map.

[0053] Attention mechanisms automatically assign weights to different features based on their importance. When location information is fused with sub-feature maps of the classification and IoU confidence branches through the attention mechanism, it automatically adjusts the fusion method between location information and other features. The model can automatically learn which location features are most important for classification and IoU confidence prediction, thus fusing features more accurately and improving detection performance.

[0054] For example, certain targets are more discriminative in certain regions of an image (such as near the image center or a specific background area), while location information in other regions may have less impact on target classification or localization. Through attention mechanisms, the model can dynamically adjust the weights of feature fusion, focusing more attention on important regions, thereby improving the prediction accuracy of classification and IoU confidence.

[0055] Traditional feature fusion methods may fail to fully capture the complex relationships between location information and other features. By using an attention mechanism, the model can adaptively focus on features at different locations in the image during feature fusion, enabling it to better understand the spatial distribution and positional relationships of targets. Especially in complex scenes, the attention mechanism can help the model accurately capture the location information of targets even when the target detection bounding boxes are compact, the background is complex, or the target is partially occluded, thereby effectively improving the target detection accuracy.

[0056] The classification branch and the IoU confidence branch have different needs and focuses regarding location information. The classification branch focuses more on the category features of the target, while the IoU confidence branch focuses more on the degree of overlap between the target bounding box and the ground truth bounding box. In traditional methods, feature fusion is usually performed using fixed methods such as concatenation and weighted averaging. However, using an attention mechanism can adaptively adjust the feature fusion method according to the specific needs of the task, ensuring that different branches obtain feature information suitable for their respective task requirements.

[0057] Furthermore, obtaining the fused feature map includes: cropping the features at the corresponding positions based on the position information of each target detection box, reducing the dimensionality through convolution and max pooling to obtain the fused feature map.

[0058] Based on the location information of the target detection box, the features of the region where the target is located can be accurately cropped from the feature map, avoiding interference from irrelevant region features on model learning. The boundary information of the target can be determined by the location information of the detection box, further improving the targeting and efficiency of feature extraction.

[0059] This object detection bounding box-based cropping method helps extract local information directly related to the target while avoiding interference from the background in the image. This effectively reduces redundant information in the feature map, thereby improving the accuracy of subsequent tasks (such as classification, localization, and IoU calculation).

[0060] After cropping the features of the target region, convolutional operations are used to further process these features, extracting higher-level and more discriminative features. Convolutional operations can effectively learn more useful information from local regions and perform feature fusion at multiple levels to adapt to the diversity and complexity of targets. Convolutional operations can also spatially smooth the feature maps, further suppressing noise and background interference, ensuring that only key information of the target is retained. This is particularly important for small targets and complex scenes in object detection tasks.

[0061] Max pooling is a common dimensionality reduction technique that reduces the dimensionality of features by selecting the maximum value in each local region to represent the feature of that region, while retaining the most important information. In this embodiment, through max pooling, the model can effectively compress the spatial dimension of the feature map, reduce computational complexity, and retain the key information of the target. This makes the fused feature map more concise and discriminative, facilitating the efficient completion of subsequent tasks.

[0062] By combining convolution and max pooling, the dimensionality of the feature map is significantly reduced, decreasing the need for subsequent computation and storage, thereby improving the model's inference speed and computational efficiency. This improvement in computational efficiency is particularly important in real-time detection or large-scale data processing scenarios. Convolution and pooling operations enhance the network's perception of local region features, especially when the target is highly similar to the background or when the target is partially occluded. Convolutional layers can better extract the core features of the target, and max pooling highlights key spatial information, thus improving the target detection accuracy.

[0063] This local feature extraction method can maintain good discriminative power for targets even in complex backgrounds. Especially for small or edge targets, convolution and max pooling operations can help the model better separate target features.

[0064] Furthermore, the method also includes: post-processing the classification results and IoU confidence information during inference using non-maximum suppression, and redundant detection boxes.

[0065] In object detection tasks, models may generate multiple overlapping bounding boxes for the same object. Non-maximum suppression (NMS) can be used to select the bounding box based on its confidence level (IoU value and classification confidence), retaining the box with the highest confidence and removing redundant bounding boxes, thus avoiding the problem of the same object being detected multiple times.

[0066] Non-maximum suppression (NVS) sorts the scores of each detection box and then progressively removes low-scoring boxes that have a high degree of overlap with the current box, thus reducing the number of boxes in the final output and ensuring that there are no duplicates. This effectively reduces the computational cost of the post-processing steps, reduces meaningless duplicate predictions, and improves the computational efficiency of the model.

[0067] In object detection, especially when objects are densely packed or targets are close together, overlapping of multiple detection boxes can easily occur, leading to redundant detection results. Non-maximum suppression (NMS) filters out boxes that significantly overlap with existing high-confidence boxes, effectively reducing false alarms and false positives, and improving detection accuracy. The NMS algorithm sets a certain IoU threshold (e.g., 0.5), retaining only two boxes if their IoU is below this threshold. This threshold can be adjusted according to specific application requirements, thus ensuring detection accuracy while avoiding over-detection of targets.

[0068] Removing redundant bounding boxes helps reduce subsequent processing steps and storage consumption. After non-maximum suppression, the number of output detection boxes is typically reduced significantly, effectively lowering the computational complexity of subsequent operations and improving inference speed. Especially in real-time object detection tasks, it can significantly improve overall inference efficiency and meet the real-time requirements of practical applications.

[0069] Furthermore, the generation of the fused feature map employs a method based on fully connected layers or feature weighting to fuse features.

[0070] A fully connected layer (FC) is a typical neural network layer used to transform input feature maps into a compact feature representation. By fusing different feature maps through a fully connected layer, information from each sub-feature map can be integrated into a unified feature space, thus forming a new fused feature map.

[0071] Fully connected layers connect and map the channels of sub-feature maps to a new feature space. This operation effectively fuses feature information from different sources, enabling subsequent classification and regression tasks to make decisions based on more comprehensive features. The application of fully connected layers can learn appropriate weights to control the influence of different features, further improving the accuracy of object detection.

[0072] The feature weighting method obtains a fused feature map by weighted summation of channels in different feature maps. In this method, each sub-feature map is evaluated using a learned weight coefficient to measure its contribution to the final output. Through this weighted fusion approach, the network can adjust the influence of each feature map based on its effectiveness, thereby optimizing feature representation.

[0073] like Figure 2 As shown, the input image undergoes preprocessing, such as normalization and data augmentation, to improve image quality and enhance the model's robustness. A deep convolutional neural network (CNN) is used to extract image features, resulting in a basic feature map of shape H×W×256, where H and W are the height and width of the feature map, respectively, and 256 represents the number of channels. This basic feature map provides rich visual features for subsequent tasks, including information such as object edges, textures, and spatial locations.

[0074] After being processed by the decoupled detection head, the feature map is decomposed into three independent task branches: classification branch (Cls.), bounding box regression branch (Reg.), and IoU confidence branch (IoU). These three branches process the feature map respectively to complete their respective prediction tasks.

[0075] Classification Branch: Extracts a set of sub-feature maps (H×W×256) for classification from the base feature map. The classification branch extracts features from the input features through two convolution operations (the feature map after each convolution operation still maintains a size of H×W×256), further enhancing the semantic information of the features and extracting features that are helpful for class determination. The final output feature map is H×W×C, where C represents the number of target classes.

[0076] The bounding box regression branch (bbox branch) extracts a set of sub-feature maps (H×W×256) from the base feature map for bounding box regression. The regression branch also uses two convolution operations to further extract features related to the target location and size (the feature map size after each convolution operation remains H×W×256). The final output feature map is H×W×4, where 4 represents the four coordinate parameters of the bounding box: center point (x, y) and width and height (w, h).

[0077] The IoU confidence branch extracts a sub-feature map (H×W×256) from the base feature map. This branch extracts features related to the overlap (IoU) between the target bounding box and the ground truth box through two convolutional operations. The final output feature map is H×W×1, where 1 represents the IoU confidence score.

[0078] To achieve task alignment among the classification branch, regression branch, and IoU confidence branch, the location information of the bounding box regression branch (Reg.) is fused with the features of other branches, enabling the classification and IoU confidence branches to utilize the location features of the target bounding box, thereby improving detection accuracy.

[0079] Extract the output feature map (H×W×4) from the bounding box regression branch. Concatenate this feature map along the channel direction with the feature maps of the classification branch and the IoU confidence branch respectively. For the classification branch, concatenate the location information with the feature map of the classification branch to generate a fused feature map.

[0080] For the IoU confidence branch, the location information predicted by the bounding box regression branch is concatenated with the feature map of the IoU branch to generate a fused feature map. Through feature concatenation, both the classification branch and the IoU confidence branch can obtain bounding box location information, thereby improving classification accuracy and IoU prediction accuracy.

[0081] The concatenated feature maps are further processed by subsequent convolutional layers to extract features, making the fused feature maps more suitable for classification and IoU confidence prediction tasks: the final output of the fused feature map in the classification branch is H×W×C (target classification result). The final output of the fused feature map in the IoU confidence branch is H×W×1 (IoU confidence prediction result).

[0082] After completing classification, regression, and IoU confidence prediction, the detected bounding boxes output by the model undergo post-processing. Based on the classification results of the classification branch and the confidence scores of the IoU branch, the final detected bounding boxes are selected. Non-maximum suppression removes redundant boxes with significant overlap by setting an IoU threshold (e.g., 0.5), retaining only the detected bounding boxes with the highest confidence, thus reducing the problem of duplicate detections.

[0083] It is not difficult to see that in the embodiments of this application, in the application scenario - autonomous driving system and roadside perception system, training datasets and test sets of the corresponding scenarios are collected. The detection effect of the method on the test set shows that, under the premise that the inference speed remains almost unchanged, the mAP is improved by about 1.4%, which significantly improves the accuracy of image target detection.

[0084] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0085] Third Embodiment

[0086] The third embodiment of this application relates to an image target detection system for a cloud control platform, such as... Figure 3 As shown, it includes:

[0087] The feature extraction module is used to extract image features after preprocessing the input image to obtain a feature map;

[0088] The decoupling head module is used to decompose the feature map into multiple sub-feature maps, and perform bounding box branch prediction task, classification branch prediction task and IoU confidence branch prediction task respectively;

[0089] The feature fusion module is used to predict the target detection box in the sub-feature map through the bbox branch and output the position information of the target detection box; and to fuse the position information output by the bbox branch with the sub-feature map corresponding to the classification branch and the IoU confidence branch to obtain the fused feature map.

[0090] The prediction module is used to output the classification result and IoU confidence information of the target based on the fused feature map, respectively by the classification branch and the IoU confidence branch.

[0091] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0092] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problem proposed in this application; however, this does not mean that other units are absent from this embodiment.

[0093] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0094] The electronic device includes: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments. Figure 4 An exemplary structural diagram of the electronic device is disclosed. For example... Figure 4 As shown, the electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0095] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103, and output device 1104 may be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0096] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0097] To provide interaction with the user, the electronic device can be a computer. This computer includes: a display device for showing information to the user (e.g., a cathode-ray tube (CRT) or an LCD monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0098] In this embodiment, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0099] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0100] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0101] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0102] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random-access memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0103] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an Application-Specific Integrated Circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0105] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive, SSD, etc.).

[0106] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0107] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0108] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for image object detection for a cloud control platform, the method comprising: The method comprises: extracting image features after preprocessing the input image to obtain a feature map; inputting the feature map into a decoupling head module, which decomposes the feature map into multiple sub-feature maps to respectively perform a bbox branch prediction task, a classification branch prediction task, and an IoU confidence branch prediction task; predicting a target bounding box for the sub-feature map through the bbox branch to output position information of the target bounding box; fusing the position information output by the bbox branch with the sub-feature maps corresponding to the classification branch and the IoU confidence branch to obtain a fused feature map; outputting a classification result and IoU confidence information of a target by the classification branch and the IoU confidence branch respectively according to the fused feature map.

2. The method of claim 1, wherein, The step of extracting image features after preprocessing the input image comprises: extracting image features through a body network and a neck network in a convolutional neural network after preprocessing the input image to obtain a feature map.

3. The method of claim 1, wherein, The step of obtaining a fused feature map comprises: concatenating the position information output by the bbox branch to the sub-feature maps corresponding to the classification branch and the IoU confidence branch respectively along the channel direction at the feature level to obtain a fused feature map.

4. The method of claim 1, wherein, The step of obtaining a fused feature map comprises: fusing the position information with the sub-feature maps corresponding to the classification branch and the IoU confidence branch through an attention mechanism to obtain a fused feature map.

5. The method of claim 1, wherein, The step of obtaining a fused feature map comprises: cropping features at corresponding positions according to the position information of each target bounding box, reducing the dimension through convolution and maximum pooling to obtain a fused feature map.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: post-processing the classification result and the IoU confidence information through non-maximum suppression during inference to remove redundant bounding boxes.

7. The method of claim 6, wherein, The fused feature map is generated by fusing features based on a full connection layer or feature weighting.

8. An image target detection system for a cloud control platform, characterized in that, The system comprises: a feature extraction module configured to extract image features after preprocessing an input image to obtain a feature map; a decoupling head module configured to decompose the feature map into multiple sub-feature maps to respectively perform a bbox branch prediction task, a classification branch prediction task, and an IoU confidence branch prediction task; a feature fusion module configured to predict a target bounding box for the sub-feature map through the bbox branch to output position information of the target bounding box, and fuse the position information output by the bbox branch with the sub-feature maps corresponding to the classification branch and the IoU confidence branch to obtain a fused feature map; a prediction module configured to output a classification result and IoU confidence information of a target by the classification branch and the IoU confidence branch respectively according to the fused feature map.

9. An electronic device, comprising: The electronic device comprises: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the method of any one of claims 1 to 7.

10. A computer readable medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method of any one of claims 1 to 7.

Images