Target detection method and device, computer device and computer readable storage medium

Abstract

The application relates to a target detection method and device, computer equipment and a computer readable storage medium. The method comprises the following steps: determining a shallow feature and a plurality of deep features with different resolutions of a to-be-detected image; splicing the shallow feature and a first target deep feature to obtain a spliced feature; the first target deep feature is a deep feature with a resolution matched with the shallow feature in the plurality of deep features; fusing a non-target deep feature and a deep feature with a higher resolution in the corresponding plurality of deep features to obtain a deep fusion feature; the non-target deep feature refers to a deep feature except the first target deep feature; performing advanced fusion on the spliced feature, the deep fusion feature and a second target deep feature to obtain an advanced fusion feature; the second target deep feature refers to a non-target deep feature with the lowest resolution; and performing target detection on the to-be-detected image based on the advanced fusion feature to obtain a detection result. The application can improve the accuracy of target detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a target detection method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] With the development of image processing technology, object detection technology has emerged, which aims to accurately detect and locate specific target objects from images or videos. The main task of object detection is to determine the category and location of the target, usually using bounding boxes or pixel-level masks to represent the target's location.

[0003] Traditional target detection techniques often perform poorly when dealing with small targets. This is because small targets often have inconspicuous features in images, weak feature signals, and are easily interfered with, making accurate detection difficult. Summary of the Invention

[0004] Therefore, it is necessary to provide a target detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product to address the aforementioned technical problems and improve the accuracy of target detection.

[0005] Firstly, this application provides a target detection method, including:

[0006] Determine the shallow features and multiple deep features at different resolutions of the image to be detected;

[0007] The shallow features are spliced ​​together with the deep features of the first target to obtain the spliced ​​features; the deep features of the first target are the deep features among multiple deep features whose resolution matches that of the shallow features;

[0008] The non-target deep features are fused with the higher-resolution deep features among the corresponding multiple deep features to obtain deep fused features; the non-target deep features refer to the deep features other than the first target deep feature among the multiple deep features.

[0009] Advanced fusion is performed on the splicing features, deep fusion features, and second target deep features to obtain advanced fusion features; the second target deep features refer to the non-target deep features with the lowest resolution.

[0010] Target detection is performed on the image to be detected based on advanced fusion features, and the detection results are obtained.

[0011] Secondly, this application provides a target detection device, comprising:

[0012] The determination module is used to determine the shallow features and multiple deep features at different resolutions of the image to be detected.

[0013] The stitching module is used to stitch shallow features with deep features of the first target to obtain stitched features; the deep features of the first target are deep features among multiple deep features whose resolution matches that of the shallow features;

[0014] The first fusion module is used to fuse non-target deep features with higher-resolution deep features among multiple corresponding deep features to obtain deep fused features; non-target deep features refer to deep features other than the first target deep feature among multiple deep features;

[0015] The second fusion module is used to perform advanced fusion of splicing features, deep fusion features, and second target deep features to obtain advanced fusion features; the second target deep features refer to the non-target deep features with the lowest resolution.

[0016] The detection module is used to perform target detection on the image to be detected based on advanced fusion features and obtain the detection results.

[0017] Thirdly, this application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method described above.

[0018] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0019] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described above.

[0020] The aforementioned target detection methods, devices, computer equipment, computer-readable storage media, and computer program products primarily focus on the shallow features of the image to be detected, particularly the detailed texture. Deep features at different resolutions emphasize the semantic information of the image. Since small targets have weaker semantic information but stronger texture information, and texture information is more intuitive and significant than semantic information, it plays a more important role in the detection of small targets. By stitching together shallow features with first target deep features that match the resolution, the resulting stitched features possess both rich texture and semantic information. Lower-resolution deep features are coarser but larger-scale semantic features, while higher-resolution deep features are finer but smaller-scale semantic features. Fusing non-target deep features with the highest-resolution deep features from among multiple corresponding deep features yields deep fused features with richer semantic information. Further fusion of the stitched features, deep fused features, and second target deep features yields advanced fused features with even richer texture and semantic information. Furthermore, the second target deep feature refers to the lowest resolution non-target deep feature. Since the lowest resolution non-target deep feature cannot form a deep fusion feature with even lower resolution deep features, the advanced fusion introduces the lowest resolution non-target deep feature to achieve full-resolution feature fusion, providing rich global information. Therefore, based on the advanced fusion feature, target detection in the image can better focus on the target in the image, resulting in more accurate detection results and improving the accuracy of target detection. Attached Figure Description

[0021] Figure 1 A schematic flowchart of a target detection method provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the structure of a backbone network provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of a target detection model provided in an embodiment of this application;

[0024] Figure 4 A structural block diagram of a target detection device provided in an embodiment of this application;

[0025] Figure 5 An internal structural diagram of a computer device provided in an embodiment of this application;

[0026] Figure 6 An internal structural diagram of another computer device provided in an embodiment of this application;

[0027] Figure 7 This is an internal structural diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] like Figure 1 As shown in the illustration, this application provides a target detection method, which will be described using an application of the method to a computer device as an example. The method includes the following steps:

[0030] S102. Determine the shallow features and multiple deep features of different resolutions of the image to be detected.

[0031] In some embodiments, the image to be detected may be, but is not limited to, a high-exposure image. It is understood that high-exposure images are more effective at imaging small and elongated targets. For example, high-exposure images can be used to image fine hairs.

[0032] In some embodiments, the computer device may perform initial convolution processing on the image to be detected to obtain shallow features. Based on the shallow features, progressive downsampling and feature fusion processing are performed to obtain multiple deep features at different resolutions.

[0033] In some embodiments, the computer device may acquire multiple initial deep features of different resolutions of the image to be detected. Convolution processing is performed on the initial deep features at each resolution to obtain the deep features for that specific resolution.

[0034] In some embodiments, a computer device can perform multi-stage feature extraction on the image to be detected using multiple branch networks corresponding to different resolutions, thereby obtaining shallow features and multiple initial deep features at different resolutions.

[0035] In some embodiments, the computer device may sequentially perform convolution, normalization, and activation operations on the initial deep features at each resolution to obtain the deep features at each resolution.

[0036] In some embodiments, the computer device can acquire shallow features output by the branch network corresponding to the highest resolution in the first stage. From the second stage to the last stage, in each stage, the input of the branch network corresponding to each resolution includes the output of the branch network corresponding to the higher resolution after downsampling in the previous stage, and the output of the branch network corresponding to the lower resolution after upsampling in the previous stage. Multiple initial deep features output by the branch networks corresponding to different resolutions in the last stage are acquired. From the first stage to the last stage, these correspond to the shallowest to the deepest layers. The smaller the resolution corresponding to the branch network, the larger the number of channels in the features output by the branch network.

[0037] In some embodiments, a computer device may input an image to be detected into a backbone network to obtain shallow features output by the backbone network in a first stage, and multiple initial deep features at different resolutions output in a final stage.

[0038] In some embodiments, the backbone network may be, but is not limited to, a lightweight, high-resolution network (High-Resolution Network-small). For example... Figure 2 The diagram shows a schematic of the backbone network. The backbone network consists of four branch networks, each corresponding to a different resolution. After the image to be detected is input into the backbone network, the shallow feature O0 output by the branch network corresponding to the highest resolution in the first stage is obtained. Four initial deep features O1, O2, O3, and O4, based on the different resolutions output by the four branch networks in the fourth stage, are then obtained.

[0039] In some embodiments, the image to be detected contains elongated targets. Elongated targets have weak semantic information but strong texture information. Combining shallow features can obtain more refined texture information, making the detection of elongated targets more accurate.

[0040] In some embodiments, the elongated target can be a filamentous or hair-like target, such as a fuzz. For example, if a workpiece is shown in the image to be detected, and fuzz inevitably adheres to the workpiece during its production, the method provided in this application embodiment can accurately detect the fuzz on the workpiece.

[0041] In some embodiments, the computer device may include at least one of a terminal and a server. It is understood that the method provided in this application embodiment can be applied to a terminal or a server, or to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal may be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices may include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0042] S104. The shallow features are spliced ​​together with the deep features of the first target to obtain the spliced ​​features; the deep features of the first target are the deep features among multiple deep features whose resolution matches that of the shallow features.

[0043] In some embodiments, the computer device can stitch together the highest resolution shallow features and the highest resolution deep features to obtain the highest resolution stitched features.

[0044] In some embodiments, the resolution corresponding to the deep features of the first target is not lower than the resolution corresponding to the shallow features. The computer device can stitch the upsampled shallow features with the higher-resolution deep features of the first target to obtain a higher-resolution stitched feature. Alternatively, it can directly stitch the shallow features with the first target deep features of the same resolution to obtain a stitched feature of that resolution.

[0045] It's understandable that shallow features primarily contain low-level image information, such as texture information, while higher-resolution deep features contain higher-level semantic information. By concatenating these two, we can fully utilize the low-level details and textures of the image, as well as its high-level semantic information, to better understand the target. However, concatenating shallow features with lower-resolution deep features may yield relatively poor results because low-resolution deep features lack detailed information, potentially introducing more noise and blurring into the feature concatenation, thus reducing accuracy. Furthermore, concatenating shallow features with deep features of all resolutions—this full-scale feature concatenation—may introduce a large amount of redundant information, increasing computational complexity, and may over-smooth the details and contextual information of the target, leading to less precise and accurate results.

[0046] In some embodiments, the computer device can initially concatenate shallow features with deep features of the first target to obtain initial concatenated features, and then perform convolution processing on the initial concatenated features to obtain concatenated features. It can be understood that the initial concatenation simply concatenates the shallow features with the deep features of the first target together, and after convolution processing, obtains concatenated features with the same number of feature channels as the deep features.

[0047] In some embodiments, a computer device may sequentially perform convolution, normalization, and activation operations on the initial splicing features to obtain splicing features.

[0048] S106. Fuse the non-target deep features with the higher resolution deep features among the corresponding multiple deep features to obtain deep fused features; the non-target deep features refer to the deep features other than the first target deep features among the multiple deep features.

[0049] Among the multiple deep features corresponding to the non-target deep features, the higher resolution deep feature refers to the deep feature with a higher resolution than the deep feature corresponding to the non-target deep feature.

[0050] In some embodiments, the computer device can upsample the non-target deep features at each resolution, and then fuse the upsampled non-target deep features at each resolution with the higher-resolution deep features among the corresponding multiple deep features to obtain higher-resolution fused deep features. It is understood that the lowest-resolution non-target deep features cannot form the lowest-resolution fused deep features with lower-resolution deep features. The aforementioned higher-resolution fused deep features do not include the lowest-resolution fused deep features. Subsequent advanced fusion requires the introduction of the lowest-resolution non-target deep features to achieve full-resolution feature fusion.

[0051] In some embodiments, the computer device may fuse non-target deep features at each resolution with deep features at the previous resolution at each resolution to obtain deep fused features at the previous resolution at each resolution.

[0052] In some embodiments, the computer device may fuse non-target deep features at each resolution with non-target deep features at the previous resolution at each resolution to obtain deep fused features at the previous resolution at each resolution.

[0053] In some embodiments, the computer device may upsample the non-target deep features at each resolution, and then superimpose the upsampled non-target deep features at each resolution with the non-target deep features at the previous resolution to obtain the deep fusion features at the previous resolution.

[0054] In some embodiments, the computer device may sequentially perform convolution, normalization, activation and upsampling operations on the non-target deep features at each resolution to obtain upsampled non-target deep features at each resolution.

[0055] In some embodiments, the computer device may perform a point-to-point addition operation on the non-target deep features of each upsampled resolution and the non-target deep features of the previous resolution of each resolution to obtain the deep fused features of the previous resolution of each resolution.

[0056] S108. Perform advanced fusion on the splicing features, deep fusion features, and second target deep features to obtain advanced fusion features; the second target deep features refer to the non-target deep features with the lowest resolution.

[0057] In some embodiments, the computer device may fuse the deep fusion features of each resolution with the stitching features of the previous resolution for each resolution to obtain the stitching fusion features of the previous resolution for each resolution. The stitching fusion features of different resolutions, the deep fusion features of different resolutions, and the deep features of the second target are then fused to obtain advanced fusion features.

[0058] In some embodiments, multiple initial deep features at different resolutions are convolved to obtain multiple deep features with the same number of feature channels. For example, the number of feature channels for multiple deep features is 64.

[0059] In some embodiments, the computer device can perform channel-by-channel stitching of various stitched and blended features at different resolutions, various deep-layer blended features at different resolutions, and a second target deep-layer feature to obtain an advanced blended feature. For example, when the number of feature channels for a stitched feature and the three deep-layer features are both 64, the number of channels for the advanced blended feature is 256.

[0060] In some embodiments, the computer device can upsample the stitching and fusion features, multiple deep fusion features at different resolutions, and the second target deep feature, so that the upsampled stitching and fusion features, the arbitrary upsampled deep fusion features, and the upsampled second target deep feature correspond to the same resolution. For example, the stitching and fusion features, multiple deep fusion features, and the second target deep feature can all be upsampled to the highest resolution.

[0061] In some embodiments, the computer device may sequentially perform convolution, normalization, activation, and upsampling operations on the stitched and fused features to obtain upsampled stitched and fused features. After sequentially performing convolution, normalization, activation, and upsampling operations on each deep fused feature, multiple upsampled deep fused features are obtained. After sequentially performing convolution, normalization, activation, and upsampling operations on the second target deep feature, upsampled second target deep feature is obtained.

[0062] In some embodiments, the computer device can perform channel splicing of upsampled stitched fusion features, multiple upsampled deep fusion features, and upsampled second target deep features to obtain advanced fusion features.

[0063] S110. Target detection is performed on the image to be detected based on advanced fusion features to obtain the detection results.

[0064] In some embodiments, the computer device may perform convolution processing on the advanced fusion features to obtain the detection result. The detection result is used to characterize the target in the image to be detected.

[0065] In some embodiments, the computer device may sequentially perform convolution and upsampling operations on the advanced fusion features, such that the features after the convolution operation are upsampled to the size of the image to be detected, thereby obtaining the detection result.

[0066] As can be seen, in this embodiment, the shallow features of the image to be detected mainly focus on the image's detailed texture. Deep features at different resolutions emphasize the image's semantic information. Since small targets have weaker semantic information but stronger texture information, and texture information is more intuitive and significant than semantic information, it plays a more important role in the detection of small targets. By stitching together shallow features with the first target deep features that match the resolution, the resulting stitched features possess both rich texture and semantic information. Lower-resolution deep features are coarser but larger-scale semantic features, while higher-resolution deep features are finer but smaller-scale semantic features. Fusing non-target deep features with the higher-resolution deep features from among multiple corresponding deep features yields deep fused features with richer semantic information. Further fusion of the stitched features, deep fused features, and second target deep features yields advanced fused features with even richer texture and semantic information. Furthermore, the second target deep feature refers to the lowest resolution non-target deep feature. Since the lowest resolution non-target deep feature cannot form a deep fusion feature with even lower resolution deep features, the advanced fusion introduces the lowest resolution non-target deep feature to achieve full-resolution feature fusion, providing rich global information. Therefore, based on the advanced fusion feature, target detection in the image can better focus on the target in the image, resulting in more accurate detection results and improving the accuracy of target detection.

[0067] In some embodiments, non-target deep features are fused with higher-resolution deep features from a plurality of corresponding deep features to obtain deep fused features, including:

[0068] The non-target deep features at the lowest resolution are fused with the non-target deep features at the previous resolution to obtain the deep fused features at the previous resolution.

[0069] Starting from the previous resolution of the lowest resolution, the deep fusion features of each resolution are sequentially fused with the non-target deep features of the previous resolution of each resolution to determine the deep fusion features of the previous resolution of each resolution, thus obtaining multiple deep fusion features of different resolutions.

[0070] The resolution above the lowest resolution refers to the second lowest resolution.

[0071] In some embodiments, the computer device may upsample the lowest-resolution non-target deep features, and then superimpose the upsampled lowest-resolution non-target deep features with the previous-resolution non-target deep features to obtain a deep fusion feature of the previous-resolution. The upsampling process may include sequentially performing convolution, normalization, activation, and upsampling operations.

[0072] In some embodiments, the computer device may upsample the deep fusion features of each resolution sequentially, starting from the resolution above the lowest resolution, and then superimpose the upsampled deep fusion features of each resolution with the non-target deep features of the previous resolution to determine the deep fusion features of the previous resolution.

[0073] As can be seen, in this embodiment, by fusing the non-target deep features of the lowest resolution with the non-target deep features of the previous resolution, the deep fused features of the previous resolution are obtained; starting from the previous resolution, the deep fused features of each resolution are fused with the non-target deep features of the previous resolution in turn, so that the deep fused features of the previous resolution of each resolution with richer semantic information can be determined.

[0074] In some embodiments, the first target deep feature is the deep feature with the highest resolution among a plurality of deep features;

[0075] Further fusion of splicing features, deep fusion features, and second target deep features is performed to obtain advanced fusion features, including:

[0076] The highest resolution stitching feature is fused with the next highest resolution deep fusion feature to obtain the highest resolution stitching fusion feature.

[0077] The highest resolution stitching and fusion features, multiple deep fusion features of different resolutions, and the deep features of the second target are fused to obtain advanced fusion features.

[0078] In some embodiments, the resolution corresponding to the stitching feature is the highest resolution. Multiple deep fusion features at different resolutions are deep fusion features ranging from the previous resolution of the lowest resolution to the next resolution of the highest resolution. The resolution corresponding to the stitching feature is higher, and there is no overlap between the resolutions corresponding to the deep fusion features.

[0079] In some embodiments, the computer device may upsample the deep fusion feature of the next highest resolution, and then superimpose the stitched feature of the highest resolution with the upsampled deep fusion feature of the next highest resolution to obtain the stitched fusion feature of the highest resolution.

[0080] In some embodiments, the computer device can upsample the highest resolution stitching and fusion feature, multiple deep fusion features of different resolutions, and the second target deep feature, respectively, and then perform channel stitching on the upsampled stitching and fusion feature, the multiple upsampled deep fusion features, and the upsampled second target deep feature to obtain an advanced fusion feature.

[0081] As can be seen, in this embodiment, by fusing the highest resolution stitching feature with the next highest resolution deep fusion feature, the highest resolution stitching fusion feature is obtained; by fusing the highest resolution stitching fusion feature, multiple deep fusion features of different resolutions, and the second target deep feature, an advanced fusion feature including richer texture and semantic information can be obtained. When predicting slender targets based on the advanced fusion feature, it is less likely that the predicted target will break, thus improving the accuracy of target detection.

[0082] In some embodiments, the detection result is the output obtained by inputting the image to be detected into the target detection model; the training steps of the target detection model include:

[0083] Acquire the original image containing the target region;

[0084] The target region in the original image is cropped according to a first preset probability to obtain a first sample image;

[0085] The original image is randomly cropped according to the second preset probability to obtain the second sample image;

[0086] The target detection model to be trained is trained based on the first sample image and the second sample image to obtain the trained target detection model.

[0087] In some embodiments, the original image refers to an image with foreign object defects such as fuzz collected in an actual scene. The fuzzy areas in the original image are labeled using polyline segments, and a label mask including the polyline segment region of a preset width is drawn. The preset width is not less than the width of the fuzzy area; for example, if the fuzz is less than 4 pixels thick and located between 1 and 3 pixels, the preset width can be 4 pixels.

[0088] In some embodiments, the first preset probability is greater than the second preset probability. The computer device can crop the image using the target region in the original image as the center, with the first preset probability, to obtain a first sample image. Then, it can crop a random region of the original image using the second preset probability to obtain a second sample image. For example, the first preset probability can be 90%, and the second preset probability can be 10%.

[0089] In some implementations, the computer device can perform at least one of the following processing methods on the first and second sample images, such as spatial augmentation or color augmentation, according to a third preset probability, to obtain training sample images. Spatial augmentation may include at least one of random rotation or random flipping. Color augmentation may include at least one of random brightness, random contrast, random saturation, random gamma transformation, or random color temperature.

[0090] In some embodiments, the computer device may spatially augment the first sample image and the second sample image with a third preset probability, and then perform color augmentation to obtain the training sample image.

[0091] In some embodiments, the object detection model includes a backbone network and a prediction head. The object detection model may be, but is not limited to, a semantic segmentation model.

[0092] In some embodiments, a computer device may input training sample images into a target detection model to be trained, obtain the prediction results output by the target detection model, calculate the loss value based on the difference between the prediction results and the label mask, and train the target detection model by gradient backpropagation based on the loss value.

[0093] In some embodiments, the computer device can determine the focal loss and Dice loss based on the prediction results and the label mask; the focal loss and Dice loss are then weighted and summed to obtain the loss value. For example, the weighting ratio of the focal loss and Dice loss can be 20:1.

[0094] In some embodiments, the computer device may train the object detection model using an optimizer with an adaptive learning rate (Adam optimizer) and a cosine descent strategy to dynamically adjust the learning rate. The maximum learning rate may be set to 1e-3.

[0095] In some embodiments, such as Figure 3The diagram illustrates the structure of an object detection model. The prediction head receives the following inputs from the backbone network: shallow features O0, initial deep features O1 at a first resolution, initial deep features O2 at a second resolution, initial deep features O3 at a third resolution, and initial deep features O4 at a fourth resolution. O0 and O1 correspond to the same resolution, representing the highest resolution. The prediction head is used for object detection based on shallow features and multiple deep features at different resolutions.

[0096] The prediction head initially concatenates O0 and O1 to obtain initial concatenated features. These initial concatenated features are then processed through a 1x1 convolution + normalization + activation workflow to obtain 64-channel concatenated features. Normalization can be, but is not limited to, batch normalization. Activation can be based on Rectified Linear Unit (ReLU).

[0097] The prediction head performs 1*1 convolution + normalization + activation on O2, O3 and O4 respectively to obtain deep features at the second resolution, the third resolution and the fourth resolution with 64 channels.

[0098] The prediction head performs a 1*1 convolution + normalization + activation + 2x upsampling on the deep features of the fourth resolution, and then adds them point-to-point with the deep features of the third resolution to obtain the deep fused features of the third resolution.

[0099] The prediction head performs a 1*1 convolution + normalization + activation + 2x upsampling on the deep fusion features of the third resolution, and then adds them point-to-point with the deep features of the second resolution to obtain the deep fusion features of the second resolution.

[0100] The prediction head performs a 1*1 convolution + normalization + activation + 2x upsampling on the deep fusion features of the second resolution, and then adds them point-to-point to the stitched features of the first resolution to obtain the stitched fusion features of the first resolution.

[0101] The prediction head performs 1*1 convolution + normalization + activation and upsampling operations on the stitched and fused features at the first resolution, the deep fused features at the second resolution, the deep fused features at the third resolution, and the deep features at the fourth resolution, respectively, to obtain upsampled stitched and fused features, multiple upsampled deep fused features, and upsampled deep features at the fourth resolution. The upsampled stitched and fused features, multiple upsampled deep fused features, and the upsampled deep features at the fourth resolution are then concatenated channel by channel to obtain a 256-channel advanced fused feature.

[0102] The prediction head performs a 1x1 convolution and upsampling on the advanced fusion features to obtain a binarized image. This binarized image can be understood as a prediction mask used to indicate targets in the image to be detected.

[0103] As can be seen, in this embodiment, by cropping the target region in the original image according to the first preset probability, a first sample image is obtained; by randomly cropping the original image according to the second preset probability, a second sample image is obtained; and the target detection model to be trained is trained based on the first sample image and the second sample image to obtain the trained target detection model, which can ensure that the target detection model pays more attention to the target region and increases the detection capability.

[0104] In some embodiments, the detection result includes a binarized image, which is used to indicate a target in the image to be detected; the method further includes:

[0105] The binarized image is skeletonized to obtain the skeleton image;

[0106] Filter the skeleton image to determine any diagonal pixel on the skeleton of the target.

[0107] The length of the target in the real scene is determined based on the lengths of the diagonal and non-diagonal pixels on the target's skeleton.

[0108] In some embodiments, the image to be detected may be a workpiece image. A computer device can acquire a high-exposure image of the workpiece. A target detection model is used to perform target detection on the workpiece image, outputting a binarized image. In the binarized image, the region containing the fuzz on the workpiece has a value of 1, while the background region without fuzz has a value of 0.

[0109] In some embodiments, a computer device can filter a skeleton image using a filter comprising two convolutional kernels to obtain two filtered results. The two convolutional kernels are used to filter out two different diagonal pixels. Each filtered result is a binarized image of the same size as the skeleton image, used to indicate the corresponding diagonal pixels. The two convolutional kernels are [1 0; 0 1] and [0 1; 1 0].

[0110] In some embodiments, the computer device can determine the length of the target in a real scene based on the number of diagonal pixels and the length corresponding to the diagonal pixels, as well as the number of non-diagonal pixels and the length corresponding to the non-diagonal pixels.

[0111] As can be seen, in this embodiment, the target detection method provided by this application has a strong detection capability for small and slender targets, and the obtained binarized image is more accurate. By performing skeletonization processing on the binarized image, a skeleton image is obtained; the skeleton image is filtered to determine any diagonal pixel on the target's skeleton; based on the lengths corresponding to the diagonal and non-diagonal pixels on the target's skeleton, the length of the target in the real scene can be accurately determined.

[0112] In some embodiments, the length of the target in the real scene is determined based on the lengths corresponding to the diagonal and non-diagonal pixels on the target's skeleton, including:

[0113] Determine the weights corresponding to the diagonal pixels on the target's skeleton; the weights are determined based on the difference between the lengths corresponding to the diagonal pixels and the lengths corresponding to the non-diagonal pixels.

[0114] The number of diagonal pixels is weighted according to the weights to obtain the weighted result;

[0115] The total number of pixels on the target's skeleton and the weighted result are superimposed to obtain the superimposed result;

[0116] The lengths of non-oblique pixels are weighted based on the superposition results to obtain the length of the target in the real scene.

[0117] In some embodiments, a pixel is considered as a square, and the length of a diagonal pixel is √2 times the length of a non-diagonal pixel. Therefore, the weight of a diagonal pixel is √2-1.

[0118] In some embodiments, the computer device can use weights to weight the number of diagonal pixels to obtain a weighted result. The total number of pixels on the target skeleton, the weighted result, and the bias parameter are summed to obtain the superposition result. The superposition result is used to characterize the pixel length of the target skeleton. Formula (1) is the formula for calculating the superposition result.

[0119] K i =Sum(S i )+(√2-1)(Sum(F1(S i ))+Sum(F2(S i )))+b(1)

[0120] Among them, K i The superposition result representing the skeleton of the i-th target. Sum(S i F1(S) represents the total number of pixels on the skeleton of the i-th target. i F2(S) represents the first filtering result for the skeleton of the i-th target. iThe expression ) represents the second filtering result for the skeleton of the i-th target. b represents the bias parameter. It's understandable that, due to the varying thickness of hair fibers, the skeleton length is insufficient to accurately describe the actual pixel length of the hair fibers. Therefore, a bias parameter b greater than 0 is typically needed to accurately describe the pixel length of the hair fibers. b is generally set to 2 pixels.

[0121] In some embodiments, a computer device can use the superposition result to weight the lengths corresponding to non-oblique pixels to obtain the length of the target in the real scene. For example, 1 pixel = a millimeters, and for the i-th target, its estimated length is L. i =a*K i .

[0122] As can be seen, in this embodiment, the weight corresponding to the diagonal pixel is determined; the weight is determined based on the difference between the length corresponding to the diagonal pixel and the length corresponding to the non-diagonal pixel; the number of diagonal pixels is weighted according to the weight to obtain a weighted result; the total number of pixels on the target skeleton and the weighted result are superimposed to obtain a superimposed result; the length corresponding to the non-diagonal pixel is weighted according to the superimposed result to obtain the length of the target in the real scene. This fully considers the difference between diagonal pixels and pixels in the horizontal or vertical direction, and can accurately measure the length of the target.

[0123] In some embodiments, skeletonization processing is performed on the binarized image to obtain a skeleton image, including:

[0124] Connectivity analysis is performed on the binarized image to obtain the connected components;

[0125] Binary sub-images are segmented from the binary image based on the circumscribed contours of connected components;

[0126] Skeletonization is performed on the binary sub-image to obtain the skeleton image.

[0127] In some embodiments, each connected component represents a hair instance at a corresponding location in the image to be detected.

[0128] In some embodiments, a computer device may slice a binarized image using the bounding rectangle of each connected component to obtain a binary sub-image. Each binary sub-image includes a connected component.

[0129] As can be seen, in this embodiment, connected component analysis is performed on the binarized image to obtain connected components; binary sub-images are segmented from the binarized image based on the circumscribed contours of the connected components; and skeletonization processing is performed on the binary sub-images to obtain skeleton images. This allows for targeted skeletonization of each connected component, which saves computational resources compared to skeletonizing the entire binarized image.

[0130] In some embodiments, the computer device can perform convolution calculations on the skeleton image to obtain the skeleton length. The skeleton length is then used as a measurement of the hair length.

[0131] In some embodiments, the computer device can determine the perimeter of the skeleton from the skeleton image, and use half of the perimeter as the pixel length of the hair. The measured value of the hair length is determined based on the pixel length of the hair and the length corresponding to the pixel.

[0132] In some embodiments, a computer device can determine the skeleton width of a hair filament from a skeleton image using an orthogonal skeleton method. The measured value of the hair filament width is determined based on the skeleton width of the hair filament and the length corresponding to the pixel.

[0133] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0134] Based on the same inventive concept, this application also provides a target detection device. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more target detection device embodiments provided below can be found in the limitations of the target detection method above, and will not be repeated here.

[0135] like Figure 4 As shown, this application embodiment provides a target detection device 400, including:

[0136] The determination module 402 is used to determine the shallow features and multiple deep features of different resolutions of the image to be detected.

[0137] The stitching module 404 is used to stitch shallow features and deep features of the first target to obtain stitched features; the deep features of the first target are deep features among multiple deep features whose resolution matches that of the shallow features;

[0138] The first fusion module 406 is used to fuse non-target deep features with higher resolution deep features among multiple corresponding deep features to obtain deep fused features; non-target deep features refer to deep features other than the first target deep feature among multiple deep features.

[0139] The second fusion module 408 is used to perform advanced fusion of the splicing features, deep fusion features, and second target deep features to obtain advanced fusion features; the second target deep features refer to the non-target deep features with the lowest resolution.

[0140] The detection module 410 is used to perform target detection on the image to be detected based on advanced fusion features and obtain the detection result.

[0141] In some embodiments, in fusing non-target deep features with higher-resolution deep features among a plurality of corresponding deep features to obtain deep fused features, the first fusion module 406 is specifically used for:

[0142] The non-target deep features at the lowest resolution are fused with the non-target deep features at the previous resolution to obtain the deep fused features at the previous resolution.

[0143] Starting from the previous resolution of the lowest resolution, the deep fusion features of each resolution are sequentially fused with the non-target deep features of the previous resolution of each resolution to determine the deep fusion features of the previous resolution of each resolution, thus obtaining multiple deep fusion features of different resolutions.

[0144] In some embodiments, the first target deep feature is the deep feature with the highest resolution among a plurality of deep features;

[0145] In terms of further fusing splicing features, deep fusion features, and second target deep features to obtain advanced fused features, the second fusion module 408 is specifically used for:

[0146] The highest resolution stitching feature is fused with the next highest resolution deep fusion feature to obtain the highest resolution stitching fusion feature.

[0147] The highest resolution stitching and fusion features, multiple deep fusion features of different resolutions, and the deep features of the second target are fused to obtain advanced fusion features.

[0148] In some embodiments, the detection result is the output obtained by inputting the image to be detected into the target detection model; the detection module 410 is further configured to:

[0149] Acquire the original image containing the target region;

[0150] The target region in the original image is cropped according to a first preset probability to obtain a first sample image;

[0151] The original image is randomly cropped according to the second preset probability to obtain the second sample image;

[0152] The target detection model to be trained is trained based on the first sample image and the second sample image to obtain the trained target detection model.

[0153] In some embodiments, the detection result includes a binarized image, which is used to indicate a target in the image to be detected; the detection module 410 is further configured to:

[0154] The binarized image is skeletonized to obtain the skeleton image;

[0155] Filter the skeleton image to determine any diagonal pixel on the skeleton of the target.

[0156] The length of the target in the real scene is determined based on the lengths of the diagonal and non-diagonal pixels on the target's skeleton.

[0157] In some embodiments, the detection module 410 is specifically used to determine the length of the target in the real scene based on the lengths corresponding to the diagonal and non-diagonal pixels on the target's skeleton, respectively:

[0158] Determine the weights corresponding to the diagonal pixels on the target's skeleton; the weights are determined based on the difference between the lengths corresponding to the diagonal pixels and the lengths corresponding to the non-diagonal pixels.

[0159] The number of diagonal pixels is weighted according to the weights to obtain the weighted result;

[0160] The total number of pixels on the target's skeleton and the weighted result are superimposed to obtain the superimposed result;

[0161] The lengths of non-oblique pixels are weighted based on the superposition results to obtain the length of the target in the real scene.

[0162] In some embodiments, in performing skeletonization processing on a binary image to obtain a skeleton image, the detection module 410 is specifically used for:

[0163] Connectivity analysis is performed on the binarized image to obtain the connected components;

[0164] Binary sub-images are segmented from the binary image based on the circumscribed contours of connected components;

[0165] Skeletonization is performed on the binary sub-image to obtain the skeleton image.

[0166] Each module in the aforementioned target detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0167] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores the image to be detected. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the target detection method described above.

[0168] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the steps in the target detection method described above. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen; the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs or touchpads set on the casing of the computer device, or external keyboards, touchpads or mice, etc.

[0169] Those skilled in the art will understand that Figure 5 or Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0170] In some embodiments, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0171] In some embodiments, such as Figure 7 The diagram shows the internal structure of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the above-described method embodiments.

[0172] In some embodiments, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target detection method, characterized in that, include: Determine the shallow features and multiple deep features at different resolutions of the image to be detected; The image to be detected contains a target with a slender shape; The shallow features are spliced ​​together with the deep features of the first target to obtain spliced ​​features; the deep features of the first target are the deep features among the plurality of deep features whose resolution matches that of the shallow features; The non-target deep features are fused with the higher-resolution deep features among the corresponding plurality of deep features to obtain deep fused features; the non-target deep features refer to the deep features other than the first target deep feature among the plurality of deep features; The highest resolution stitching feature is fused with the next highest resolution deep fusion feature to obtain the highest resolution stitching fusion feature. The highest resolution stitched fusion feature, multiple deep fusion features of different resolutions, and the second target deep feature are upsampled respectively. The upsampled stitched fusion feature, multiple upsampled deep fusion features, and the upsampled second target deep feature are then channel-stitched to obtain the advanced fusion feature. The second target deep feature refers to the lowest resolution non-target deep feature. The target detection is performed on the image to be detected based on the advanced fusion features, and the detection result is obtained.

2. The method according to claim 1, characterized in that, The process of fusing non-target deep features with higher-resolution deep features from among the corresponding plurality of deep features to obtain deep fused features includes: The non-target deep features of the lowest resolution are fused with the non-target deep features of the previous resolution of the lowest resolution to obtain the deep fused features of the previous resolution of the lowest resolution. Starting from the previous resolution of the lowest resolution, the deep fusion features of each resolution are sequentially fused with the non-target deep features of the previous resolution of each resolution to determine the deep fusion features of the previous resolution of each resolution, thereby obtaining multiple deep fusion features of different resolutions.

3. The method according to claim 1, characterized in that, The first target deep feature is the deep feature with the highest resolution among the plurality of deep features.

4. The method according to claim 1, characterized in that, The detection result is the output obtained by inputting the image to be detected into the target detection model; The training steps of the object detection model include: Acquire the original image containing the target region; The target region in the original image is cropped according to a first preset probability to obtain a first sample image; The original image is randomly cropped according to a second preset probability to obtain a second sample image; The target detection model to be trained is trained based on the first sample image and the second sample image to obtain the trained target detection model.

5. The method according to any one of claims 1 to 4, characterized in that, The detection result includes a binarized image, which is used to indicate the target in the image to be detected; the method further includes: The binarized image is then subjected to skeletonization processing to obtain a skeleton image; The skeleton image is filtered to determine any diagonal pixel on the skeleton of the target. The length of the target in the real scene is determined based on the lengths of the diagonal and non-diagonal pixels on the target's skeleton.

6. The method according to claim 5, characterized in that, Determining the length of the target in the real scene based on the lengths corresponding to the diagonal and non-diagonal pixels on the target's skeleton includes: Determine the weights corresponding to the diagonal pixels on the skeleton of the target; the weights are determined based on the difference between the lengths corresponding to the diagonal pixels and the lengths corresponding to the non-diagonal pixels. The number of diagonal pixels is weighted according to the weights to obtain a weighted result; The total number of pixels on the skeleton of the target and the weighted result are superimposed to obtain the superimposed result; The lengths of the non-oblique pixels are weighted according to the superposition result to obtain the length of the target in the real scene.

7. The method according to claim 5, characterized in that, The process of skeletonizing the binarized image to obtain a skeleton image includes: Connectivity analysis is performed on the binarized image to obtain the connected components; Binary sub-images are segmented from the binarized image based on the circumscribed contour of the connected components; The binary subgraph is skeletonized to obtain a skeleton image.

8. A target detection device, characterized in that, include: The determination module is used to determine the shallow features and multiple deep features at different resolutions of the image to be detected. The image to be detected contains a target with a slender shape; The stitching module is used to stitch the shallow features and the first target deep features to obtain stitched features; the first target deep features are the deep features among the plurality of deep features whose resolution matches that of the shallow features; The first fusion module is used to fuse non-target deep features with higher-resolution deep features among the corresponding plurality of deep features to obtain deep fused features; the non-target deep features refer to deep features other than the first target deep feature among the plurality of deep features; The second fusion module is used to fuse the highest resolution stitching feature with the next lower resolution deep fusion feature to obtain the highest resolution stitching fusion feature; upsampling is performed on the highest resolution stitching fusion feature, multiple deep fusion features of different resolutions, and the second target deep feature; the upsampled stitching fusion feature, multiple upsampled deep fusion features, and the upsampled second target deep feature are then channel-stitched to obtain an advanced fusion feature; the second target deep feature refers to the lowest resolution non-target deep feature. The detection module is used to perform target detection on the image to be detected based on the advanced fusion features, and obtain the detection result.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images