A method and system for identifying the depth of defects in images of second-hand goods

Abstract

This application provides a method and system for deep defect recognition of second-hand goods images. It determines a multi-level feature pyramid of the second-hand goods image; by combining multiple dilated convolution branches with channel attention adaptive weights, it extracts feature representations of defects at different scales from the feature maps of each level in the multi-level feature pyramid, and then fuses them to generate a fused feature map; based on the semantic difference features of defects in the second-hand goods image, it determines a saliency prior map for image defect recognition, and then performs cross-spatial attention modulation on the fused feature map to obtain a defect feature map that suppresses the texture of the goods itself and enhances the response weight of the defect region; it classifies and locates defects in the defect feature map, and then integrates the category confidence and localization bounding boxes of each defect in the second-hand goods image to generate the defect recognition result. This application can achieve effective suppression of complex background texture interference, adaptive capture of multi-scale defect features, and robust defect deep recognition under small sample conditions.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and more specifically, to a method and system for deep identification of defects in images of second-hand goods. Background Technology

[0002] With the rapid growth of online transactions for secondhand goods, the rapid, objective, and accurate automated identification and evaluation of product appearance defects has become a key technological link in improving transaction efficiency and protecting the rights and interests of both buyers and sellers. Traditional inspection methods that rely on manual visual inspection are inefficient, subjective, and costly, making it difficult to meet the needs of large-scale platform applications. Therefore, utilizing computer vision, especially deep learning technology, to achieve intelligent identification and location of various defects such as scratches, stains, and dents in secondhand product images has significant industrial application value.

[0003] Currently, deep learning-based defect detection methods have been widely used in industrial quality inspection. Mainstream techniques typically employ deep convolutional neural networks (such as Faster R-CNN and YOLO series) for feature extraction and object detection, and improve performance through data augmentation and attention mechanisms (such as CBAM). However, these methods face significant bottlenecks when directly applied to complex and varied secondhand goods scenarios: First, secondhand goods images often have cluttered backgrounds and uneven lighting, making it easy to confuse the inherent rich textures of the goods with real defects. Second, defects vary greatly in shape and size, ranging from minor scratches to large dents, and existing fixed-structure feature extraction networks are insufficiently adaptable to multi-scale features. Third, labeled data for specific product categories (such as mobile phones and bags) is scarce, limiting the model's generalization ability. Therefore, how to effectively suppress interference from complex background textures, adaptively capture multi-scale defect features, and robustly operate under small sample conditions for deep defect recognition has become a major challenge for the industry. Summary of the Invention

[0004] This application provides a method and system for deep identification of defects in images of second-hand goods, which can effectively suppress interference from complex background textures, adaptively capture multi-scale defect features, and robustly operate under small sample conditions for defect depth identification.

[0005] Firstly, this application provides a method for deep identification of defects in images of second-hand goods, including: Hierarchical feature encoding is performed on the input second-hand goods image to obtain a multi-level feature pyramid of the second-hand goods image; By combining multiple dilated convolution branches with channel attention adaptive weights, feature representations of defects at different scales are extracted from the feature maps of each level in the multi-level feature pyramid, and then all feature representations are fused to generate a fused feature map of defects in second-hand goods. Based on the semantic difference features of defects in second-hand product images, a saliency prior map for image defect recognition is determined. Then, the fusion feature map is subjected to cross-spatial attention modulation with the saliency prior map as a condition to obtain a defect feature map that suppresses the texture of the product body and enhances the response weight of the defect area. The defect feature map is classified and located, and then the category confidence and location bounding box of each defect in the second-hand goods image are integrated to generate defect recognition results.

[0006] In some embodiments, hierarchical feature encoding is performed on the input secondhand goods image to obtain a multi-level feature pyramid of the secondhand goods image, specifically including: The input second-hand goods image is forward-propagated using a pre-trained lightweight convolutional neural network to extract multiple initial feature maps from shallow to deep. All initial feature maps are input into the feature pyramid network for top-down and lateral connection feature fusion to generate multiple enhanced feature maps; A multi-level feature pyramid is constructed based on all the enhanced feature maps to represent second-hand goods.

[0007] In some embodiments, by combining multiple dilated convolutional branches with channel attention adaptive weights, feature representations of defects at different scales are extracted from the feature maps of each level in the multi-level feature pyramid, and then all feature representations are fused to generate a fused feature map of defects in second-hand goods. Specifically, this includes: Each enhanced feature map in the multi-level feature pyramid is input into an independent parallel dilated convolution branch group for convolution operation. Each dilated convolution branch group contains convolution kernels with different dilation rates, resulting in defect feature representations at different levels and with different receptive fields. Global average pooling is performed on the feature representations of all levels to generate hierarchical channel attention weight vectors; The feature representations of the corresponding levels are weighted and fused using the channel attention weight vectors of the hierarchical levels, and then the cross-level fused features are spliced ​​and dimensionality reduced to generate a fused feature map of defects in second-hand goods.

[0008] In some embodiments, determining the saliency prior map for image defect identification based on the semantic difference features of defects in second-hand goods images specifically includes: A pre-trained lightweight defect semantic segmentation network is used to perform forward reasoning on second-hand product images to obtain binary semantic mask maps of the product body region and potential defect regions. The binary semantic mask image is subjected to morphological dilation and erosion operations to eliminate isolated noise points at the mask edges, resulting in a preliminary semantic saliency map. The preliminary semantic saliency map is subjected to contrast stretching and Gaussian filtering to obtain a saliency prior map of image defects.

[0009] In some embodiments, the fusion feature map is subjected to cross-spatial attention modulation conditioned on the saliency prior map to obtain a defect feature map that suppresses product body texture and enhances the response weight of defect regions. Specifically, this includes: The saliency prior map is projected into a conditional feature map that matches the shape of the fused feature map; The fused feature map is projected into a query vector, and the conditional feature map is projected into a key vector and a value vector, respectively. Then, the dot product similarity between the query vector and the key vector is calculated to generate a spatial attention weight matrix. The value vector is weighted and summed based on the spatial attention weight matrix, and then the summation result is residually connected with the fused feature map to obtain a defect feature map that suppresses the texture of the product body and enhances the response weight of the defect area.

[0010] In some embodiments, classifying and locating defects in the defect feature map, and then integrating the category confidence and location bounding boxes of each defect in the second-hand goods image to generate defect recognition results specifically includes: The defect feature map is classified and located to obtain the category confidence and location bounding box of each defect in the second-hand goods image; By weighted merging of overlapping bounding boxes of each defect, multiple merged bounding boxes are obtained. The final defect identification result for the second-hand goods image is determined based on all merged bounding boxes and corresponding category confidence scores.

[0011] In some embodiments, classifying and locating defects in the defect feature map to obtain the category confidence score and location bounding box of each defect in the second-hand goods image specifically includes: At each spatial location of the defect feature map, predefine multi-scale and multi-proportion anchor boxes, and determine the initial confidence level of the presence of defects within each anchor box and the corresponding bounding box offset. Based on the initial confidence level, multiple candidate defect areas were selected from all anchor frames; The coordinates of each candidate defect region are refined by using all bounding box offsets, and then the local feature blocks of each refined candidate defect region are extracted. Based on all local feature blocks, perform defect category logistic regression and bounding box coordinate regression respectively, and output the category confidence and local bounding box of each defect in the second-hand goods image.

[0012] Secondly, this application provides a system for deep recognition of image defects in second-hand goods, comprising: The encoding module is used to perform hierarchical feature encoding on the input second-hand goods image to obtain a multi-level feature pyramid of the second-hand goods image; The processing module is used to extract feature representations of defects at different scales from the feature maps of each level in the multi-level feature pyramid by combining multiple dilated convolution branches with channel attention adaptive weights, and then fuse all feature representations to generate a fused feature map of defects in second-hand goods. The processing module is used to determine a saliency prior map for image defect recognition based on the semantic difference features of defects in second-hand commodity images, and then perform cross-spatial attention modulation on the fused feature map using the saliency prior map as a condition to obtain a defect feature map that suppresses the texture of the commodity body and enhances the response weight of the defect region. The execution module is used to classify and locate defects in the defect feature map, and then integrate the category confidence and location bounding box of each defect in the second-hand goods image to generate defect recognition results.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described method for deep identification of defects in second-hand goods images.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for deep identification of defects in second-hand goods images.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The method and system for deep recognition of defects in second-hand commodity images provided in this application first performs hierarchical feature encoding on the input second-hand commodity image to obtain a multi-level feature pyramid of the second-hand commodity image; By combining multiple dilated convolution branches with channel attention adaptive weights, feature representations of defects at different scales are extracted from the feature maps of each level in the multi-level feature pyramid, and then all feature representations are fused to generate a fused feature map of defects in second-hand goods. Based on the semantic difference features of defects in second-hand product images, a saliency prior map for image defect recognition is determined. Then, the fusion feature map is subjected to cross-spatial attention modulation with the saliency prior map as a condition to obtain a defect feature map that suppresses the texture of the product body and enhances the response weight of the defect area. The defect feature map is classified and located, and then the category confidence and location bounding box of each defect in the second-hand goods image are integrated to generate defect recognition results.

[0016] Therefore, this application integrates the category confidence and localization bounding box of each defect to obtain the final defect recognition result of the second-hand goods image. First, the fusion feature map is determined to obtain a unified feature map that integrates complementary contextual information in the multi-scale feature representation. The determination of the fusion feature map can be achieved by introducing a hierarchical dilated convolution branch + channel attention adaptive weight structure to dynamically fuse the multi-scale features extracted by the feature pyramid. This process is not a simple concatenation or fixed weight summation of multi-scale features, but rather a lightweight attention subnetwork that learns and assigns fusion weights of each scale feature in real time according to the input image content in a hierarchical manner. This makes the generated fusion feature map more robust. The combined feature map adaptively strengthens the scale context information most relevant to the current defect while suppressing the response of irrelevant background features. This provides a high-quality feature base for the subsequent attention modulation module, combining multi-scale information completeness and focus, fundamentally improving the model's ability to capture diverse defects and distinguish complex backgrounds. Then, determining the defect feature map yields the final optimized feature map, which fuses contextual information with prior guidance. The determination of the defect feature map fundamentally addresses the core challenge pointed out in background techniques: the complex background texture and variable defect scale in secondhand product images. This defect feature map utilizes high-level semantic priors derived from semantic segmentation. The saliency prior map serves as a guiding signal, enabling the attention mechanism to dynamically and discriminatively enhance the response of potential defect areas and effectively suppress interference caused by the inherent texture of the product (such as leather texture and fabric pattern) in the feature space, thereby significantly reducing the false detection rate caused by complex backgrounds. Simultaneously, since the fused feature map it is based on integrates adaptive multi-scale features, the defect feature map itself possesses rich contextual information representing defects at all scales, from minor scratches to large dents. This effectively alleviates the missed detection problem caused by the insufficient perception ability of traditional methods for small target defects, and is the key technology for this scheme to achieve the dual optimization goals of suppressing interference and enhancing the target. The new feature provides a high-quality feature representation with strong discriminative power and low noise for subsequent classification and localization tasks, which is the foundation for achieving accurate and reliable recognition results. Finally, by determining the category confidence and localization bounding box, we can obtain the probability distribution vector of the image region belonging to each category of defect and the coordinates of the rectangular box that accurately defines the location and range of the defect in the image space. The category confidence quantifies the credibility of defect classification in probabilistic form, providing a reliable decision threshold for distinguishing real defects from background noise in complex texture backgrounds, effectively reducing the false detection rate. The localization bounding box provides the precise spatial coordinates and range of the defect, solving the problem of inaccurate localization of small targets, especially in multi-scale defects.The combination of these two approaches not only provides interpretable and measurable structured information for the model output, supporting subsequent integrated decisions to improve robustness, but also directly provides indispensable quantitative evidence with both semantic and spatial dimensions for downstream tasks such as automated quality grading and residual value assessment of second-hand goods. This effectively bridges the key technology chain from image recognition to business decision-making. In summary, based on the above scheme, it is possible to effectively suppress interference from complex background textures, adaptively capture multi-scale defect features, and robustly perform defect depth recognition under small sample conditions. Attached Figure Description

[0017] Figure 1 This is an exemplary flowchart of a method for identifying the depth of defects in images of second-hand goods according to some embodiments of this application; Figure 2 This is an exemplary flowchart illustrating the determination of defect identification results according to some embodiments of this application; Figure 3 This is a schematic diagram of the structure of a second-hand goods image defect depth recognition system according to some embodiments of this application; Figure 4 This is an internal structural diagram of a computer device that implements a method for deep identification of image defects of second-hand goods according to some embodiments of this application. Detailed Implementation

[0018] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] refer to Figure 1 The figure is an exemplary flowchart of a method for deep identification of defects in second-hand goods images according to some embodiments of this application. The method for deep identification of defects in second-hand goods images mainly includes the following steps: In step 101, the input second-hand goods image is subjected to hierarchical feature encoding to obtain a multi-level feature pyramid of the second-hand goods image.

[0020] In some embodiments, hierarchical feature encoding of the input secondhand goods image to obtain a multi-level feature pyramid of the secondhand goods image can be achieved by the following steps: The input second-hand goods image is forward-propagated using a pre-trained lightweight convolutional neural network to extract multiple initial feature maps from shallow to deep. All initial feature maps are input into the feature pyramid network for top-down and lateral connection feature fusion to generate multiple enhanced feature maps; A multi-level feature pyramid is constructed based on all the enhanced feature maps to represent second-hand goods.

[0021] In specific implementation, the input secondhand goods image is forward-propagated using a pre-trained lightweight convolutional neural network to extract multiple initial feature maps from shallow to deep. This can be achieved as follows: First, the input secondhand goods image is forward-propagated using a lightweight convolutional neural network pre-trained on a large general image dataset. This lightweight convolutional neural network consists of inverse residual modules based on depthwise separable convolution. It extracts initial feature maps from the secondhand goods image with decreasing resolution but increasing semantic information through multiple hierarchical downsampling layers, resulting in multiple initial feature maps from shallow to deep. The initial feature maps represent hierarchical original features in the secondhand goods image, from detailed texture to abstract semantics. They can provide a complete basic feature source covering different receptive fields and semantic levels for subsequent high-level feature fusion.

[0022] In practice, all initial feature maps are input into the feature pyramid network for top-down and lateral feature fusion to generate multiple enhanced feature maps. This can be achieved by inputting all initial feature maps into the feature pyramid network for high-level semantic and detail information fusion. Specifically, the deepest and most semantically strongest initial feature map is first upsampled to increase its resolution to match that of adjacent shallower feature maps. Figure 1 Then, through lateral connections, the upsampled deep features are added and fused element-by-element with the corresponding shallow feature maps, thereby injecting rich semantic information from the deep layers into shallow features containing more details. Upsampling and lateral connection fusion are performed layer by layer to finally generate a new set of enhanced feature maps that fuse strong semantics and high detail information at multiple scales. The enhanced feature maps are optimized features with enhanced representation capabilities generated after cross-level semantic-detail fusion. They can achieve complementarity and enhancement between high-level semantic information and low-level detail features, significantly improving the ability to capture and distinguish flawed features at different scales.

[0023] In practice, the construction of a multi-level feature pyramid for the second-hand goods image based on all the enhanced feature maps can be achieved in the following way: the feature set obtained by organizing all the enhanced feature maps in descending order of resolution is used as the multi-level feature pyramid for the second-hand goods image.

[0024] It should be noted that in this application, the multi-level feature pyramid is a structured feature that integrates all initial feature maps and enhanced feature maps and has continuous spatial resolution and rich semantic hierarchy. It can provide a unified and complete multi-scale feature representation basis for parallel processing of various defects with a wide size range, from minor scratches to large dents.

[0025] In step 102, multiple dilated convolution branches combined with channel attention adaptive weights are used to extract feature representations of defects at different scales from the feature maps of each level in the multi-level feature pyramid, and then all feature representations are fused to generate a fused feature map of defects in second-hand goods.

[0026] In some embodiments, by combining multiple dilated convolutional branches with channel attention adaptive weights, feature representations of defects at different scales are extracted from the feature maps of each level in the multi-level feature pyramid, and then all feature representations are fused to generate a fused feature map of defects in second-hand goods. This can be achieved through the following steps: Each enhanced feature map in the multi-level feature pyramid is input into an independent parallel dilated convolution branch group for convolution operation. Each dilated convolution branch group contains convolution kernels with different dilation rates, resulting in defect feature representations at different levels and with different receptive fields. Global average pooling is performed on the feature representations of all levels to generate hierarchical channel attention weight vectors; The feature representations of the corresponding levels are weighted and fused using the channel attention weight vectors of the hierarchical levels, and then the cross-level fused features are spliced ​​and dimensionality reduced to generate a fused feature map of defects in second-hand goods.

[0027] In specific implementation, the enhanced feature maps of each level in the multi-level feature pyramid are input into independent parallel dilated convolution branches for convolution operations. The defect feature representations at different levels and with different receptive fields can be achieved in the following way: First, the enhanced feature maps of each level in the multi-level feature pyramid are input into a set of dedicated parallel dilated convolution branches. The dilation rate of each branch is preset to different values, such as 1, 3, 6, and 9. This allows each dilated convolution branch to capture feature information of different receptive field ranges on the feature map of that level without increasing the number of parameters. Finally, a set of feature representations focusing on different scale contexts are output. The feature representations are multi-scale features with differentiated receptive fields focusing on different scale contexts. They can capture contextual information of defects of different physical sizes, from minor scratches to large dents, providing a rich multi-scale feature foundation for adaptive fusion.

[0028] In specific implementation, global average pooling is performed on the feature representations of all levels to generate hierarchical channel attention weight vectors. This can be achieved as follows: global average pooling is performed on the feature representations generated by each dilated convolution branch under each level, that is, each feature representation is compressed into a scalar vector with global representativeness in the spatial dimension. Then, all the scalar vectors obtained after pooling under the same level are sequentially input into a shared, lightweight multilayer perceptron for nonlinear transformation and information interaction. The output is an intermediate vector whose dimension corresponds to the number of branches at that level. This intermediate vector is then normalized by an activation function (such as the sigmoid activation function) to map it to the interval between 0 and 1, thereby generating hierarchical channel attention weight vectors that represent the relative importance of features of each branch under that level. The hierarchical channel attention weight vector is a weight distribution vector that quantifies the relative importance of feature representations of each branch within the same level. It enables the system to adaptively evaluate and highlight key scale features and suppress secondary or interfering features based on the content of the current level feature map, thus achieving intelligent feature selection.

[0029] In specific implementation, the feature representations of the corresponding levels are weighted and fused using the hierarchical channel attention weight vectors. Then, the fused features across levels are concatenated and dimensionality reduced to generate a fused feature map of defects in second-hand goods. This can be achieved in the following way: multiply the feature representations within each level with the elements in the corresponding level's channel attention weight vector channel by channel to complete adaptive weighting. This weighting process amplifies the scale features that are more important to the current level while suppressing secondary or interfering feature responses. Then, the weighted feature representations of all levels are concatenated along the channel dimension to obtain a high-dimensional concatenated feature. Standard convolution operation is then performed on this high-dimensional concatenated feature to achieve the fusion of cross-level features and channel dimensionality reduction, resulting in a feature map with a uniform number of channels and highly integrated information, which serves as a fused feature map that integrates multi-scale contextual information.

[0030] It should be noted that in this application, the fused feature map is a unified feature map that integrates complementary contextual information in multi-scale feature representation. It can integrate the feature advantages under different receptive fields to form an enhanced feature that can simultaneously represent multiple scale defects and highlight the key points, providing a robust feature representation for subsequent high-precision recognition.

[0031] In step 103, a saliency prior map for image defect recognition is determined based on the semantic difference features of defects in the second-hand product image. Then, the fusion feature map is subjected to cross-spatial attention modulation based on the saliency prior map to obtain a defect feature map that suppresses the texture of the product body and enhances the response weight of the defect area.

[0032] In some embodiments, determining the saliency prior map for image defect identification based on the semantic difference features of defects in second-hand goods images can be achieved through the following steps: A pre-trained lightweight defect semantic segmentation network is used to perform forward reasoning on second-hand product images to obtain binary semantic mask maps of the product body region and potential defect regions. The binary semantic mask image is subjected to morphological dilation and erosion operations to eliminate isolated noise points at the mask edges, resulting in a preliminary semantic saliency map. The preliminary semantic saliency map is subjected to contrast stretching and Gaussian filtering to obtain a saliency prior map of image defects.

[0033] In specific implementation, the binary semantic mask map of the product body region and the potential defect region can be obtained by forward reasoning on the second-hand product image through a pre-trained lightweight defect semantic segmentation network. This can be achieved in the following way: a lightweight semantic segmentation model (such as MobileNetV2+UNet) pre-trained on the second-hand product defect dataset is used to reason on the input image. The model output channel is divided into two categories: "product body" and "potential defect". The binary semantic mask map is generated through Softmax activation and thresholding. In the mask, the area with a pixel value of 1 represents the potential defect region, and the area with a pixel value of 0 represents the product body region. The binary semantic mask map is the basic feature map that reflects the semantic division between the target and the background in the image. It can accurately distinguish between defects and product body from a semantic perspective, providing the core basis for generating a reliable saliency prior map.

[0034] In specific implementation, the binary semantic mask image is subjected to morphological dilation and erosion operations to eliminate isolated noise points at the mask edges. The preliminary semantic saliency map can be obtained in the following way: first, morphological erosion is performed on the binary semantic mask image to eliminate small areas of isolated noise points, and then morphological dilation is performed to restore the complete outline of the defective region, resulting in a preliminary semantic saliency map with smooth edges and complete regions. The preliminary semantic saliency map is an optimized mask image after noise interference is eliminated, which can avoid the interference of isolated noise on subsequent attention modulation and improve the reliability of the prior map.

[0035] In specific implementation, the preliminary semantic saliency map is contrast stretched and Gaussian filtered to obtain the saliency prior map of image defects. This can be achieved by transforming the preliminary semantic saliency map with a nonlinear contrast stretching function (e.g., gamma correction or histogram stretching) to increase the difference between the defect area and the body area. Then, a Gaussian low-pass filter is applied to the stretched image to further smooth the mask edges, thus obtaining the final saliency prior map of image defects.

[0036] It should be noted that in this application, the saliency prior map is a semantic-level preprocessed feature map used to guide network attention. It can accurately amplify the response of the real defect area while suppressing the texture background of the product itself, providing a high signal-to-noise ratio and strong directional spatial guidance signal for the subsequent attention modulation module.

[0037] In some embodiments, the fusion feature map is subjected to cross-spatial attention modulation conditioned on the saliency prior map to obtain a defect feature map that suppresses product body texture and enhances the response weight of defect regions. This can be achieved by the following steps: The saliency prior map is projected into a conditional feature map that matches the shape of the fused feature map; The fused feature map is projected into a query vector, and the conditional feature map is projected into a key vector and a value vector, respectively. Then, the dot product similarity between the query vector and the key vector is calculated to generate a spatial attention weight matrix. The value vector is weighted and summed based on the spatial attention weight matrix, and then the summation result is residually connected with the fused feature map to obtain a defect feature map that suppresses the texture of the product body and enhances the response weight of the defect area.

[0038] In specific implementation, projecting the saliency prior map into a conditional feature map that matches the shape of the fused feature map can be achieved in the following way: First, the saliency prior map is convolved and the number of channels is adjusted through a standard convolutional layer, projecting it into a conditional feature map that is completely consistent with the fused feature map to be modulated in terms of spatial resolution and number of channels. This allows the semantic prior information from the original image domain to effectively interact and fuse with the deep semantic features in the same feature space. The conditional feature map is an intermediate feature map that is spatially aligned with the fused feature map and serves as the input condition for attention calculation. It can encode the high-level semantic prior information from the image into a structured guiding signal that can directly interact with the deep semantic features.

[0039] In specific implementation, the fused feature map is projected as a query vector, and the conditional feature map is projected as a key vector and a value vector, respectively. The dot product similarity between the query vector and the key vector is then calculated to generate the spatial attention weight matrix. This can be achieved as follows: First, the fused feature map can be mapped to a query vector through a linear projection layer, and the conditional feature map can be mapped to a key vector and a value vector through two independent linear projection layers, respectively. Then, the dot product similarity between the query vector and the key vector is calculated to obtain a similarity matrix to evaluate the correlation between each position in the fused feature map and the salient region indicated by the semantic prior. The Softmax function is then applied to normalize the similarity matrix to generate the spatial attention weight matrix. The spatial attention weight matrix is ​​a weight distribution matrix that characterizes the degree of attention that cross-attention should give to different spatial positions in the fused feature map based on the semantic prior. It can dynamically and interpretably allocate differentiated attention to different spatial positions in the feature map based on the semantic prior, thereby achieving spatial recalibration of the features.

[0040] In specific implementation, the value vector is weighted and summed based on the spatial attention weight matrix, and the summation result is then residually connected with the fused feature map to obtain a defect feature map that suppresses product texture and enhances the response weight of defective areas. This can be achieved in the following way: the value vector is weighted and summed based on the generated spatial attention weight matrix, that is, information from the conditional feature map is selectively aggregated according to the attention weight to generate a context feature modulated by prior information. In order to retain all semantic information in the original fused feature map and avoid information loss caused by the modulation process, the context feature obtained by weighted summation is residually connected with the fused feature map, that is, element-wise addition is performed to finally generate an optimized feature map that contains rich semantics and has been guided by semantic priors to focus on potential defective areas as a defect feature map that suppresses product texture and enhances the response weight of defective areas.

[0041] It should be noted that in this application, the defect feature map is the final optimized feature map that integrates contextual information and prior guidance. It can significantly enhance the response of defect-related features while preserving the original complete semantic information, and effectively suppress the interference of non-defective texture background, providing a feature representation with strong discriminative power for high-precision detection.

[0042] In step 104, the defect feature map is classified and located, and then the category confidence and location bounding box of each defect in the second-hand goods image are integrated to generate defect recognition results.

[0043] In some embodiments, reference Figure 2The figure is an exemplary flowchart illustrating the determination of defect identification results according to some embodiments of this application. In this application, defect classification and localization of the defect feature map, followed by integrating the category confidence and localization bounding boxes of each defect in the second-hand goods image to generate defect identification results, can be achieved through the following steps: In step 1041, the defect feature map is classified and located to obtain the category confidence and location bounding box of each defect in the second-hand goods image. In step 1042, multiple merged bounding boxes are obtained by weighted merging of overlapping bounding boxes of each defect's location bounding box; In step 1043, the final defect identification result of the second-hand goods image is determined based on all merged bounding boxes and corresponding category confidence scores.

[0044] Preferably, in some embodiments, classifying and locating defects in the defect feature map to obtain the category confidence and location bounding box of each defect in the second-hand goods image can be achieved by the following steps: At each spatial location of the defect feature map, predefine multi-scale and multi-proportion anchor boxes, and determine the initial confidence level of the presence of defects within each anchor box and the corresponding bounding box offset. Based on the initial confidence level, multiple candidate defect areas were selected from all anchor frames; The coordinates of each candidate defect region are refined by using all bounding box offsets, and then the local feature blocks of each refined candidate defect region are extracted. Based on all local feature blocks, perform defect category logistic regression and bounding box coordinate regression respectively, and output the category confidence and local bounding box of each defect in the second-hand goods image.

[0045] In specific implementation, multi-scale and multi-ratio anchor boxes are predefined at each spatial location of the defect feature map. The initial confidence level of the presence of a defect within each anchor box and the corresponding bounding box offset can be determined in the following way: First, for each pixel on the defect feature map, a set of rectangular anchor boxes with different scales and aspect ratios needs to be predefined, such as covering three basic scales (small, medium, and large), as well as a height-to-slender ratio for linear scratches, a square ratio for blocky stains, and a width-to-flat ratio for irregular depressions, to match the common shapes of defects in second-hand goods. Then, a convolutional layer with a kernel size of 3x3 is applied to the defect feature map to perform a convolution operation, obtaining the values ​​of each pixel in the defect feature map. The initial prediction output for a set of predefined anchor boxes at a spatial location includes the initial confidence level indicating the probability that the anchor box contains a defective target, and the bounding box offset indicating that the anchor box needs to be adjusted to more tightly surround the defect, including the center point coordinate offset and the scaling values ​​of width and height. The initial confidence level is an initial probability estimate of the presence of a defect within the preset anchor box, which can provide a rapid initial screening basis for a large number of candidate regions, significantly reducing the subsequent computational complexity and improving processing efficiency. The bounding box offset is an adjustment parameter used to geometrically transform the preset anchor box to initially fit the potential defect location, which can achieve initial calibration of the rough anchor box position and provide a basis for obtaining a more accurate feature extraction region.

[0046] In specific implementation, the selection of multiple candidate defect regions from all anchor boxes based on the initial confidence level can be achieved in the following way: the initial confidence level of all anchor boxes can be processed by non-maximum suppression to select candidate defect regions. Specifically, background anchor boxes with an initial confidence level lower than a preset score threshold (e.g., 0.05) are filtered out. The remaining anchor boxes are then sorted according to their confidence scores, and the anchor box with the highest score is selected in turn. All other anchor boxes with a spatial overlap exceeding a preset threshold (e.g., 0.5) are suppressed. This process is iterated to finally obtain multiple candidate defect regions that are controllable in number and relatively independent of each other. The candidate defect regions are candidate image regions that may contain defects. They can focus the detection focus on high-probability areas, eliminate a large amount of background interference, and provide a high-quality input candidate set for fine classification and localization.

[0047] In specific implementation, the coordinates of each candidate defect region are refined using all bounding box offsets, and then the local feature blocks of each refined candidate defect region are extracted. This can be achieved in the following way: For each candidate defect region, the coordinates of the candidate region are refined by offsetting the center point coordinates and scaling the width and height values ​​in the corresponding bounding box offsets to obtain a floating region that better fits the shape of the potential defect. Then, bilinear interpolation sampling is performed at the corresponding position of the defect feature map based on the coordinates of the floating region to obtain a standardized region feature block of a fixed size (e.g., 7x7) as the local feature block of the refined candidate defect region. The local feature block of each refined candidate defect region can be obtained through the above steps. The local feature block is a fixed-dimensional feature representation containing the regional context information of the candidate defect region. It can retain the details and semantic information of each candidate region and provide a unified feature input for the final parallel fine classification and regression tasks.

[0048] It should be noted that in this application, the category confidence is a probability distribution vector for determining whether an image region belongs to each category of defect. It can quantify the category judgment of defects in probabilistic form, not only providing classification results but also providing a measure of prediction reliability, thus enhancing the interpretability and decision-making basis of the system. The localization bounding box is a rectangular box coordinate that precisely defines the location and extent of defects in the image space. It can pinpoint the specific location and size of defects with pixel-level precision and is an indispensable spatial information output for downstream tasks such as quality assessment and pricing.

[0049] In practice, based on all local feature blocks, logistic regression of defect categories and bounding box coordinate regression are performed separately. The output of the category confidence and local bounding box of each defect in the second-hand goods image can be achieved in the following way: Each local feature block can be flattened into a vector and fed into two parallel independent fully connected sub-networks for fine prediction. One sub-network performs logistic regression of defect categories on the local feature block through a series of fully connected layers and the Softmax activation function, and outputs the probability distribution of the candidate defect region corresponding to the local feature block belonging to each defect category as the category confidence. The other sub-network performs bounding box coordinate regression on the local feature block through another series of fully connected layers, and outputs a more accurate local bounding box of the candidate defect region corresponding to the local feature block. Through the above steps, the category confidence and local bounding box of each defect in the second-hand goods image can be obtained.

[0050] In specific implementation, multiple merged bounding boxes can be obtained by weighted merging of overlapping bounding boxes of each defect location bounding box. This can be achieved in the following way: all location bounding boxes can be merged using a weighted box fusion algorithm. First, the cross-union ratio (CUI) of each pair of location bounding boxes is calculated, and they are divided into different clusters according to a preset CUI threshold (e.g., 0.5). Each cluster represents multiple observations of the same defect entity. For each cluster, the coordinates of each location bounding box within the cluster are weighted and averaged according to the aspect ratio of each location bounding box to generate a merged bounding box with more stable and accurate coordinates. At the same time, the category confidence of the merged bounding box is obtained by vector averaging of the category confidence of all location bounding boxes within the cluster. The merged bounding box is a single candidate box with more accurate location and higher stability. It can effectively aggregate multiple observation information, significantly improve the accuracy and robustness of single defect location, and eliminate duplicate predictions.

[0051] In practice, the final defect recognition result of the second-hand goods image can be determined based on all merged bounding boxes and their corresponding category confidence scores in the following way: all merged bounding boxes and their corresponding category confidence scores can be summarized and output as a structured list of detection results as the final defect recognition result of the second-hand goods image.

[0052] It should be noted that in this application, the defect identification result is a structured data set containing the precise location, category determination and corresponding confidence level of all defects, which can provide a unique, reliable and directly usable automated detection conclusion for subsequent tasks such as quality assessment and pricing of second-hand goods.

[0053] In another aspect, in some embodiments, this application provides a system for deep identification of image defects in second-hand goods, with reference to... Figure 3 The figure is a schematic diagram of the structure of a second-hand goods image defect depth recognition system 300 according to some embodiments of this application. The second-hand goods image defect depth recognition system 300 includes: an encoding module 301, a processing module 302, and an execution module 303, which are described below: Encoding module 301, in this application, is mainly used to perform hierarchical feature encoding on the input second-hand goods image to obtain a multi-level feature pyramid of the second-hand goods image; Processing module 302, in this application, is mainly used to extract feature representations of defects at different scales from the feature maps of each level in the multi-level feature pyramid by combining multiple dilated convolution branches with channel attention adaptive weights, and then fuse all feature representations to generate a fused feature map of defects in second-hand goods. It should be noted that the processing module 302 in this application is also used to determine the saliency prior map for image defect recognition based on the semantic difference features of defects in second-hand commodity images, and then to perform cross-spatial attention modulation on the fused feature map with the saliency prior map as a condition to obtain a defect feature map that suppresses the texture of the commodity body and enhances the response weight of the defect area. The execution module 303 in this application is mainly used to classify and locate defects in the defect feature map, and then integrate the category confidence and location bounding box of each defect in the second-hand goods image to generate defect recognition results.

[0054] Each module in the aforementioned secondhand goods image defect depth recognition system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0055] In another embodiment, this application provides a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 4 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores data related to a method for deep identification of defects in images of used goods. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program can implement a method for deep identification of defects in images of used goods.

[0056] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0057] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above embodiment of the method for deep identification of defects in images of second-hand goods.

[0058] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above embodiment of the method for deep identification of defects in images of second-hand goods.

[0059] In one embodiment, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps described in the embodiment of the method for deep identification of image defects in secondhand goods.

[0060] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0061] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for deep identification of defects in images of second-hand goods, characterized in that, Includes the following steps: Hierarchical feature encoding is performed on the input second-hand goods image to obtain a multi-level feature pyramid of the second-hand goods image; By combining multiple dilated convolution branches with channel attention adaptive weights, feature representations of defects at different scales are extracted from the feature maps of each level in the multi-level feature pyramid, and then all feature representations are fused to generate a fused feature map of defects in second-hand goods. Based on the semantic difference features of defects in second-hand product images, a saliency prior map for image defect recognition is determined. Then, the fusion feature map is subjected to cross-spatial attention modulation with the saliency prior map as a condition to obtain a defect feature map that suppresses the texture of the product body and enhances the response weight of the defect area. The defect feature map is classified and located, and then the category confidence and location bounding box of each defect in the second-hand goods image are integrated to generate defect recognition results.

2. The method as described in claim 1, characterized in that, Hierarchical feature encoding is performed on the input secondhand goods image to obtain a multi-level feature pyramid of the secondhand goods image, specifically including: The input second-hand goods image is forward-propagated using a pre-trained lightweight convolutional neural network to extract multiple initial feature maps from shallow to deep. All initial feature maps are input into the feature pyramid network for top-down and lateral connection feature fusion to generate multiple enhanced feature maps; A multi-level feature pyramid is constructed based on all the enhanced feature maps to represent second-hand goods.

3. The method as described in claim 1, characterized in that, By combining multiple dilated convolutional branches with channel attention adaptive weights, feature representations of defects at different scales are extracted from the feature maps of each level in the multi-level feature pyramid. Then, all feature representations are fused to generate a fused feature map of defects in second-hand goods. Specifically, this includes: Each enhanced feature map in the multi-level feature pyramid is input into an independent parallel dilated convolution branch group for convolution operation. Each dilated convolution branch group contains convolution kernels with different dilation rates, resulting in defect feature representations at different levels and with different receptive fields. Global average pooling is performed on the feature representations of all levels to generate hierarchical channel attention weight vectors; The feature representations of the corresponding levels are weighted and fused using the channel attention weight vectors of the hierarchical levels, and then the cross-level fused features are spliced ​​and dimensionality reduced to generate a fused feature map of defects in second-hand goods.

4. The method as described in claim 1, characterized in that, The saliency prior map for image defect recognition, determined based on semantic difference features of defects in second-hand product images, specifically includes: A pre-trained lightweight defect semantic segmentation network is used to perform forward reasoning on second-hand product images to obtain binary semantic mask maps of the product body region and the potential defect region. The binary semantic mask image is subjected to morphological dilation and erosion operations to eliminate isolated noise points at the mask edges, resulting in a preliminary semantic saliency map. The preliminary semantic saliency map is subjected to contrast stretching and Gaussian filtering to obtain a saliency prior map of image defects.

5. The method as described in claim 1, characterized in that, Using the saliency prior map as a condition, cross-spatial attention modulation is applied to the fused feature map to obtain a defect feature map that suppresses product body texture and enhances the response weight of defect regions. Specifically, this includes: The saliency prior map is projected into a conditional feature map that matches the shape of the fused feature map; The fused feature map is projected into a query vector, and the conditional feature map is projected into a key vector and a value vector, respectively. Then, the dot product similarity between the query vector and the key vector is calculated to generate a spatial attention weight matrix. The value vector is weighted and summed based on the spatial attention weight matrix, and then the summation result is residually connected with the fused feature map to obtain a defect feature map that suppresses the texture of the product body and enhances the response weight of the defect area.

6. The method as described in claim 1, characterized in that, The defect feature map is used to classify and locate defects, and then the category confidence scores and location bounding boxes of each defect in the second-hand goods image are integrated to generate defect recognition results. Specifically, this includes: The defect feature map is classified and located to obtain the category confidence and location bounding box of each defect in the second-hand goods image; By weighted merging of overlapping bounding boxes of each defect, multiple merged bounding boxes are obtained. The final defect identification result for the second-hand goods image is determined based on all merged bounding boxes and corresponding category confidence scores.

7. The method as described in claim 6, characterized in that, The defect feature map is used to classify and locate defects, resulting in the category confidence score and location bounding box of each defect in the second-hand goods image. Specifically, this includes: At each spatial location of the defect feature map, predefine multi-scale and multi-proportion anchor boxes, and determine the initial confidence level of the presence of defects within each anchor box and the corresponding bounding box offset. Based on the initial confidence level, multiple candidate defect areas were selected from all anchor frames; The coordinates of each candidate defect region are refined by using all bounding box offsets, and then the local feature blocks of each refined candidate defect region are extracted. Based on all local feature blocks, perform defect category logistic regression and bounding box coordinate regression respectively, and output the category confidence and local bounding box of each defect in the second-hand goods image.

8. A system for deep recognition of image defects in second-hand goods, characterized in that, include: The encoding module is used to perform hierarchical feature encoding on the input second-hand goods image to obtain a multi-level feature pyramid of the second-hand goods image; The processing module is used to extract feature representations of defects at different scales from the feature maps of each level in the multi-level feature pyramid by combining multiple dilated convolution branches with channel attention adaptive weights, and then fuse all feature representations to generate a fused feature map of defects in second-hand goods. The processing module is used to determine a saliency prior map for image defect recognition based on the semantic difference features of defects in second-hand commodity images, and then perform cross-spatial attention modulation on the fused feature map using the saliency prior map as a condition to obtain a defect feature map that suppresses the texture of the commodity body and enhances the response weight of the defect region. The execution module is used to classify and locate defects in the defect feature map, and then integrate the category confidence and location bounding box of each defect in the second-hand goods image to generate defect recognition results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for deep identification of defects in second-hand commodity images as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for deep identification of defects in second-hand commodity images as described in any one of claims 1 to 7.

Images