Sorting method and system based on differential upsampling, computer equipment and medium

Abstract

The invention relates to a sorting method and system based on differential upsampling, computer equipment and a medium. The differential upsampling-based sorting method comprises the following steps: respectively extracting multi-scale feature maps of different types of to-be-sorted objects by adopting a Backbone network; obtaining a category exclusive channel weight template based on the multi-scale feature map; performing category prediction on the multi-scale feature map by adopting a lightweight classifier to obtain category probability distribution; fusing the category exclusive channel weight template with the category probability distribution to obtain a dynamic channel weight matrix; the dynamic channel weight matrix is embedded into an FPN layer of YOLOv5 for adaptive fusion, and a weighted feature map is obtained; performing differential up-sampling on the weighted feature map to obtain a final feature map; and sorting the to-be-sorted objects based on the final feature map. The feature expression ability of different types of targets can be improved, sorting real-time performance and precision balance is achieved, and the real-time performance requirement of 30 FPS is met.

Description

Technical Field

[0001] This application relates to the field of sorting technology, and in particular to a sorting method, system, computer equipment, and medium based on differential upsampling. Background Technology

[0002] In industrial sorting scenarios, the visual characteristics of items to be sorted (such as metal parts and cardboard boxes) vary significantly. For example, metal parts have sharp edges, while cardboard boxes have complex textures. These differences make it difficult for traditional object detection models to simultaneously achieve both real-time performance and classification accuracy. Existing detection models cannot dynamically adjust the weights of key feature channels for different types of objects based on their visual characteristics. For instance, the edge features of metal parts are crucial for positioning accuracy, while the texture features of cardboard boxes are key to distinguishing them from other objects. However, the fixed channel weights of existing detection models cannot enhance these differentiated features, leading to key features being interfered with by redundant background information or the loss of low-level texture features, ultimately reducing classification accuracy. At the same time, the complex feature processing modules introduced to improve accuracy increase computational load, further compressing the real-time performance space and making it difficult to achieve a balance between accuracy and efficiency. Summary of the Invention

[0003] Therefore, it is necessary to provide a sorting method, system, computer equipment, and medium based on differential upsampling to address the aforementioned technical problems.

[0004] In a first aspect, this application provides a sorting method based on differential upsampling, the sorting method based on differential upsampling comprising: extracting multi-scale feature maps of different categories of items to be sorted using a Backbone network; obtaining category-specific channel weight templates based on the multi-scale feature maps; performing category prediction on the multi-scale feature maps using a lightweight classifier to obtain a category probability distribution; fusing the category-specific channel weight templates with the category probability distribution to obtain a dynamic channel weight matrix; embedding the dynamic channel weight matrix into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map; performing differential upsampling on the weighted feature map to obtain a final feature map; and sorting the items to be sorted based on the final feature map.

[0005] Optionally, obtaining category-specific channel weight templates based on the multi-scale feature maps includes: performing feature analysis on the multi-scale feature maps of each category of items to be sorted using depthwise separable convolution to obtain the gradient importance score of each channel in the classification task; and obtaining category-specific channel weight templates based on the gradient importance scores through Softmax normalization.

[0006] Optionally, the lightweight classifier is a classifier constructed using three depthwise separable convolutions; the lightweight classifier is used to predict the class of the multi-scale feature map to obtain the class probability distribution, including: processing the multi-scale feature map using a 3×3 depthwise convolution, and processing the multi-scale feature map after the 3×3 depthwise convolution using a 1×1 pointwise convolution to obtain a dimensionality-reduced feature map; and processing the dimensionality-reduced feature map based on global average pooling and fully connected layers to obtain the class probability distribution.

[0007] Optionally, the category-specific channel weight template is fused with the category probability distribution to obtain a dynamic channel weight matrix, including: fusing the category-specific channel weight template with the category probability distribution to obtain initial dynamic weight values; and performing nonlinear mapping and normalization processing on the initial dynamic weight values ​​using an improved Sigmoid function to obtain the dynamic channel weight matrix.

[0008] Optionally, the dynamic channel weight matrix is ​​embedded into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map, including: embedding the dynamic channel weight matrix into the FPN layer of YOLOv5; inputting the multi-scale feature map into the FPN layer for channel-dimensional slicing to obtain the feature map of the t-th channel; and performing channel-by-channel weighting based on the dynamic channel weight matrix and the feature map of the t-th channel to obtain a weighted feature map.

[0009] Optionally, the items to be sorted include metal parts and cardboard boxes; differential upsampling of the weighted feature map to obtain a final feature map includes: using a category feature analyzer to perform channel compression processing on the weighted feature map to obtain a category feature descriptor; mapping the category feature descriptor to a policy probability space through learnable parameters to obtain an initial policy probability; generating a final policy probability based on the initial policy probability using an improved Gumbel-Softmax algorithm; processing the metal parts using edge-preserving bilinear interpolation to obtain a metal part feature map; processing the cardboard boxes using depthwise separable transposed convolution to obtain a cardboard box feature map; and weighted fusion of the metal part feature map and the cardboard box feature map based on the final policy probability to obtain the final feature map.

[0010] Optionally, after embedding the dynamic channel weight matrix into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map, the method further includes: dynamically optimizing the template parameters of the category-specific channel weight template based on an online learning strategy.

[0011] Secondly, this application also provides a sorting system based on differential upsampling, the sorting system based on differential upsampling comprising: a category-specific channel weight template generation module, used to extract multi-scale feature maps of different categories of items to be sorted using a Backbone network; and to obtain a category-specific channel weight template based on the multi-scale feature maps; a pre-lightweight classifier, used to perform category prediction on the multi-scale feature maps using the lightweight classifier to obtain a category probability distribution; a dynamic fusion module, used to fuse the category-specific channel weight template with the category probability distribution to obtain a dynamic channel weight matrix; and to embed the dynamic channel weight matrix into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map; a differential upsampling module, used to perform differential upsampling on the weighted feature map to obtain a final feature map; and an item sorting module, used to sort items based on the final feature map.

[0012] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the sorting methods based on differential upsampling described above.

[0013] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the sorting methods based on differential upsampling described above.

[0014] The aforementioned sorting method, system, computer equipment, and medium based on differential upsampling dynamically adapt to the visual characteristics of the target by establishing category-specific channel weight templates, designing guiding feature fusion directions, and combining differential upsampling with dynamic channel weight adjustment to optimize multi-scale feature transfer. Ultimately, without significantly increasing detection latency, it enhances the feature representation capabilities of different categories of targets, achieves a balance between sorting real-time performance and accuracy, and meets the 30FPS real-time requirement. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a sorting method based on differential upsampling provided in one embodiment;

[0016] Figure 2 This is a block diagram of a sorting system based on differential upsampling provided in another embodiment;

[0017] Figure 3 This is an internal structural diagram of a computer device provided in yet another embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] In one embodiment, such as Figure 1 As shown, this application provides a sorting method based on differential upsampling, which may include the following steps: S10~S60.

[0020] S10: Use the Backbone network to extract multi-scale feature maps of different categories of items to be sorted; and obtain category-specific channel weight templates based on the multi-scale feature maps.

[0021] S20: Use a lightweight classifier to predict the category of the multi-scale feature map to obtain the category probability distribution.

[0022] S30: The category-specific channel weight template is fused with the category probability distribution to obtain a dynamic channel weight matrix.

[0023] S40: Embed the dynamic channel weight matrix into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map.

[0024] S50: Perform differential upsampling on the weighted feature map to obtain the final feature map.

[0025] S60: Sorting of items to be sorted based on the final feature map.

[0026] In the sorting method based on differential upsampling in this application, the visual characteristics of the target are dynamically adapted by establishing a category-specific channel weight template, and the direction of feature fusion is designed. The multi-scale feature transfer is optimized by combining differential upsampling and dynamic channel weight adjustment. Finally, without significantly increasing the detection latency, the feature expression ability of different categories of targets is improved, and the real-time performance and accuracy of sorting are balanced, meeting the real-time requirement of 30FPS.

[0027] In step S10, please refer to Figure 1 In step S10, the Backbone network is used to extract multi-scale feature maps of different categories of items to be sorted; and category-specific channel weight templates are obtained based on the multi-scale feature maps.

[0028] As an example, in step S10, for different categories of items to be sorted (e.g., metal parts or cardboard boxes), multi-scale feature maps can be extracted from the Backbone network (e.g., CSP Darknet53), typically selecting three layers of features: P3 (small target), P4 (medium target), and P5 (large target).

[0029] As an example, in step S10, obtaining the category-specific channel weight template based on the multi-scale feature map may include the following: performing feature analysis on the multi-scale feature map of each category of items to be sorted using depthwise separable convolution to obtain the gradient importance score of each channel in the classification task; and obtaining the category-specific channel weight template based on the gradient importance score through Softmax normalization.

[0030] As an example, depthwise separable convolution is used to perform feature analysis on the multi-scale feature maps of each category of items to be sorted, in order to obtain the gradient importance score of each channel in the classification task. This can include the following: For the multi-scale feature map of each category of items to be sorted, a combination of depthwise separable convolution ("3×3 depthwise convolution + 1×1 pointwise convolution") is used for feature analysis. Among them, the depthwise convolution (3×3) independently processes the spatial features of each input channel without interfering with each channel, and accurately captures the local features of a single channel (such as the edge texture of metal parts, the surface texture of cardboard boxes, etc.). The pointwise convolution (1×1) realizes the linear combination between channels and calculates the gradient importance score of each channel in the classification task. The higher the gradient importance score, the greater the contribution of the channel to the current category discrimination.

[0031] As an example, based on the gradient importance score, a category-specific channel weight template is obtained through Softmax normalization, which may include the following: Based on the gradient importance score, the weights of all channels are normalized using the Softmax function to generate a category-specific channel weight template; wherein, for key channels (such as the edge channel of metal parts, the texture channel of cardboard boxes): the weight can be assigned a value of 0.8~1.2 to enhance their feature expression; for non-key channels (such as redundant background channels): the weight can be suppressed to 0~0.3 to reduce interference from invalid features.

[0032] As an example, all category-specific channel weight templates can be stored in a lightweight lookup table (LUT) as key-value pairs, with the key being the target category number and the value being the corresponding channel weight vector, supporting fast lookup with O(1) time complexity.

[0033] As an example, in step S10, the feature map channel response patterns of different categories of items to be sorted can be analyzed using a depthwise separable convolutional network to construct a category-specific channel weight template with category discrimination.

[0034] In step S20, please refer to Figure 1 In step S20, a lightweight classifier is used to predict the category of the multi-scale feature map to obtain the category probability distribution.

[0035] As an example, in step S20, the lightweight classifier can be a classifier constructed using three layers of depthwise separable convolutions.

[0036] As an example, in step S20, a lightweight classifier is used to predict the category of the multi-scale feature map to obtain the category probability distribution, including: S201~S203.

[0037] S201: The multi-scale feature map is processed by 3×3 depth convolution, and the multi-scale feature map after 3×3 depth convolution is processed by 1×1 pointwise convolution to obtain a dimensionality-reduced feature map.

[0038] S202: The dimensionality-reduced feature map is processed based on global average pooling and a fully connected layer to obtain the class probability distribution.

[0039] As an example, in step S201, a 3×3 depthwise convolution is used as the first layer, independently processing the spatial information of each input channel to capture local features of the items to be sorted (such as the edges of metal parts, the texture of cardboard boxes, etc.) while maintaining channel independence. The corresponding formula can be as follows:

[0040]

[0041] in, The multi-scale feature map after 3×3 depthwise convolution is located at... eigenvalues ​​at that location represents the spatial dimension (such as rows and columns of a multi-scale feature map), and k is the third dimension (such as the depth channel, time frame, or disparity dimension in stereo vision). The original pixel values ​​of the input multi-scale feature map within the region covered by the convolution kernel; These are the learnable weight parameters for the depthwise convolution kernel.

[0042] As an example, in step S201, during the process of processing the multi-scale feature map using 3×3 depth convolution, spatial convolution can be performed independently on 256 channels, and the local spatial feature map of each channel (i.e., the multi-scale feature map after 3×3 depth convolution processing) can be extracted.

[0043] As an example, in step S201, during the processing of the multi-scale feature map after 3×3 depthwise convolution using 1×1 pointwise convolution, channel fusion is achieved, reducing the dimensionality to 1 / 4 of the original number of channels to reduce computational load. The corresponding formula can be as follows:

[0044]

[0045] in, The feature value at the specified location is the feature map of the reduced dimension output after 1×1 pointwise convolution. The multi-scale feature map after 3×3 depthwise convolution is located at... Eigenvalues ​​at; is the weight of the 1×1 pointwise convolution kernel, used to map the input channel k to the output channel l; C is the total number of input channels.

[0046] As an example, in step S202, the dimensionality-reduced feature map is processed based on global average pooling and a fully connected layer to obtain the class probability distribution. The corresponding formula can be as follows:

[0047]

[0048] in, H is the feature value at position 1 in the reduced feature map output after 1×1 pointwise convolution; H is the height of the reduced feature map; W is the width of the reduced feature map. To perform global average pooling on the dimensionality-reduced feature map; This is the weight matrix of the fully connected layer.

[0049] As an example, step S20 may include dynamic pruning and computational optimization after step S202, specifically including the following:

[0050] Channel pruning: Close low-contribution channels based on pre-classification confidence; if the mean activation value of a certain channel k is... If so, subsequent calculations for that channel are skipped, which can reduce redundant calculations by 30%. It is the maximum mean;

[0051] Mixed-precision inference: for high-confidence categories (e.g., category probability distributions) For values ​​>0.8, FP16 (half-precision) calculation can be used, while the rest can be calculated using FP32 (single-precision) to balance accuracy and speed.

[0052] As an example, in step S20, after dynamic pruning and computational optimization, class-guided feature enhancement can also be included, specifically including the following: a class-aware gate can be inserted after a 1×1 pointwise convolution, and the corresponding formula can be as follows:

[0053]

[0054] in, This is the gate vector; It is the sigmoid activation function; The weight matrix is ​​a learnable matrix; For bias terms; This represents the category probability distribution.

[0055] As an example, the enhanced feature map can be obtained based on the gate vector, and the corresponding formula is as follows:

[0056]

[0057] in, To enhance the feature map; For dimensionality reduction feature maps; This is the gate vector; This is a channel-by-channel multiplication.

[0058] In step S30, please refer to Figure 1 In step S30, the category-specific channel weight template is fused with the category probability distribution to obtain a dynamic channel weight matrix.

[0059] As an example, in step S30, the category-specific channel weight template is fused with the category probability distribution to obtain a dynamic channel weight matrix, which may include the following steps S301~S302.

[0060] S301: The category-specific channel weight template is fused with the category probability distribution to obtain the initial dynamic weight value.

[0061] S302: The dynamic weight initial value is nonlinearly mapped and normalized using an improved Sigmoid function to obtain the dynamic channel weight matrix.

[0062] As an example, in step S301, the category-specific channel weight template can be weighted and fused based on the category probability distribution to generate dynamic initial weight values. The corresponding formula can be as follows:

[0063]

[0064] Where C represents the total number of categories (e.g., metal parts, cardboard boxes). For class probability distribution; , representing the preset weight strength of category c in channel t.

[0065] As an example, in step S302, a modified Sigmoid function is used to perform nonlinear mapping and normalization on the initial values ​​of the dynamic weights to obtain the dynamic channel weight matrix. The formula can be expressed as follows:

[0066]

[0067] in, These are the initial values ​​for the dynamic weights; Control the slope to enhance the weight discrimination; Achieve zero centralization and avoid weight bias towards extreme values.

[0068] In step S40, please refer to Figure 1 In step S40, the dynamic channel weight matrix is ​​embedded into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map.

[0069] As an example, in step S40, the dynamic channel weight matrix is ​​embedded into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map, which may include the following steps: S401~S403.

[0070] S401: Embed the dynamic channel weight matrix into the FPN layer of YOLOv5.

[0071] S402: Input the multi-scale feature map into the FPN layer to perform channel-dimensional slicing to obtain the feature map of the t-th channel.

[0072] S403: Perform channel-by-channel weighting based on the dynamic channel weight matrix and the feature map of the t-th channel to obtain a weighted feature map.

[0073] As an example, in step S401, the dynamic channel weight matrix is... Embedded within the FPN layer of YOLOv5, it achieves adaptive fusion of feature map channel dimensions.

[0074] As an example, in step S402, the multi-scale feature map is input into the FPN layer for channel-dimensional slicing to obtain the feature map of the t-th channel. .

[0075] As an example, in step S403, based on the dynamic channel weight matrix... and the feature map of the t-th channel Perform channel-by-channel weighting to obtain a weighted feature map. The formula can be expressed as follows:

[0076] .

[0077] As an example, after embedding the dynamic channel weight matrix into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map, the method further includes: dynamically optimizing the template parameters of the category-specific channel weight template based on an online learning strategy.

[0078] In step S50, please refer to Figure 1In step S50, the weighted feature map is differentially upsampled to obtain the final feature map.

[0079] As an example, in step S50, the items to be sorted include metal parts and cartons; differential upsampling of the weighted feature map to obtain the final feature map may include the following steps: S501~S506.

[0080] S501: Use a category feature analyzer to perform channel compression processing on the weighted feature map to obtain a category feature descriptor.

[0081] S502: Map the category feature descriptor to the policy probability space using learnable parameters to obtain the initial policy probability.

[0082] S503: Based on the initial policy probability, the improved Gumbel-Softmax algorithm is used to generate the final policy probability.

[0083] S504: Bilinear interpolation with edge preservation of the sampling band is used to process the metal part to obtain the feature map of the metal part.

[0084] S505: The carton is processed using depthwise separable transpose convolution to obtain a feature map of the carton.

[0085] S506: The feature map of the metal part and the feature map of the carton are weighted and fused based on the final strategy probability to obtain the final feature map.

[0086] As an example, in step S501, the weighted feature map can be processed by a 1×1 convolutional layer. Channel compression is performed to generate category feature descriptors. , This means that each element of the category feature descriptor is a real number, the corresponding feature map has a height of 1, a width of 1, and a number of channels (number of categories) of C; the corresponding formula can be as follows:

[0087]

[0088] in, It is a 1×1 convolutional layer; C represents the category of the item to be sorted. The category feature descriptor... Global average pooling is used to capture the statistical distribution of features of each category, providing a basis for the selection of subsequent upsampling strategies.

[0089] As an example, in step S502, learnable parameters can be used. Category feature descriptors Mapping to the policy probability space yields the initial policy probabilities. The corresponding formula can be as follows:

[0090]

[0091] Where Softmax(.) is the softmax activation function; The weight matrix is ​​a learnable matrix; For category feature descriptors; This is a bias term.

[0092] As an example, in step S503, based on the initial policy probability, an improved Gumbel-Softmax algorithm can be used to achieve continuous selection of discrete policies, thereby generating the final policy probability. The corresponding formula can be as follows:

[0093]

[0094] in, Let be the initial policy probability of the i-th policy; The Gumbel noise for the i-th strategy; Let be the initial policy probability of the j-th policy; For the j-th strategy, there is Gumbel noise; This refers to the temperature parameter.

[0095] As an example, in step S504, the formula for processing the metal part using bilinear interpolation with edge preservation of the sampling band to obtain the feature map of the metal part can be as follows:

[0096]

[0097] in, This represents the value of the feature map of the metal part at coordinates (x, y) after upsampling; This represents the value of the weighted feature map at coordinates (i,j); Interpolation weights in the x-direction; The interpolation weights are for the y-direction.

[0098] As an example, in step S505, a depthwise separable transposed convolution is used to process the cardboard box to obtain a feature map of the cardboard box. The formula can be expressed as follows:

[0099]

[0100] in, This is a depthwise separable transpose convolution operation; For weighted feature maps; This indicates that the kernel size is 3×3.

[0101] As an example, in step S06, the feature map of the metal part and the feature map of the cardboard box are weighted and fused based on the final policy probability to obtain the final feature map. That is, the two upsampling results are mixed by dynamic weights to obtain the final feature map. The corresponding formula can be as follows:

[0102]

[0103] in, Feature diagram of a metal part; This is a feature diagram of a cardboard box; Based on the final policy probability The obtained metal component fusion weights; Based on the final policy probability The obtained cardboard box fusion weight.

[0104] In step S60, please refer to Figure 1 In step S60, the items to be sorted are sorted based on the final feature map.

[0105] As an example, different categories of items to be sorted can be sorted based on the final feature map.

[0106] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0107] In one embodiment, such as Figure 2As shown, a sorting system based on differential upsampling is provided. The system may include: a category-specific channel weight template generation module 10, a pre-processor lightweight classifier 20, a dynamic fusion module 30, a differential upsampling module 40, and a sorting module 50 for items to be sorted. The category-specific channel weight template generation module 10 is used to extract multi-scale feature maps of different categories of items to be sorted using a Backbone network; and to obtain category-specific channel weight templates based on the multi-scale feature maps. The pre-processor lightweight classifier 20 is used to... A lightweight classifier is used to predict the category of the multi-scale feature map to obtain the category probability distribution; the dynamic fusion module 30 is used to fuse the category-specific channel weight template with the category probability distribution to obtain a dynamic channel weight matrix; the dynamic channel weight matrix is ​​embedded into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map; the differential upsampling module 40 is used to perform differential upsampling on the weighted feature map to obtain a final feature map; the sorting module 50 is used to sort the items to be sorted based on the final feature map.

[0108] The sorting system based on differential upsampling in this embodiment can be used to perform tasks such as... Figure 1 The sorting method based on differential upsampling described in the corresponding embodiments, and the specific limitations of the sorting system can be found in the limitations of the sorting method based on differential upsampling above, and will not be repeated here. Each module in the above-mentioned sorting system based on differential upsampling can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0109] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data such as category-specific channel weight templates, category probability distributions, dynamic channel weight matrices, weighted feature maps, and final feature maps. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a sorting method based on differential upsampling.

[0110] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a sorting method based on differential upsampling. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0111] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0112] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement, for example... Figure 1 The steps of the sorting method based on differential upsampling in its corresponding embodiments.

[0113] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program implementing, when executed by a processor, as shown in the figure. Figure 1 The steps of the sorting method based on differential upsampling in its corresponding embodiments.

[0114] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A sorting method based on differential up-sampling, characterized in that, The method comprises the following steps: extracting multi-scale feature maps of different categories of to-be-sorted objects by using a Backbone network; obtaining a category-specific channel weight template based on the multi-scale feature maps; performing category prediction on the multi-scale feature maps by using a lightweight classifier to obtain a category probability distribution; fusing the category-specific channel weight template and the category probability distribution to obtain a dynamic channel weight matrix; embedding the dynamic channel weight matrix into an FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map; performing differential upsampling on the weighted feature map to obtain a final feature map; sorting the to-be-sorted objects based on the final feature map.

2. The method of claim 1, wherein, The method for obtaining the category-specific channel weight template based on the multi-scale feature maps comprises the following steps: performing feature analysis on the multi-scale feature maps of each category of to-be-sorted objects by using a depth separable convolution to obtain gradient importance scores of each channel in the classification task; obtaining the category-specific channel weight template by performing Softmax normalization processing based on the gradient importance scores.

3. The method of claim 1, wherein, The lightweight classifier is a classifier constructed by using three layers of depth separable convolution; the method for performing category prediction on the multi-scale feature maps by using the lightweight classifier to obtain a category probability distribution comprises the following steps: performing processing on the multi-scale feature maps by using a 3*3 depth convolution, and performing processing on the multi-scale feature maps processed by the 3*3 depth convolution by using a 1*1 point-by-point convolution to obtain a dimension-reduced feature map; performing processing on the dimension-reduced feature map by using a global average pooling and a fully connected layer to obtain a category probability distribution.

4. The method of claim 1, wherein, The method for fusing the category-specific channel weight template and the category probability distribution to obtain a dynamic channel weight matrix comprises the following steps: fusing the category-specific channel weight template and the category probability distribution to obtain a dynamic weight initial value; performing non-linear mapping and normalization processing on the dynamic weight initial value by using an improved Sigmoid function to obtain the dynamic channel weight matrix.

5. The method of claim 1, wherein, The method for embedding the dynamic channel weight matrix into an FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map comprises the following steps: embedding the dynamic channel weight matrix into the FPN layer of YOLOv5; inputting the multi-scale feature maps into the FPN layer for channel dimension slicing to obtain a feature map of the tth channel; performing channel-by-channel weighting based on the dynamic channel weight matrix and the feature map of the tth channel to obtain a weighted feature map.

6. The method of claim 1, wherein, The to-be-sorted objects comprise metal pieces and cartons; the method for performing differential upsampling on the weighted feature map to obtain a final feature map comprises the following steps: performing channel compression processing on the weighted feature map by using a category feature analyzer to obtain a category feature descriptor; mapping the category feature descriptor to a strategy probability space by using a learnable parameter to obtain an initial strategy probability; generating a final strategy probability by using an improved Gumbel-Softmax algorithm based on the initial strategy probability; performing processing on the metal pieces by using a bilinear interpolation with band edge preservation sampling to obtain a metal piece feature map; performing processing on the cartons by using a depth separable transpose convolution to obtain a carton feature map; The metal part feature map and the carton feature map are weighted and fused based on the final strategy probability to obtain the final feature map.

7. The method of claim 1, wherein, After embedding the dynamic channel weight matrix into the FPN layer of YOLOv5 for adaptive fusion to obtain the weighted feature map, the method further comprises dynamically optimizing template parameters of the class-specific channel weight template based on an online learning strategy.

8. A singulation system based on differential up-sampling, characterized in that, The sorting system based on differential upsampling comprises: a class-specific channel weight template generation module configured to extract multi-scale feature maps of different categories of to-be-sorted objects by using a Backbone network, and obtain a class-specific channel weight template based on the multi-scale feature maps; a front lightweight classifier configured to predict the categories of the multi-scale feature maps by using a lightweight classifier to obtain a category probability distribution; a dynamic fusion module configured to fuse the class-specific channel weight template and the category probability distribution to obtain a dynamic channel weight matrix, and embed the dynamic channel weight matrix into the FPN layer of YOLOv5 for adaptive fusion to obtain a weighted feature map; a differential upsampling module configured to perform differential upsampling on the weighted feature map to obtain a final feature map; a to-be-sorted object sorting module configured to sort to-be-sorted objects based on the final feature map. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images