Material recognition method, sorting method, and sorting apparatus

Abstract

The present disclosure relates to the technical field of computer vision, in particular to a material identification method, a sorting method and a sorting device. The material identification method applied to the sorting device comprises: acquiring images of materials in a conveying process by a dual-energy ray identification device to obtain low-energy ray images and high-energy ray images of the materials; performing global perception processing on the low-energy ray images and the high-energy ray images respectively, and fusing the obtained processing results to obtain a fusion feature map; matching feature vectors of the fusion feature map with a plurality of preset dynamic class prototype features respectively to determine similarities between the feature vectors and the dynamic class prototype features; and determining a material class of the materials based on a comparison result of the plurality of similarities and a similarity threshold. The judgment process of the material class is simple and efficient, and the material identification efficiency can be improved on the basis of ensuring the identification accuracy, thereby facilitating the acceleration of the material sorting process and improving the overall sorting performance of the sorting device.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, specifically to a material identification method, sorting method, and sorting equipment. Background Technology

[0002] With the development of the global circular economy, the sorting accuracy of recyclable resources such as waste metals, waste plastics, and electronic waste directly affects their recycling value. Among related technologies, recyclable resource sorting mainly relies on manual sorting or machine vision sorting based on color or a single spectrum. However, manual sorting is costly, inefficient, and its quality is easily affected by the sorting conditions of personnel. Traditional machine vision relies solely on appearance features to identify materials, making it difficult to distinguish between materials that look similar but have different materials, resulting in poor identification accuracy and impacting the sorting performance of the equipment.

[0003] Therefore, there is an urgent need for a material identification method that can balance sorting efficiency and identification accuracy. Summary of the Invention

[0004] To overcome the problems existing in related technologies, an exemplary embodiment of this disclosure provides a material identification method applied to a sorting device. The sorting device includes a dual-energy X-ray identification device. The method includes: acquiring images of materials during the conveying process using the dual-energy X-ray identification device to obtain low-energy X-ray images and high-energy X-ray images of the materials; performing global perception processing on the low-energy X-ray images and high-energy X-ray images respectively, and fusing the processing results to obtain a fused feature map; matching the feature vector of the fused feature map with multiple preset dynamic category prototype features to determine the similarity between each dynamic category prototype feature, wherein the multiple dynamic category prototype features correspond one-to-one with multiple candidate material categories, and the dynamic category prototype features are obtained after clustering based on the sample feature vector set of the corresponding candidate material category; and determining the material category of the material based on the comparison results of multiple similarities and similarity thresholds.

[0005] In some embodiments, global perception processing is performed on low-energy ray images and high-energy ray images respectively, and the processing results are fused to obtain a fused feature map, including: performing global perception processing on low-energy ray images and high-energy ray images respectively using a convolution method that expands the receptive field to generate corresponding low-energy convolution feature maps and high-energy convolution feature maps; and fusing the low-energy convolution feature maps and high-energy convolution feature maps to obtain a fused feature map.

[0006] In some embodiments, fusing low-energy convolutional feature maps and high-energy convolutional feature maps to obtain a fused feature map includes: performing feature enhancement processing on the low-energy convolutional feature maps and high-energy convolutional feature maps respectively through a multivariate self-attention mechanism to obtain enhanced low-energy convolutional feature maps and high-energy convolutional feature maps; and fusing the enhanced low-energy convolutional feature maps and high-energy convolutional feature maps to obtain a fused feature map.

[0007] In some embodiments, the process of determining the dynamic category prototype features includes: acquiring a standard low-energy transmission image set and a standard high-energy transmission image set corresponding to the current candidate material category using a dual-energy ray recognition device, wherein the standard low-energy transmission image set and the standard high-energy transmission image set correspond one-to-one; performing global perception processing on each set of corresponding standard low-energy transmission images and standard high-energy transmission images, and fusing the processing results to obtain a sample fusion feature map set; extracting features from the sample fusion feature map set using a first model to obtain a sample feature vector set; and performing clustering processing on the sample feature vector set in the feature space using a clustering algorithm to obtain the dynamic category prototype features of the current candidate material category.

[0008] In some embodiments, the method includes: extracting features from the fused feature map using a second model to obtain a feature vector, wherein the second model and the first model are two branches of a Siamese network, and the second model and the first model share weights and have the same structure.

[0009] In some embodiments, the feature vector of the fused feature map is matched with multiple preset dynamic category prototype features respectively to determine the similarity between each dynamic category prototype feature, including: determining the feature distance between the feature vector and the current dynamic category prototype feature; and determining the similarity between the feature vector and the current dynamic category prototype feature based on the feature distance.

[0010] In some embodiments, determining the material category of a material based on comparison results of multiple similarities and similarity thresholds includes: comparing multiple similarities with the similarity thresholds respectively; if there is at least one target similarity, then taking the candidate material category corresponding to the target similarity with the largest value as the material category of the material, wherein the target similarity is a similarity greater than or equal to the similarity threshold; if there is no target similarity, then determining the material category of the material as an unknown material.

[0011] In some embodiments, if the material category of the material is determined to be an unknown material, the method further includes: collecting image features of the unknown material and clustering them to generate dynamic category prototype features corresponding to the unknown material; obtaining the true category of the unknown material; establishing a mapping relationship between the dynamic category prototype features corresponding to the unknown material and the true category; expanding the dynamic category prototype features corresponding to the unknown material into multiple dynamic category prototype features according to the mapping relationship, and updating the candidate material category.

[0012] Secondly, this disclosure also provides a sorting method applied to a sorting device, the sorting device including a dual-energy X-ray identification device, the method comprising: acquiring an image of a material through the dual-energy X-ray identification device; determining the material category of the material through the material identification method provided in any of the foregoing aspects; and sorting the material based on the material category.

[0013] Thirdly, this disclosure also provides a sorting device, comprising: a conveying device for conveying materials; a dual-energy X-ray identification device for determining the material category of the materials using the material identification method provided in any of the foregoing aspects; and a sorting device for sorting materials based on the material category.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0015] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: According to the material identification method provided by this disclosure, by performing global perception processing on dual-energy X-ray images separately, the overall transmission distribution characteristics of the material within the image can be fully explored, reducing the adverse effects caused by various feature interferences. Fusing the processing results after global perception processing of the dual-energy X-ray images can compensate for the shortcomings of single-energy-level image feature information being one-sided and insufficient in representation, obtaining a fused feature map containing comprehensive material material information, which helps improve material identification accuracy. Furthermore, determining the material category based on the similarity between the feature vector extracted from the fused feature map and multiple preset dynamic category prototype features makes the material category judgment process simple and efficient, improving material identification efficiency while ensuring identification accuracy, thereby accelerating the material sorting process and improving the overall sorting performance of the sorting equipment. Attached Figure Description

[0016] This disclosure can be better understood by describing exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which: Figure 1 This is a schematic diagram of the architecture of a sorting device according to an exemplary embodiment of the present disclosure; Figure 2 This is a schematic flowchart illustrating a material identification method according to an exemplary embodiment of the present disclosure; Figure 3 This is a flowchart illustrating a sorting method according to an exemplary embodiment of the present disclosure; Figure 4 This is a schematic diagram of the structure of a sorting device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0017] The following describes specific embodiments of this disclosure. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. It should be understood that, in the actual implementation of any embodiment, just as in any engineering or design project, various specific decisions are often made to achieve the developer's specific goals and to meet system-related or business-related constraints, and this can change from one embodiment to another. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this disclosure, changes in design, manufacturing, or production based on the technical content disclosed in this disclosure are merely conventional technical means and should not be construed as insufficient content of this disclosure.

[0018] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. The terms “a” or “one,” etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising,” “including,” etc., mean that the element or object preceding “comprising” or “including” encompasses the elements or objects listed following “comprising” or “including” and their equivalents, and do not exclude other elements or objects. The terms “connected,” “linked,” etc., are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0019] Sorting equipment typically uses a conveyor belt to feed materials into the equipment. A radiographic testing device emits X-rays to acquire X-ray images of the materials. These images are then analyzed and processed using a pre-defined material sorting algorithm to identify and sort the materials. For example... Figure 1 As shown, the sorting device 100 may include a feeding device 110, a conveying device 120, a detection device 130, and a sorting device 140. The feeding device 110 feeds the material to be sorted into the conveying device 120. The conveying device 120 may be a conveyor belt or a chute, etc., used to transport the material to be sorted fed by the feeding device 110. The detection device 130 detects the material conveyed on the conveying device 120 to determine whether the material is to be removed; the material to be removed refers to the material that will be separated by the sorting device. The material to be removed can be desired material or undesirable material, as long as the material can be sorted. The sorting device 140 removes the material to be removed.

[0020] In related technologies, the sorting of recycled resources mainly relies on manual sorting or machine vision sorting based on color or a single spectrum. However, manual sorting is costly, inefficient, and its quality is easily affected by the condition of the personnel. Traditional machine vision relies solely on appearance features to identify materials, but in scenarios with mixed materials, it is difficult to distinguish between materials that look similar but have different materials, such as different types of plastics with similar colors or various metals with similar appearances. Insufficient recognition accuracy will affect the actual sorting effect of the sorting equipment.

[0021] In view of this, the present disclosure provides a material identification method for use in a sorting equipment. The sorting equipment includes a dual-energy X-ray identification device. For example... Figure 2 As shown, the material identification method may include the following steps: Step S210: The material during the conveying process is image acquired by the dual-energy X-ray identification device to obtain low-energy X-ray images and high-energy X-ray images of the material.

[0022] A dual-energy X-ray identification device is a device that emits both high-energy and low-energy X-rays, utilizing the difference in attenuation coefficients of different materials to achieve material imaging. The image acquisition area of ​​the dual-energy X-ray identification device corresponds to the position of the transmission device in the sorting equipment.

[0023] During the material transport process, when the material enters the image acquisition area of ​​the dual-energy X-ray identification device, the dual-energy X-ray identification device emits high-energy X-rays and low-energy X-rays to simultaneously acquire and image the material, obtaining high-energy X-ray images and low-energy X-ray images of the material.

[0024] In some optional application scenarios, the dual-energy X-ray identification device includes a dual-energy X-ray generator and a flat panel detector. The dual-energy X-ray generator can emit two levels of X-rays: high-energy and low-energy. The low-energy parameter is preferably set to 80kV, and the high-energy parameter is preferably set to 140kV. These X-rays can penetrate dirt and coatings on the surface of materials. Based on the differences in atomic number of different materials, different X-ray attenuation characteristics are generated. The flat panel detector is used to receive the X-ray attenuation signal after penetrating the material, acquiring image data from the material dimension without being affected by dirt on the material surface. For example, in the scenario of sorting scrap aluminum, when the material on the conveyor belt passes through the detection area of ​​the dual-energy X-ray identification device, the dual-energy X-ray generator sequentially emits 80kV low-energy X-rays and 140kV high-energy X-rays to penetrate scrap aluminum, iron, copper, plastic, and other materials. The flat panel detector simultaneously acquires the attenuation signal after penetration, generating corresponding frames of low-energy transmission images and high-energy transmission images. The two types of images completely preserve the X-ray attenuation difference information of different materials.

[0025] Step S220: Perform global perception processing on the low-energy ray image and the high-energy ray image respectively, and fuse the processing results to obtain a fused feature map.

[0026] To improve material recognition accuracy and reduce feature interference caused by factors such as partial material occlusion, image noise, and surface contamination, global perception processing is performed on both low-energy and high-energy X-ray images. Global perception processing expands the receptive field of feature extraction by performing global feature extraction on both low-energy and high-energy X-ray images separately. This overcomes the limitations of traditional local convolution, which relies solely on fragmented regional information, and fully exploits the overall transmission distribution characteristics of the material within the entire image. This yields separate global feature results for the low-energy and high-energy images. Global perception processing can be implemented using methods including, but not limited to, dilated convolution and large-size convolution. This effectively expands the receptive field without significantly increasing network computation and parameter count, fully capturing the global attenuation information of the material under X-rays, and improving the stability and robustness of material feature representation.

[0027] Low-energy X-ray images are more sensitive to the material characteristics of lightweight materials, while high-energy X-ray images are better suited for distinguishing high-density, heavy materials. The material information of the two types of images is highly complementary. Therefore, by fusing the processing results of their respective global perception processing, the differentiated material characteristics corresponding to different energy levels of X-rays can be effectively integrated. This overcomes the problem of one-sided and insufficient representation of feature information in single-energy-level images, ultimately resulting in a fused feature map that contains comprehensive material material information, providing reliable feature support for subsequent accurate feature matching and material category determination.

[0028] Step S230: Match the feature vector of the fused feature map with multiple preset dynamic category prototype features respectively to determine the similarity between each dynamic category prototype feature.

[0029] Among these features, multiple dynamic category prototype features correspond one-to-one with multiple candidate materials. Candidate materials can be understood as pre-defined, known materials, and the specific category can be determined according to actual needs. For example, in the scenario of sorting recycled resources, multiple candidate materials can be: metals such as aluminum, copper, and iron, plastics, and electronic waste, etc.

[0030] In this disclosure, the dynamic category prototype feature is a pre-constructed standard feature, obtained by clustering the sample feature vector set corresponding to the candidate material category. It can accurately characterize the typical material characteristics of various known materials. It should be noted that this dynamic category prototype feature is not a fixed parameter, but is continuously updated based on the identified feature vectors corresponding to the same material category as the sorting equipment operates, in order to ensure the accuracy of material identification.

[0031] Since the material corresponding to the current fused feature map is an unknown material, in order to quickly determine the material category, the feature vector of the fused feature map is matched with multiple dynamic category prototype features respectively to determine the similarity between the feature vector and each dynamic category prototype feature. Based on the magnitude of the similarity, the closest material category is determined from multiple candidate categories.

[0032] Step S240: Based on the comparison results of multiple similarities and similarity thresholds, determine the material category of the material.

[0033] The similarity threshold is the minimum similarity required to determine if a material and a candidate material are the same type of material. If the similarity is greater than or equal to the threshold, the feature vector representing the material highly matches the dynamic category prototype feature of the corresponding candidate material, and therefore they can be considered the same material. If the similarity is less than the threshold, the feature vector representing the material differs significantly from the dynamic category prototype feature of the corresponding candidate material, and therefore they can be considered different materials.

[0034] By using similarity threshold comparison to determine material categories, the judgment logic is simpler and clearer, the recognition computation is small and the judgment speed is fast. Thus, while ensuring the accuracy of material identification, it can effectively improve the efficiency of online material identification, enabling the sorting equipment to operate continuously, stably and efficiently, and improving the overall sorting performance of the sorting equipment.

[0035] According to the material identification method provided in this disclosure, by performing global perception processing on dual-energy X-ray images separately, the overall transmission distribution characteristics of the material within the image can be fully explored, reducing the adverse effects of various feature interferences. Fusing the processing results of the global perception processing of the dual-energy X-ray images can compensate for the shortcomings of single-energy-level image feature information being one-sided and insufficient in representation, obtaining a fused feature map containing comprehensive material material information, which helps improve material identification accuracy. Furthermore, determining the material category based on the similarity between the feature vector extracted from the fused feature map and multiple preset dynamic category prototype features makes the material category judgment process simple and efficient, improving material identification efficiency while ensuring recognition accuracy, thereby accelerating the material sorting process and improving the overall sorting performance of the sorting equipment.

[0036] In some embodiments, step S220 above may include the following steps: Step a1: Use a convolution method that expands the receptive field to perform global perception processing on the low-energy ray image and the high-energy ray image respectively, and generate the corresponding low-energy convolution feature map and high-energy convolution feature map. Step a2: Fuse the low-energy convolutional feature map and the high-energy convolutional feature map to obtain the fused feature map.

[0037] Specifically, a convolutional approach that expands the receptive field is used to perform global perception processing on low-energy and high-energy transmission images respectively. This broadens the feature extraction perception range without significantly increasing computational load, overcomes the limitations of local region features, fully mines the deep texture information of materials, reduces interference from local material occlusion, surface stains, and image noise, and completely captures the X-ray attenuation distribution characteristics of materials throughout the entire image. This results in the generation of low-energy and high-energy convolutional feature maps with stronger representational capabilities and higher robustness. The size of the convolutional kernel used for global perception processing can be configured according to requirements; for example, a 3×3 convolutional kernel can be used.

[0038] Because low-energy X-ray imaging is relatively sensitive to the material characteristics of lightweight materials, while high-energy X-ray imaging is more suitable for identifying high-density heavy materials, low-energy convolutional feature maps can improve the accuracy of characterizing the material information of lightweight materials, while high-energy convolutional feature maps can effectively highlight the material differences of heavy materials.

[0039] By fusing low-energy and high-energy convolutional feature maps, we can combine their advantages to achieve feature complementarity, integrate global material features extracted from X-rays at different energy levels, make up for the shortcomings of single convolutional feature maps in terms of limited information and incomplete material representation, and improve the problem of missing features in single-energy-level images. This results in a fused feature map that contains comprehensive and multi-dimensional material information, which can provide reliable feature data support for subsequent material category determination.

[0040] In some examples, step a2 above may include: Step b1: Through a multivariate self-attention mechanism, feature enhancement processing is performed on the low-energy convolutional feature map and the high-energy convolutional feature map respectively to obtain the enhanced low-energy convolutional feature map and the high-energy convolutional feature map. Step b2 involves fusing the enhanced low-energy convolutional feature map and the high-energy convolutional feature map to obtain a fused feature map.

[0041] Specifically, in the actual feature extraction process, some effective features may not be obvious, and there may also be some noise interference, which will affect the quality of subsequent feature fusion. Therefore, in order to ensure that the fused feature map can clearly and effectively represent the material properties of the material in different dimensions, a multivariate self-attention mechanism is used to perform feature enhancement processing on the low-energy convolutional feature map and the high-energy convolutional feature map respectively. During the enhancement processing, the low-energy convolutional feature map and the high-energy convolutional feature map are adaptively weighted pixel by pixel and channel by channel to automatically focus on the key effective features corresponding to the material in the image, weaken the interference of invalid background and noise, and focus on strengthening the subtle differences between different materials. This results in enhanced low-energy convolutional feature maps and high-energy convolutional feature maps with higher feature recognition and stronger anti-interference ability.

[0042] Because the enhanced low-energy convolutional feature map can effectively preserve the fine material features of lightweight materials, and the enhanced high-energy convolutional feature map can highlight the material differences of high-density heavy materials, the feature fusion of the two can fully integrate the fine material features of dual-energy X-ray images, make up for the shortcomings of single feature map representation dimension and insufficient feature discrimination, and thus obtain a high-precision and highly robust fused feature map.

[0043] In other examples, the low-energy and high-energy convolutional feature maps, after multivariate self-attention enhancement, maintain consistency in feature dimension and feature scale. Based on this, the system performs feature fusion operations on the low-energy and high-energy convolutional feature maps to generate the desired fused feature map. It should be noted that feature fusion operations include, but are not limited to: element-wise addition, element-wise multiplication, weighted mixing, channel concatenation, or other complex operations that integrate multidimensional feature information between the low-energy and high-energy convolutional feature maps. Through this fusion mechanism, deep integration of complementary high- and low-energy features can be effectively achieved.

[0044] In some embodiments, the process of determining dynamic category prototype features may include the following steps: Step b1: Collect the standard low-energy transmission image set and standard high-energy transmission image set corresponding to the current candidate material category through the dual-energy X-ray identification device, wherein the standard low-energy transmission image set and the standard high-energy transmission image set correspond one-to-one. Step b2: Perform global perception processing on each set of corresponding standard low-energy transmission images and standard high-energy transmission images, and fuse the processing results to obtain a sample fusion feature map set; Step b3: Extract features from the sample fusion feature map set using the first model to obtain the sample feature vector set; Step b4: Using a clustering algorithm, the sample feature vector set is clustered in the feature space to obtain the dynamic category prototype features of the current candidate material category.

[0045] Specifically, when creating the dynamic category prototype features of the current candidate material category, a dual-energy X-ray identification device is used to image and acquire standard material samples of the same type, simultaneously obtaining the standard low-energy transmission image and standard high-energy transmission image corresponding to each image sample. The standard material samples are pure, impurity-free material samples whose material category is known and belongs to the current candidate material category.

[0046] To fully exploit the image features that characterize the current candidate material category, feature extraction and feature fusion are performed on the standard dual-energy projection image of each standard material sample to obtain a sample fusion feature map for each standard material sample. The generation principle of the sample fusion feature map is the same as that of the fusion feature map generated during online recognition, and will not be elaborated further here.

[0047] The first model encodes the feature maps of each standard material sample's fusion map, extracting sample feature vectors that accurately represent the material properties of each standard material sample. These vectors are then aggregated to form a set of sample feature vectors for the corresponding category. The first model is a pre-trained feature extraction model.

[0048] Because the same material sample is affected by factors such as imaging angle, material posture, and environmental noise, the corresponding sample feature vectors will have a certain distribution deviation in the feature space, and there will also be a small number of abnormal and redundant features. Therefore, clustering algorithms are used to perform cluster analysis on the sample feature vector set to remove discrete and abnormal features, discover the common feature distribution of samples of the same type, and determine the feature centers that can stably and commonly represent the current candidate material category, thereby generating the dynamic category prototype features corresponding to that category. For example, the clustering algorithm can be K-Means clustering or mean-shift clustering, depending on the actual application scenario. K-Means is suitable for scenarios where the number of categories is predetermined, while mean-shift is suitable for scenarios where the number of categories changes dynamically.

[0049] The dynamic category prototype feature construction method provided in this disclosure eliminates the need for massive amounts of labeled samples; it allows for the rapid construction of prototype features for each category using only a small number of standard material samples. This effectively reduces sample collection and model training costs and improves the efficiency of prototype library construction. In some examples, for complex scenarios involving diverse types and varying properties of recycled materials, this method of constructing dynamic category prototype features can effectively enhance the recognition system's adaptability and generalization performance to various materials, providing a stable and reliable category benchmark for subsequent open-set material recognition and accurate sorting.

[0050] In some embodiments, the feature vector is obtained by extracting features from the fused feature map using a second model. The second model and the first model are two branches of a Siamese network, and the second model shares weights and has the same structure as the first model.

[0051] Specifically, in the online recognition stage, the feature vectors of the fused feature map are obtained through feature extraction using a pre-set second model. The second model and the first model are two branches of a Siamese network, sharing weights and having the same structure. Therefore, during model training, only the first model needs to be trained. Furthermore, because the weights of the first and second models are shared, the same feature mapping rules can be used in both the online recognition stage and the offline prototype feature construction stage. This maps the online material to be identified and the offline standard material samples to the same feature space, effectively avoiding feature vector distribution shifts caused by differences in feature extraction rules and ensuring comparability between different feature vectors.

[0052] In some optional application scenarios, the Siamese network can adopt a lightweight network structure consisting of 4 convolutional layers and 2 fully connected layers. For example, both subnetworks of the Siamese network adopt a modular stacking structure of "convolutional layer - batch normalization layer - ReLU activation layer - max pooling layer". The two subnetworks correspond to the network architectures of the first and second models, respectively. The number of channels in the four convolutional layers is set to 32, 64, 128, and 256 respectively, and the back end is connected to two fully connected layers, ultimately outputting a 128-dimensional feature vector. The convolutional kernel parameters and fully connected weight parameters of the two subnetworks are updated synchronously. During the network inference phase, only forward propagation is performed to complete the feature distance calculation, without the need to enable a full classification layer. Similarity measurement based on the shared weights of the Siamese network ensures a unified feature measurement standard. Furthermore, compared to networks for image semantic segmentation and instance segmentation, the lightweight network structure has lower computational overhead and faster inference speed, making it suitable for the high-speed, real-time sorting requirements of industrial production lines.

[0053] In some embodiments, step S230 may include: Step c1: Determine the feature distance between the feature vector and the current dynamic category prototype feature; Step c2: Determine the similarity between the feature distance and the current dynamic category prototype feature.

[0054] Within a unified feature space, the feature distance between the feature vector and the prototype feature of the current dynamic category is calculated using the Euclidean distance algorithm to quantify the differences between the two types of features. The magnitude of the feature distance directly reflects the feature similarity. A larger feature distance indicates a more significant difference between the two features, and a lower degree of matching between the material to be identified and the current candidate category. A smaller feature distance indicates a more similar distribution of the two features, and a higher consistency between the material to be identified and the current candidate category.

[0055] Since feature distance and similarity are negatively correlated, the larger the feature distance, the smaller the corresponding similarity value; conversely, the smaller the feature distance, the larger the corresponding similarity value. Therefore, based on the obtained feature distance, the similarity between the material's feature vector and the current dynamic category prototype features can be quickly determined.

[0056] By transforming abstract high-dimensional feature distances into intuitive and quantifiable similarity metrics, the matching degree of different material categories can be standardized and quantified, enabling subsequent unified judgment based on similarity thresholds and achieving accurate and efficient differentiation of material categories.

[0057] In some embodiments, step S240 may include: Step d1: Compare multiple similarities with a similarity threshold respectively; Step d2: If there is at least one target similarity, then the candidate material category corresponding to the target similarity with the largest value is taken as the material category of the material. Step d3: If no target similarity exists, the material category of the material is determined to be an unknown material.

[0058] Specifically, the similarity between each dynamic category prototype feature and a similarity threshold is compared, and the similarity score greater than or equal to the threshold is taken as the target similarity. If at least one target similarity exists, it indicates that among the multiple dynamic category prototype features, there is a target dynamic category prototype feature that highly matches the feature vector. Since a larger similarity value indicates a higher probability that the two belong to the same material category, to ensure the accuracy of material identification, the candidate material category corresponding to the target similarity with the largest value is taken as the material category. The similarity threshold can be pre-calibrated according to the sorting accuracy requirements and sample conditions, and is not limited here.

[0059] If there is no target similarity, the feature vector representing the material does not match the prototype features of each dynamic category, and the actual material category of the material does not belong to any of the multiple candidate materials. Therefore, the material category of the material is determined to be an unknown material.

[0060] In some examples, if the deviation between the feature vector of the material to be identified and the prototype feature of the target dynamic category is within a preset allowable range, the feature vector can be used as a new sample feature to participate in the clustering operation. The cluster centers corresponding to the prototype feature of the target dynamic category are then iteratively optimized to complete the online update of the prototype feature of the dynamic category. By continuously incorporating online real sample features that meet the conditions for prototype feature optimization, it is possible to adapt to subtle changes in material operating conditions, which helps to improve the accuracy of subsequent material identification.

[0061] In some embodiments, if the material category of the material is determined to be an unknown material, the material identification method may further include: Step g1: Collect image features of unknown materials and cluster them to generate dynamic category prototype features corresponding to the unknown materials; Step g2: Obtain the true category of the unknown material; Step g3: Establish the mapping relationship between the dynamic category prototype features and the real category of the unknown material; Step g4: Based on the mapping relationship, expand the dynamic category prototype features corresponding to the unknown material into multiple dynamic category prototype features, and update the candidate material category.

[0062] Specifically, since the material category is unknown, to enrich the types of candidate materials and enable accurate identification when similar materials are detected later, the true category of the unknown material can be determined through methods such as manual annotation, and a dynamic category prototype feature corresponding to the true category can be constructed. The principle of constructing the dynamic category prototype feature corresponding to the unknown material is similar to the implementation principle of steps b1-b4, and will not be elaborated further here.

[0063] Establish a mapping relationship between the dynamic category prototype features and the real material categories, incorporate the newly added dynamic category prototype features into the existing prototype feature set, and update the candidate material categories simultaneously to achieve incremental expansion of the identification model category library and continuously improve the adaptability of the material identification system.

[0064] In some optional application scenarios, taking the sorting of scrap aluminum as an example, the specific process of material identification of the currently conveyed material by the sorting equipment can be as follows. The conveyor belt of the sorting equipment is equipped with a dual-energy X-ray emitter and a flat panel detector. The dual-energy X-ray emitter uses 80kV low-energy rays and 140kV high-energy rays to complete the imaging.

[0065] A Siamese network consisting of 4 convolutional layers and 2 fully connected layers is pre-built. The Siamese network includes a first model and a second model that shares weights and has the same structure as the first model. Using a small number of images of pure aluminum, copper, and plastic blocks, the first model generates corresponding sample feature vector sets for each, and a clustering algorithm is used to form prototype features for aluminum, copper, and impurities.

[0066] When aluminum alloy material with oily surface contamination passes through the detection area, a dual-energy X-ray identification device acquires a corresponding dual-energy transmission image of the aluminum alloy. Global perception processing is performed on each dual-energy transmission image, and feature fusion is performed on the processing results to obtain a fused feature map. Feature vectors are extracted from the fused feature map using a second model, and these feature vectors are matched with prototype features of aluminum, copper, and impurities to determine the similarity between them and the prototype features of each type.

[0067] If the feature vector has the highest similarity value with the aluminum prototype feature, and this similarity value is higher than a preset similarity threshold, then the current material category is determined to be aluminum. If the feature vector has the highest similarity value with the copper prototype feature, and this similarity value is higher than a similarity threshold, then the current material category is determined to be copper.

[0068] Furthermore, if there is a deviation between the feature vector of the material to be identified and the aluminum prototype features, but the deviation is within a preset allowable range, the sorting equipment identification system can participate in the iterative optimization of the aluminum prototype feature cluster center, complete the online update of the aluminum prototype features, and realize the continuous self-optimization of the identification system.

[0069] By integrating dual-energy X-ray material recognition with a dynamic category prototype online update mechanism, it can achieve accurate sorting of recycled materials under high-speed production line conditions, effectively resisting interference caused by surface stains and morphological changes of materials. At the same time, it has the ability to identify unknown materials and continuously learn, significantly improving the accuracy of material sorting and the robustness of the system.

[0070] Based on the same inventive concept, this disclosure also provides a sorting method applied to sorting equipment. For example... Figure 3 As shown, the sorting method may include the following steps: Step S310: Acquire images of the material using a dual-energy X-ray identification device.

[0071] Step S320: Determine the material category of the material using a material identification method. This material identification method can be any of the material identification methods provided in this disclosure.

[0072] Step S330: Sort materials based on material category.

[0073] The sorting equipment also includes a sorting device. This sorting device is located at the end of the conveying device and can be selected as the actuating component, using a high-speed pneumatic spray valve or a robotic arm, based on the material type, weight, and size. The system generates corresponding control signals based on the material identification results, driving the sorting device to separate different categories of materials into their respective collection boxes.

[0074] For example, in a scrap aluminum sorting scenario, if the material is identified as aluminum, the sorting device can be controlled to not perform the blowing action, allowing the material to fall directly into the aluminum collection box, or the corresponding pneumatic spray valve can be activated to push the material to the aluminum collection box. If the material is identified as copper, plastic, or other impurities, the corresponding pneumatic spray valve in the sorting device is triggered to blow and separate the impurities to the corresponding impurity collection area. If the material is identified as an unknown material, it is transported to the area awaiting manual verification, facilitating subsequent sample collection and category expansion.

[0075] According to the sorting method provided in this disclosure, the sorting of materials using the identification results output by the aforementioned material identification method can provide a reliable basis for automated sorting control, realize the orderly diversion of various materials, thereby effectively improving the material sorting accuracy and automation level, and enhancing the sorting system's adaptability to diverse materials.

[0076] Based on the same inventive concept, this disclosure also provides a sorting device. For example... Figure 4 As shown, the sorting device may include: Conveying device 410 is used for conveying materials; The dual-energy X-ray identification device 420 is used to determine the material category of a material by any of the material identification methods provided in this disclosure; Sorting device 430 is used to sort materials based on material category.

[0077] Based on the same inventive concept, this disclosure also provides a computer-readable storage medium storing a program for executing the material identification method or sorting method of any of the foregoing embodiments.

[0078] This disclosure uses specific terms to describe embodiments of the present disclosure. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of the present disclosure. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics in one or more embodiments of the present disclosure can be appropriately combined.

[0079] In the context of this disclosure, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0080] Similarly, it should be noted that, in order to simplify the description of this disclosure and thus aid in the understanding of one or more embodiments, the foregoing description of embodiments of this disclosure may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this disclosure method does not imply that the subject matter of this disclosure requires more features than the features claimed. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

[0081] The basic concepts have been described above. It is obvious that the above disclosure is merely illustrative and does not constitute a limitation of this disclosure. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this disclosure by those skilled in the art. Such modifications, improvements, and corrections are suggested in this disclosure and therefore remain within the spirit and scope of the embodiments of this disclosure.

Claims

1. A material identification method, characterized in that, The method is applied to a sorting device, which includes a dual-energy X-ray identification device, and includes: The dual-energy X-ray identification device acquires images of the material during the conveying process, obtaining low-energy X-ray images and high-energy X-ray images of the material. Global perception processing is performed on the low-energy ray image and the high-energy ray image respectively, and the processing results are fused to obtain a fused feature map; The feature vector of the fused feature map is matched with multiple preset dynamic category prototype features to determine the similarity between each dynamic category prototype feature. The multiple dynamic category prototype features correspond one-to-one with multiple candidate material categories. The dynamic category prototype features are obtained by clustering based on the sample feature vector set of the corresponding candidate material category. The material category of the material is determined based on the comparison results of multiple similarities and similarity thresholds.

2. The material identification method according to claim 1, characterized in that, The process of performing global perception processing on the low-energy ray image and the high-energy ray image respectively, and fusing the processing results to obtain a fused feature map, includes: The low-energy ray image and the high-energy ray image are processed globally using a convolution method that expands the receptive field, respectively, to generate corresponding low-energy convolution feature maps and high-energy convolution feature maps. The low-energy convolutional feature map and the high-energy convolutional feature map are fused to obtain the fused feature map.

3. The material identification method according to claim 2, characterized in that, The process of fusing the low-energy convolutional feature map and the high-energy convolutional feature map to obtain the fused feature map includes: By employing a multivariate self-attention mechanism, feature enhancement processing is performed on the low-energy convolutional feature map and the high-energy convolutional feature map respectively, resulting in enhanced low-energy convolutional feature maps and high-energy convolutional feature maps. The enhanced low-energy convolutional feature map and the high-energy convolutional feature map are fused to obtain the fused feature map.

4. The material identification method according to claim 1, characterized in that, The process of determining the dynamic category prototype features includes: The dual-energy ray identification device acquires a standard low-energy transmission image set and a standard high-energy transmission image set corresponding to the current candidate material category, wherein the standard low-energy transmission image set and the standard high-energy transmission image set correspond one-to-one. Global perception processing is performed on each set of corresponding standard low-energy transmission images and standard high-energy transmission images, and the processing results are fused to obtain a sample fusion feature map set; The first model is used to extract features from the sample fusion feature map set to obtain a sample feature vector set; By using a clustering algorithm, the sample feature vector set is clustered in the feature space to obtain the dynamic category prototype features of the current candidate material category.

5. The material identification method according to claim 4, characterized in that, The method includes: The second model extracts features from the fused feature map to obtain the feature vector. The second model and the first model are two branches of a Siamese network, and the second model and the first model share weights and have the same structure.

6. The material identification method according to claim 1, characterized in that, The step of matching the feature vector of the fused feature map with multiple preset dynamic category prototype features to determine the similarity between the feature vector and each of the dynamic category prototype features includes: Determine the feature distance between the feature vector and the current dynamic category prototype feature; Based on the feature distance, the similarity between the feature and the current dynamic category prototype feature is determined.

7. The material identification method according to claim 1, characterized in that, The process of determining the material category based on comparison results of multiple similarities and similarity thresholds includes: Each of the aforementioned similarities is compared with the aforementioned similarity threshold; If there is at least one target similarity, then the candidate material category corresponding to the target similarity with the largest value is taken as the material category of the material, wherein the target similarity is a similarity greater than or equal to the similarity threshold; If the target similarity does not exist, the material category of the material is determined to be an unknown material.

8. The material identification method according to claim 7, characterized in that, If the material category of the material is determined to be an unknown material, the method further includes: The image features of the unknown material are collected, and the dynamic category prototype features corresponding to the unknown material are generated by clustering. Obtain the true category of the unknown material; Establish a mapping relationship between the dynamic category prototype features corresponding to the unknown material and the real category; Based on the mapping relationship, the dynamic category prototype features corresponding to the unknown material are expanded to multiple dynamic category prototype features, and the candidate material category is updated.

9. A sorting method, characterized in that, The method is applied to a sorting device, which includes a dual-energy X-ray identification device, and includes: Images of materials are acquired using a dual-energy X-ray identification device; The material category of the material is determined by the material identification method according to any one of claims 1-8; The materials are sorted based on the material category.

10. A sorting device, characterized in that, include: Conveying devices are used to transport materials; A dual-energy X-ray identification device for determining the material category of the material by means of the material identification method according to any one of claims 1-8; A sorting device for sorting the materials based on the material category.

Images