Material sorting method, material sorting device, material sorting equipment and storage medium

Abstract

The present disclosure relates to the field of material sorting, and specifically to a material sorting method, a material sorting device, a material sorting equipment and a storage medium. The material sorting method includes: obtaining an image to be processed of a material to be sorted; inputting the image to be processed into a material sorting model, performing sorting processing on the image to be processed by calling at least one layer of sub-model in the material sorting model, determining and outputting a target sorting result corresponding to the material to be sorted, wherein the material sorting model includes different multi-layer sub-models, and the multi-layer sub-models have the same functions and are independent of each other. The high-level sub-models have larger resource usage and higher sorting accuracy than the low-level sub-models, and the sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms; sorting the material to be sorted according to the target sorting results. The accuracy and reliability of the target sorting results can be guaranteed, thereby improving the efficiency of material sorting and ensuring the performance of the material sorting equipment.

Description

Technical Field

[0001] The present disclosure relates to the field of material sorting, and in particular to a material sorting method, a material sorting device, a material sorting equipment, and a computer-readable storage medium. Background Art

[0002] During the material sorting process, images are captured to detect, identify, and sort the materials being transported. Related technologies utilize a material sorting system deployed on the material sorting equipment to process the image to be processed, determine the corresponding material category in the image, and then perform targeted sorting on the material. The material sorting system includes an algorithm for material sorting. This algorithm can be a traditional material sorting algorithm or one based on an artificial intelligence (AI) model.

[0003] However, if the algorithm is a traditional material sorting algorithm, although its execution efficiency is high, it will be limited by the execution logic designed by the expert system, which will in turn affect the sorting accuracy. However, if the algorithm is based on an AI model framework, although its accuracy is high, its execution efficiency is low and it is limited by the computing power of the equipment. Summary of the Invention

[0004] In order to overcome the problems existing in the related art, an exemplary embodiment of the present disclosure provides a material sorting method, which is applied to material sorting equipment, and the method includes: obtaining a to-be-processed image of the material to be sorted; inputting the to-be-processed image into a material sorting model, and performing sorting processing on the to-be-processed image by calling at least one layer of sub-model in the material sorting model, determining and outputting a target sorting result corresponding to the material to be sorted, wherein the material sorting model includes different multi-layer sub-models, and the multi-layer sub-models have the same functions and are independent of each other. The high-level sub-models have larger resource occupancy and higher sorting accuracy than the low-level sub-models, and the sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms; and sorting the material to be sorted according to the target sorting results.

[0005] In some embodiments, the image to be processed is input into a material sorting model, and the image to be processed is sorted by calling at least one layer of sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted, including: inputting the image to be processed into the material sorting model, and sorting the image to be processed by calling the lowest layer of sub-model in the material sorting model to obtain a first sorting result and a first confidence level corresponding to the first sorting result; in response to the first confidence level being greater than or equal to a first threshold value, the first sorting result is used as the target sorting result corresponding to the material to be sorted; and the target sorting result is output.

[0006] In some embodiments, the image to be processed is input into a material sorting model, and the image to be processed is sorted by calling at least one layer of material sorting sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted. It also includes: in response to the first confidence level being less than a first threshold, calling the second layer sub-model to sort the image to be processed to obtain a second sorting result and a second confidence level corresponding to the second sorting result, wherein the lowest layer sub-model is the sub-model with the highest execution efficiency in the multi-layer sub-model; in response to the second confidence level being greater than or equal to the second threshold, the second sorting result is used as the target sorting result corresponding to the material to be sorted.

[0007] In some embodiments, the material sorting model includes multiple intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models; inputting the image to be processed into the material sorting model, and performing sorting processing on the image to be processed by calling at least one layer sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted, further comprising: in response to the second confidence being less than the second threshold, calling other intermediate layer sub-models layer by layer to perform sorting processing on the image to be processed, and determining the corresponding candidate sorting result and the sorting result confidence; if there is a sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, then using the candidate sorting result as the target sorting result corresponding to the material to be sorted; in response to the completion of calling all intermediate layer sub-models and the absence of a sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, calling the highest layer sub-model to perform sorting processing on the image to be processed, and using the obtained sorting result as the target sorting result, wherein the highest layer sub-model is the sub-model with the largest resource usage and the highest sorting accuracy among the multi-layer sub-models.

[0008] In some embodiments, other intermediate layer sub-models are called layer by layer to perform sorting processing on the processed image, and the corresponding candidate sorting results and the confidence of the sorting results are determined, including: calling the current intermediate layer sub-model to perform sorting processing on the processed image to obtain the candidate sorting results and the confidence of the candidate sorting results; obtaining the confidence of the previous candidate sorting result, which is the sorting result obtained by the previous intermediate layer sub-model for sorting the processed image; determining the sorting result confidence of the candidate sorting result based on the first confidence weight corresponding to the previous intermediate layer sub-model, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate layer sub-model and the confidence of the candidate sorting result.

[0009] In some embodiments, the material sorting model includes multiple intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models; the image to be processed is input into the material sorting model, and the image to be processed is sorted by calling at least one layer sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted, and also includes: in response to the second confidence level being less than the second threshold, each intermediate layer sub-model is called separately to sort the image to be processed to obtain a third sorting result and a corresponding third confidence level corresponding to each intermediate layer sub-model; based on the confidence weight, the corresponding third sorting result and the corresponding third confidence level corresponding to each intermediate layer sub-model, the comprehensive sorting result and the corresponding comprehensive confidence level determined by multiple intermediate layer sub-models are determined; in response to the comprehensive confidence level being greater than or equal to the third threshold, the comprehensive sorting result is used as the target sorting result.

[0010] In some embodiments, the image to be processed is input into a material sorting model, and the image to be processed is sorted by calling at least one layer of sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted. It also includes: in response to the comprehensive confidence being less than a third threshold, calling the highest-level sub-model to sort the image to be processed, and using the obtained sorting result as the target sorting result, wherein the highest-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

[0011] In some embodiments, the image to be processed is input into a material sorting model, and the image to be processed is sorted by calling at least one layer of sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted, including: inputting the image to be processed into the material sorting model; in response to sufficient remaining idle resources, the image to be processed is sorted by calling the highest-level sub-model in the material sorting model, and the obtained sorting result is used as the target sorting result, wherein the highest-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy in the multi-layer sub-model; in response to insufficient remaining idle resources, each sub-model is called separately based on the hierarchical order of each sub-model in the material sorting model to sort the image to be processed, until a sorting result with a confidence level greater than or equal to the corresponding threshold is obtained, and is used as the target sorting result corresponding to the material to be sorted.

[0012] In some embodiments, in the material sorting model, the lowest level sub-model is calculated based on a traditional material sorting algorithm, and other sub-models are different from the lowest level sub-model framework and are trained based on an artificial intelligence algorithm.

[0013] In some embodiments, the method further includes: obtaining actual sorting results of the material to be sorted; and optimizing the material sorting model based on a comparison result between the actual sorting results and the target sorting results.

[0014] In some embodiments, the method further includes: in response to the received version update instruction, performing a hot update process on the model parameters corresponding to the material sorting model to obtain the target material sorting model.

[0015] In the second aspect, the present disclosure also provides a material sorting device, which is applied to material sorting equipment, and the device includes: an acquisition module, which is used to acquire a to-be-processed image of the material to be sorted; a first processing module, which is used to input the to-be-processed image into a material sorting model, and sort the to-be-processed image by calling at least one layer of sub-model in the material sorting model, and determine and output the target sorting result corresponding to the material to be sorted, wherein the material sorting model includes different multi-layer sub-models, and the multi-layer sub-models have the same functions and are independent of each other. The high-level sub-models have larger resource consumption and higher sorting accuracy than the low-level sub-models, and the sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms; the second processing module is used to sort the material to be sorted according to the target sorting results.

[0016] In a third aspect, the present disclosure also provides a material sorting device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the material sorting method provided in any of the above aspects.

[0017] In a fourth aspect, the present disclosure further provides a computer-readable storage medium, which stores the following program, and the program is used to perform the material sorting method provided in any of the above aspects.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0019] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: According to the material sorting method provided by the present disclosure, by calling at least one layer of sub-models in the material sorting model to sort the image to be processed, the accuracy and reliability of the target sorting results can be guaranteed, thereby improving the efficiency of material sorting when sorting the materials to be sorted and ensuring the performance of the material sorting equipment. In addition, the material sorting model includes both sub-models calculated based on traditional material sorting algorithms and sub-models trained based on artificial intelligence algorithms. Each sub-model is independent of each other. Therefore, when performing material sorting, the advantages of traditional algorithms and artificial intelligence can be combined, which can improve the flexibility and execution efficiency of material sorting while ensuring the accuracy of material sorting. It also helps to expand and maintain the material sorting model. Because high-level sub-models consume more resources and have higher sorting accuracy than low-level sub-models, multiple layers of sub-models, especially high-level sub-models, can capture more subtle feature differences, thereby achieving fine sorting of materials, thereby improving the accuracy of material sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present disclosure may be better understood by describing exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0021] Figure 1 is a schematic structural diagram of a material sorting device according to an exemplary embodiment of the present disclosure;

[0022] Figure 2 is a flow chart of a material sorting method according to an exemplary embodiment of the present disclosure;

[0023] Figure 3 is a schematic flow chart of another material sorting method according to an exemplary embodiment of the present disclosure;

[0024] Figure 4 is a schematic flow chart of another material sorting method according to an exemplary embodiment of the present disclosure;

[0025] Figure 5 is a schematic structural block diagram of a material sorting device according to an exemplary embodiment of the present disclosure;

[0026] Figure 6 It is a schematic block diagram of the structure of a material sorting device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The specific embodiments of the present disclosure will be described below. It should be noted that in the specific description of these embodiments, in order to provide a concise description, this specification cannot provide a detailed description of all the features of the actual embodiments. It should be understood that in the actual implementation of any embodiment, just as in the process of any engineering project or design project, in order to achieve the specific goals of the developer and to meet system-related or business-related restrictions, various specific decisions are often made, and this will also change from one embodiment to another. In addition, it is also understandable that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the content disclosed by this disclosure, some design, manufacturing or production changes based on the technical content disclosed by this disclosure are just conventional technical means and should not be understood as the content of this disclosure being insufficient.

[0028] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the usual meanings understood by persons of ordinary skill in the technical field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "one" or "a" do not indicate a quantity limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before "include" or "comprises" cover the elements or objects listed after "include" or "comprises" and their equivalents, and do not exclude other elements or objects. Words such as "connect" or "connected" and similar terms are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0029] Related technologies involve a material sorting system deployed on material sorting equipment to process images, determine the corresponding material categories in the images, and then perform targeted sorting on the materials. The material sorting system includes an algorithm for material sorting. This algorithm can be a traditional material sorting algorithm or one based on an AI model.

[0030] However, if the algorithm is a traditional material sorting algorithm, while efficient, it will be limited by the execution logic of the expert system design, which in turn affects sorting accuracy and has poor generalization. However, if the algorithm is based on an AI model framework, while it has strong generalization, adaptability, and reliability, it will have low execution efficiency and be limited by the computing power of the equipment.

[0031] To solve the above problems, the exemplary embodiments of the present disclosure provide a material sorting method that can be applied to material sorting equipment. Figure 1As shown, the material sorting equipment 100 may include a feeding mechanism 110, a transmission mechanism 120, a detection mechanism 130 and a sorting device 140. The feeding mechanism 110 is used to feed the material to be sorted into the transmission mechanism 120. The transmission mechanism 120 may be a structure such as a conveyor belt or a chute, which is used to transport the material to be sorted fed by the feeding mechanism 110. The detection mechanism 130 is used to detect the material transported on the transmission mechanism 120 to detect whether the material is the material to be rejected; the material to be rejected refers to the material to be separated by the sorting equipment. The material to be rejected can be the required material or the unrequired material, as long as the material can be sorted. The sorting device 140 is used to reject the material to be rejected. The material sorting equipment 100 can be used for sorting tasks such as ore sorting, plastic bottle sorting or other material sorting, depending on the actual application requirements.

[0032] like Figure 2 As shown, the material sorting method may include:

[0033] Step S210: obtaining an image of the material to be sorted to be processed.

[0034] The image to be processed can be understood as an image containing the material to be sorted. This image to be processed can be acquired during the material transport process by the transport mechanism of the material sorting equipment. By acquiring this image to be processed, the material category in the image to be processed can be specifically identified and determined, thereby improving the efficiency of material sorting during subsequent material sorting by the sorting mechanism of the material sorting equipment.

[0035] Step S220: input the image to be processed into the material sorting model, perform sorting processing on the image to be processed by calling at least one layer of sub-model in the material sorting model, and determine and output the target sorting result corresponding to the material to be sorted.

[0036] A material sorting model can be understood as a model pre-deployed in the material sorting equipment and used for material sorting. The material sorting model can include different multi-layer sub-models, each with the same independent functionality. Higher-level sub-models utilize more resources and offer higher sorting accuracy than lower-level sub-models. Sub-models can be calculated based on traditional material sorting algorithms or trained using artificial intelligence algorithms. Specifically, the material sorting model includes multiple sub-models for performing material sorting, distributed across different hierarchies. During the actual material sorting process, each sub-model can independently perform material sorting. The higher the hierarchical level of a sub-model, the greater the resource usage and the higher the sorting accuracy. Within the material sorting model, some sub-models can be calculated based on traditional material sorting algorithms, while others can be trained using artificial intelligence algorithms. Traditional material sorting algorithms include, but are not limited to, those based on feature extraction algorithms, optical sorting algorithms, or other algorithms.

[0037] In order to determine the target sorting result corresponding to the material to be sorted in the image to be processed, after obtaining the image to be processed, the image to be processed is input into the material sorting model, and the image to be processed is sorted by calling at least one layer of sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted through at least one layer of sub-model. After obtaining the target sorting result corresponding to the material to be sorted, the target sorting result is output so that the material to be sorted can be subsequently sorted according to the target sorting result.

[0038] In some examples, at least one layer of sub-models can be called to perform sorting processing on the image to be processed based on the hierarchical structure of the sub-models in the material sorting model. For example, the image to be processed can be processed first by calling the lowest layer of sub-models. If the confidence level of the output of the lowest layer of sub-models is high, the sorting result output by the lowest layer of sub-models can be used as the target sorting result. At this time, only one layer of sub-models in the material sorting model is called to determine the target sorting result. If the confidence level of the output of the lowest layer of sub-models is high, other layers of sub-models are continued to be called to process the image to be processed until the target sorting result is determined. At this time, it is necessary to call at least two layers of sub-models in the material sorting model to determine the target sorting result.

[0039] In other examples, at least one sub-model layer can be invoked to sort the image to be processed based on the remaining idle resources of the material sorting equipment. For example, if the remaining idle resources are sufficient, the highest-level sub-model in the material sorting model can be used to process the image to obtain the most accurate target analysis results. If the remaining idle resources are insufficient, at least one sub-model at a lower level can be invoked to process the image to obtain the desired target sorting result.

[0040] Step S230: sorting the materials to be sorted according to the target sorting result.

[0041] The target sorting results can be used to determine the category of the material to be sorted, and then sort the material according to that category to meet the material sorting requirements. For example, if the material to be sorted is ore, the target sorting results can be used to determine whether the ore is concentrate or waste, and then perform targeted sorting on the ore. For another example, if the material to be sorted is plastic bottles, the target sorting results can be used to determine whether the plastic bottles are conservable or discarded, and then perform targeted sorting on the plastic bottles.

[0042] According to the material sorting method provided by the present disclosure, by calling at least one layer of sub-model in the material sorting model to sort the image to be processed, the accuracy and reliability of the target sorting result can be guaranteed, and then when sorting the materials to be sorted, the efficiency of material sorting can be improved, and the performance of the material sorting equipment can be guaranteed. In addition, the material sorting model includes both sub-models calculated based on traditional material sorting algorithms and sub-models trained based on artificial intelligence algorithms. Each sub-model is independent of each other, and then when performing material sorting, the advantages of traditional algorithms and artificial intelligence can be combined, which can improve the flexibility and execution efficiency of material sorting while ensuring the accuracy of material sorting, and also help to expand and maintain the material sorting model. Since high-level sub-models have a larger resource usage and higher sorting accuracy than low-level sub-models, multiple layers of sub-models, especially high-level sub-models, can capture more subtle feature differences, thereby achieving fine sorting of materials, thereby improving the accuracy of material sorting.

[0043] In some embodiments, as Figure 3 As shown, the above step S220 may include:

[0044] In step S221 , the image to be processed is input into the material sorting model, and the image to be processed is sorted by calling the lowest level sub-model in the material sorting model to obtain a first sorting result and a first confidence level corresponding to the first sorting result.

[0045] In the material sorting model, the lower the sub-model level, the more efficient the model is. In other words, the lower the latency is when performing the sorting process. Therefore, the lowest sub-model is the most efficient sub-model in the multi-layer sub-model.

[0046] After obtaining the image to be processed, it is input into the material sorting model. The lowest-level sub-model in the material sorting model is preferentially called to sort the image to be processed to improve sorting efficiency, thereby obtaining a first sorting result and a first confidence level corresponding to the first sorting result. The first sorting result can be understood as the sorting result determined by the lowest-level sub-model after sorting the image to be processed.

[0047] Step S222: In response to the first confidence level being greater than or equal to the first threshold, the first sorting result is used as the target sorting result corresponding to the material to be sorted.

[0048] The first threshold can be understood as the minimum confidence value used to measure whether the sorting result output by the lowest-level sub-model is reliable. For example, the first threshold can be 0.95. The specific value can be determined according to needs and is not limited here.

[0049] In response to the first confidence being greater than or equal to the first threshold, it indicates that the sorting result of sorting the image to be processed by the lowest level sub-model is reliable. Therefore, the first sorting result can be used as the target sorting result corresponding to the material to be sorted.

[0050] Step S223: output the target sorting result.

[0051] Output the target sorting results to clarify the sorting type corresponding to the material to be sorted, so as to facilitate subsequent targeted sorting.

[0052] By determining the target sorting result according to the above method and preferentially calling the lowest-level sub-model to classify the image to be processed, the efficiency of determining the target sorting result can be improved as much as possible and the delay can be reduced, thereby helping to speed up the efficiency of subsequent sorting of the materials to be sorted and improve the performance of the material sorting equipment.

[0053] In other embodiments, Figure 3 As shown, the above step S220 may further include:

[0054] In step S224 , in response to the first confidence being less than the first threshold, the second layer sub-model is called to perform sorting processing on the image to be processed to obtain a second sorting result and a second confidence corresponding to the second sorting result.

[0055] In response to the first confidence level being less than the first threshold, the result of sorting the image to be processed by the lowest-level sub-model is unreliable, and the first sorting result is invalid. Therefore, to determine the target sorting result corresponding to the material to be sorted, the second-level sub-model is called to sort the image to be processed, so as to determine the sorting category corresponding to the material to be sorted by the second-level sub-model, thereby obtaining a second sorting result and a second confidence level corresponding to the second sorting result. The second-level sub-model is at a higher level than the lowest-level sub-model. Therefore, the second-level sub-model consumes more resources and has a higher sorting accuracy than the lowest-level sub-model, but it has a certain delay compared to the lowest-level sub-model. The second sorting result can be understood as the sorting result identified and determined by the second-level sub-model after sorting the image to be processed.

[0056] Step S225: In response to the second confidence level being greater than or equal to the second threshold, the second sorting result is used as the target sorting result corresponding to the material to be sorted.

[0057] The second threshold can be understood as the minimum confidence level used to determine whether the sorting results output by the second-layer sub-model are reliable. For example, the second threshold can be 0.95. The specific value can be determined based on needs, and the second threshold can be the same as or different from the first threshold, and is not limited here.

[0058] In response to the second confidence being greater than or equal to the second threshold, it indicates that the sorting result of sorting the image to be processed by the second layer sub-model is reliable. Therefore, the second sorting result can be used as the target sorting result corresponding to the material to be sorted.

[0059] By determining the target sorting results according to the above method, when the sorting results output by the lowest-level sub-model are unreliable, sub-models at other levels can continue to be called to perform sorting processing on the image to be processed. This can meet the processing requirements for determining the target sorting results and can ensure the accuracy and reliability of the final target sorting results.

[0060] In some other embodiments, the material sorting model includes multiple intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models, such as Figure 3 As shown, the above step S220 may further include:

[0061] Step S226 , in response to the second confidence being less than the second threshold, calling other intermediate layer sub-models layer by layer to perform sorting processing on the image to be processed, and determining corresponding candidate sorting results and sorting result confidences.

[0062] In response to the second confidence level being less than the second threshold, the result of sorting the image to be processed by the second-layer sub-model is unreliable, and the second sorting result is invalid. Therefore, to determine the target sorting result corresponding to the material to be sorted, the other intermediate-layer sub-models are called layer by layer to sort the image to be processed, and the candidate sorting results and corresponding sorting result confidence levels obtained by each called intermediate-layer sub-model after sorting the image to be processed are determined.

[0063] In some examples, step S226 may include the following steps:

[0064] Step a1: call the current middle layer sub-model to perform sorting processing on the image to be processed, and obtain candidate sorting results and the confidence of the candidate sorting results;

[0065] Step a2, obtaining the confidence level of the previous candidate sorting result;

[0066] Step a3: Determine the sorting result confidence of the candidate sorting result based on the first confidence weight corresponding to the previous intermediate layer submodel, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate layer submodel, and the confidence of the candidate sorting result.

[0067] Specifically, for the current middle layer sub-model, the current middle layer sub-model is called to perform sorting processing on the image to be processed to obtain candidate sorting results and confidence levels of the candidate sorting results.

[0068] Because the current intermediate-layer submodel is located above the previous intermediate-layer submodel, it can achieve relatively better sorting efficiency and results than the previous intermediate-layer submodel. However, when actually calling submodels, the previous intermediate-layer submodel is called first. The current intermediate-layer submodel is only called for sorting when the confidence level of the candidate sorting result determined by the previous intermediate-layer submodel is lower than the confidence threshold of the corresponding intermediate-layer submodel.

[0069] Therefore, in order to make the result output by the current intermediate layer sub-model more reliable, the confidence of the previous candidate sorting result is obtained, wherein the previous candidate sorting result is the sorting result obtained by the previous intermediate layer sub-model by sorting the image to be processed.

[0070] The sorting result confidence of the candidate sorting result corresponding to the current intermediate submodel is determined based on the first confidence weight corresponding to the previous intermediate submodel, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate submodel, and the confidence of the candidate sorting result. This allows the sorting result confidence to be determined by reference to the sorting result determination of the previous intermediate submodel, thereby making the resulting sorting result confidence more reliable and helping to improve the robustness and decision quality of the material sorting model. The sum of the first confidence weight corresponding to the previous intermediate submodel and the second confidence weight corresponding to the current intermediate submodel is 1. The specific weight distribution can be determined based on the reliability of the actual model and is not limited here.

[0071] In some examples, the first confidence weight corresponding to the previous intermediate sub-model, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate sub-model, and the confidence of the candidate sorting result can be fused together through weighted averaging or a more complex fusion function to obtain a sorting result confidence that can represent the candidate sorting result. For example, taking weighted averaging as an example, if the first confidence weight corresponding to the previous intermediate sub-model is 0.7 and the first confidence weight of the current intermediate sub-model is 0.3, then the sorting result confidence of the candidate sorting result = the confidence of the previous candidate sorting result * 0.7 + the confidence of the candidate sorting result * 0.3.

[0072] In other examples, the confidence level of the candidate sorting result determined by the current layer sub-model can be directly used as the sorting result confidence level corresponding to the candidate sorting result, thereby helping to improve the determination efficiency and facilitate rapid determination of the target sorting result.

[0073] Step S227: If there is a sorting result confidence that is greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, the candidate sorting result is used as the target sorting result corresponding to the material to be sorted.

[0074] If, during the layer-by-layer calling process, there is a sorting result confidence that is greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, it indicates that the candidate sorting result detected by the intermediate layer sub-model is reliable. Therefore, the candidate sorting result detected by the intermediate layer sub-model can be directly used as the target sorting result corresponding to the material to be sorted, without having to continue calling the next intermediate layer sub-model for processing, which helps to avoid waste of resources.

[0075] In step S228, if all intermediate layer sub-models have been called and there is no sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, the highest layer sub-model is called to perform sorting on the image to be processed, and the obtained sorting result is used as the target sorting result.

[0076] If, after the highest intermediate sub-model is called, no sorting result confidence level is greater than or equal to the confidence threshold of the corresponding intermediate sub-model, then none of the multiple intermediate sub-models in the material sorting model are suitable for sorting the image to be sorted. Therefore, the highest sub-model is called to sort the image to be processed, and the resulting sorting result is used as the target sorting result to ensure the accuracy of the target sorting result. The highest sub-model is the sub-model with the largest resource usage and the highest sorting accuracy among the multiple sub-models. The sorting result determined by the highest sub-model can be assumed to be the most accurate and reliable by default.

[0077] By determining the target sorting result in the above manner, the image to be processed can be sorted by calling the intermediate layer sub-model layer by layer, which can improve the flexibility and execution efficiency of material sorting while ensuring the accuracy of material sorting. Moreover, since the high-level sub-model can capture more subtle feature differences, it can ensure that the image to be processed can be sorted more finely, thereby making the final target sorting result more accurate and reliable.

[0078] In some further embodiments, the material sorting model includes a plurality of intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the plurality of intermediate layer sub-models, such as Figure 3 As shown, the above step S220 may further include:

[0079] In step S229, in response to the second confidence being less than the second threshold, each intermediate layer sub-model is called to perform sorting processing on the image to be processed, and a third sorting result and a corresponding third confidence corresponding to each intermediate layer sub-model are obtained.

[0080] In response to the second confidence being less than the second threshold, it is indicated that the result of sorting the image to be processed by the second layer sub-model is unreliable, and the second sorting result is an invalid result.

[0081] In the material sorting model, the execution efficiency corresponding to the sub-models at the same level is relatively close. However, since the highest-level sub-model is the sub-model with the highest accuracy, its execution efficiency is the lowest compared to other sub-models. Therefore, in order to ensure the efficiency of determining the target sorting results and reduce the delay as much as possible, each intermediate-level sub-model is called separately to perform sorting processing on the image to be processed, and the third sorting result and the corresponding third confidence level corresponding to each intermediate-level sub-model are obtained, so that the target sorting result can be determined based on all the third sorting results and the corresponding third confidence levels in the future.

[0082] Step S2210: Based on the confidence weight corresponding to each intermediate layer sub-model, the corresponding third sorting result and the corresponding third confidence, determine the comprehensive sorting result determined by multiple intermediate layer sub-models and the corresponding comprehensive confidence.

[0083] The comprehensive performance of multiple intermediate-layer sub-models is relatively similar. Therefore, to ensure the reliability of the sorting results determined by calling the intermediate-layer sub-models and to eliminate recognition anomalies caused by malfunctions in some intermediate-layer sub-models during image processing, a weighted summation is performed based on the confidence weight, third sorting result, and third confidence corresponding to each intermediate-layer sub-model. The combined sorting result and the corresponding comprehensive confidence are then determined. This comprehensive sorting result represents the sorting results corresponding to the multiple intermediate-layer sub-models. For example, consider two intermediate-layer sub-models, including a second-layer sub-model and a third-layer sub-model. The third confidence weight corresponding to the second-layer sub-model is 0.7, and the third confidence weight corresponding to the third-layer sub-model is 0.3. Therefore, the comprehensive confidence = the third confidence weight corresponding to the second-layer sub-model * 0.7 + the third confidence weight corresponding to the third-layer sub-model * 0.3.

[0084] Step S2211: In response to the comprehensive confidence being greater than or equal to the third threshold, the comprehensive sorting result is used as the target sorting result.

[0085] The third threshold can be understood as a minimum confidence value used to measure whether the sorting results output by the multiple intermediate layer sub-models are reliable. For example, the third threshold can be 0.8. The specific value can be determined according to needs and is not limited here.

[0086] In response to the comprehensive confidence being greater than or equal to the third threshold, it indicates that the sorting result obtained by sorting the image to be processed through multiple intermediate layer sub-models is reliable. Therefore, the comprehensive sorting result can be used as the target sorting result corresponding to the material to be sorted.

[0087] Determining the target sorting result according to the above method can reduce the delay as much as possible and improve the efficiency of determining the target sorting result, thereby helping to promote the execution process of material sorting.

[0088] In some other embodiments, such as Figure 3 As shown, the above step S220 may further include:

[0089] Step S2212: In response to the comprehensive confidence being less than the third threshold, the highest level sub-model is called to perform sorting processing on the image to be processed, and the obtained sorting result is used as the target sorting result.

[0090] In response to the combined confidence level being less than the third threshold, it indicates that the sorting results obtained by processing the image to be processed by multiple intermediate sub-models are still unreliable. Therefore, the highest-level sub-model is invoked to sort the image to be processed, and the resulting sorting result is used as the target sorting result to ensure the accuracy of the target sorting result. The highest-level sub-model is the sub-model with the largest resource usage and the highest sorting accuracy among the multiple sub-models. The sorting result determined by the highest-level sub-model can be assumed to be the most accurate and reliable by default.

[0091] Determining the target sorting result according to the above method can ensure that the target sorting result can be output smoothly, the material sorting task can be carried out in an orderly manner, and the materials to be sorted can be sorted normally.

[0092] In some other embodiments, in order to save the remaining idle resources, some of the lowest intermediate layer sub-models can be called first to jointly determine the comprehensive sorting results and the corresponding comprehensive confidence. If the comprehensive confidence at this time is greater than or equal to the third confidence, the comprehensive sorting results jointly determined by some of the lowest intermediate layer sub-models are directly used as the target sorting results. If the comprehensive confidence at this time is less than the third confidence, other intermediate layer sub-models are called to jointly determine the comprehensive sorting results and the corresponding comprehensive confidence, so as to reduce the probability of the highest layer sub-model being called as much as possible. In this case, the confidence weight corresponding to each intermediate layer sub-model is determined based on the corresponding performance index and the number of intermediate layer sub-models called, and can be dynamically adjusted according to the actual calling situation to ensure that the value range corresponding to the determined comprehensive confidence is in [0,1].

[0093] In some application scenarios, at least one or more of the first confidence level, the second confidence level, and the third confidence level can be determined by random forest, gradient boosting tree, entropy, or Bayesian error estimation, and the specific determination method can be determined based on the characteristics and requirements of the corresponding sub-model.

[0094] In some application scenarios, a distributed stream processing platform (Apache Kafka) can be used to control the invocation of material sorting models or switch sub-models to sort images. This distributed stream processing platform can support decision trees or reinforcement learning (DQN) to generate management rules for invoking or switching sub-models, reducing manual intervention and enhancing the flexibility of sub-model invocation, thereby providing more intelligent and efficient model management and decision support for material sorting models.

[0095] In some embodiments, as Figure 4 As shown, the above step S220 may further include:

[0096] Step S2211: input the image to be processed into the material sorting model.

[0097] Step S2212: In response to the fact that the remaining amount of idle resources is sufficient, the image to be processed is sorted by calling the highest-level sub-model in the material sorting model, and the obtained sorting result is used as the target sorting result.

[0098] In response to the sufficient remaining idle resources, it indicates that the sorting process for the pending image is not limited by the inability to determine the target sorting result due to insufficient resources. Therefore, the highest-level submodel in the material sorting model can be directly called to sort the pending image, so that the highest-level submodel can fully utilize the remaining idle resources to obtain accurate and reliable target sorting results, thereby ensuring the efficiency and accuracy of subsequent material sorting. The highest-level submodel is the submodel with the largest resource usage and the highest sorting accuracy among the multi-level submodels.

[0099] In some application scenarios, Prometheus (an open source monitoring tool) can be used to monitor the remaining amount of idle resources to ensure timely determination of the remaining idle resources.

[0100] In step S2213, in response to the insufficient remaining amount of idle resources, each sub-model is called to perform sorting processing on the image to be processed based on the hierarchical order of each sub-model in the material sorting model, until a sorting result with a confidence level greater than or equal to the corresponding threshold is obtained, and it is used as the target sorting result corresponding to the material to be sorted.

[0101] In response to insufficient remaining idle resources, it indicates that if the highest-level sub-model is directly used to sort the image to be processed, it will affect the sorting performance of the material sorting equipment. Therefore, in order to reduce the delay, based on the hierarchical order of each sub-model in the material sorting model, each sub-model is called in turn to sort the image to be processed. If the confidence level corresponding to the sorting result determined by a sub-model is greater than or equal to the sorting result of the corresponding threshold, it indicates that the sorting result determined by the sub-model is reliable, and the sorting result can be used as the target sorting result corresponding to the material to be sorted, so as to save resources and avoid waste of resources.

[0102] In some embodiments, the hierarchical order can be from low to high. Since the lower-level sub-models require relatively less resources, they are preferentially called to sort the image to be processed. This ensures that sufficient resources are available during the sorting process, thereby speeding up the determination of the target sorting results and reducing latency.

[0103] In other embodiments, the hierarchical order may be from high level to low level, thereby ensuring the efficiency of determining the target sorting results as much as possible, reducing the number of times the material sorting model calls the sub-model, and simplifying the management logic of the material sorting model.

[0104] In some examples, the material sorting model can be run by calling any or a combination of the remaining idle resources in the graphics processing unit (GPU) and the central processing unit (CPU), which can be specifically determined according to the hardware configuration requirements of the material sorting equipment and is not limited here. For example, if the GPU configuration requirements determine that the GPU can independently undertake the operation of the material sorting model, then during the operation of the material sorting model, the remaining idle resources on the GPU can be called to support the operation of each sub-model in the material sorting model. If the GPU configuration requirements determine that the GPU cannot independently undertake the operation of the material sorting model, then the remaining idle resources on the GPU and the CPU can be called to jointly support the operation of the material sorting model. If the GPU configuration requirements determine that the GPU cannot undertake the operation of the material sorting model, then during the operation of the material sorting model, the remaining idle resources on the CPU can be called to support the operation of each sub-model in the material sorting model.

[0105] In some embodiments, in a material sorting model, the lowest-level sub-model can be calculated based on a traditional material sorting algorithm, while other sub-models have a different framework from the lowest-level sub-model and are trained based on an artificial intelligence algorithm. By deploying the material sorting model in this way, when sorting images to be processed, the sub-model calculated based on the traditional material sorting algorithm can be preferentially used for sorting, so as to determine the target sorting result as quickly as possible, reduce latency, and improve execution efficiency. In addition, by combining traditional material sorting algorithms with artificial intelligence algorithms and deploying multiple sub-models, the advantages of traditional material sorting algorithms and artificial intelligence algorithms can be fully utilized to achieve more efficient and accurate material sorting.

[0106] In some examples, a material sorting model can include three layers of sub-models, each tier being divided based on efficiency (latency, resource usage) and effectiveness (accuracy, recall). For example, the lowest-level sub-model is the most efficient, but has lower effectiveness than the other sub-models. The highest-level sub-model is the least efficient, but has the best performance. The number of sub-models in the middle tier can be greater than or equal to one. This number can be determined based on the task requirements or deployment requirements.

[0107] In some application scenarios, assuming the number of intermediate sub-models is two, the hierarchical architecture of a material sorting model might include: a lowest-level sub-model calculated based on a traditional material sorting algorithm; a second-level sub-model, a lightweight model trained using an AI algorithm (e.g., a model trained using the MobileNet framework); a third-level sub-model, a balanced model trained using an AI algorithm (e.g., a model trained using the ResNet-50 framework); and a top-level sub-model, a high-precision baseline model trained using an AI algorithm (e.g., a model trained using the ResNet-152 framework). The second and third-level sub-models are intermediate-level sub-models. The sub-models in the material sorting model are trained using different methods. For example, the lowest-level sub-model might be trained using feature extraction and analysis based on prior knowledge provided by an expert system. Lightweight models can employ knowledge distillation, transferring knowledge learned from a large-scale teacher model to a smaller-scale student model for training, thereby obtaining the desired sub-model. A balanced model can be trained using data augmentation (for example, through random masking or MixUp). A high-precision guaranteed model can be trained using the full data set and have the largest model size.

[0108] In other application scenarios, the lowest sub-model of the material sorting model is a sub-model calculated based on a traditional material sorting algorithm, with a latency of less than 1 millisecond and a resource usage of less than 100MB of memory. The order of the second- and third-level sub-models can be determined based on actual processing efficiency and effectiveness. For example, if sub-model a has a latency of less than 50ms and a resource usage of less than 500MB, and sub-model b has a latency between 50-200ms and a resource usage of more than 500MB, then sub-model a is the second-level sub-model and sub-model b is the third-level sub-model. The highest-level sub-model is the one with the highest accuracy but consumes the most resources.

[0109] Note that the order and model types of sub-models in the second and higher layers of the material sorting model are not fixed and can be determined based on the actual model size, task processing efficiency, and task processing results. Material sorting models corresponding to different materials can include different or the same sub-models, depending on the actual training results.

[0110] In some application scenarios, multiple sub-models in a material sorting model are trained independently. Taking one of the sub-models as an example, when training the sub-model, the training sample expressions used for training can be the complexity labels and domain labels of the input data annotated by the front-end rule engine or lightweight model (FastText). Among them, the complexity labels can include: high complexity, medium complexity, or low complexity. The domain label can be determined based on the actual material sorting task. For example, if it is ore sorting, the domain label can include ore identification and classification. If it is plastic bottle sorting, the domain label can include plastic bottle identification and classification.

[0111] When determining the degree of training of a sub-model, a confidence scoring system can be used for this purpose. The confidence scoring system may include, but is not limited to, determining the convergence of the sub-model by performing any one or more of the classification and regression tasks, thereby determining the degree of training of the sub-model. For example, when performing a classification task, the convergence of the sub-model can be determined by calculating the probability entropy of the normalized processing (e.g., processing using the Softmax function) and calibrating the confidence through Platt Scaling (a method of converting the output of a classification model into a class probability distribution). When performing a regression task, the uncertainty can be estimated through the prediction variance or Monte-Carlo Dropout (MC Dropout) to determine the convergence of the sub-model.

[0112] In other application scenarios, APM tools (Datadog) can be deployed in advance to monitor the processing delays of each sub-model in real time. If the delay duration exceeds the delay duration threshold of the corresponding level, the level of the sub-model is adjusted to ensure that the deployment of the sub-model meets the deployment requirements of the material sorting model where multiple sub-models can meet the requirements that high-level sub-models occupy more resources than low-level sub-models and have higher sorting accuracy.

[0113] Step S2214, output the target sorting result.

[0114] In some embodiments, the material sorting method may further include:

[0115] Step b1, obtaining the actual sorting results of the materials to be sorted;

[0116] Step b2: Optimizing the material sorting model based on the comparison result between the actual sorting result and the target sorting result.

[0117] Specifically, to ensure the performance of the material sorting model, after determining the target sorting result output by the material sorting model, the target sorting result can be compared with the actual sorting result of the material to be sorted. If the two are identical, the performance of the material sorting model is good and no optimization is required. However, if there is a difference between the two, the performance of the material sorting model may be abnormal and optimization is required.

[0118] In some application scenarios, in order to avoid over-optimization due to errors, when it is determined that the performance of the material sorting model may be abnormal, the number of abnormalities in the material sorting model is monitored, and when the number is greater than the threshold, the material sorting model is optimized to ensure the performance of the material sorting model.

[0119] In other application scenarios, during the material sorting process, data can be collected based on the processing status of the sub-model, and then the parameters of the sub-model can be optimized in a targeted manner based on the collected data results, so that the sub-model can meet the corresponding hierarchical deployment requirements and ensure the performance of the material sorting model. Among them, data collection can be performed by recording online inference logs so that incremental training data sets can be constructed for the optimization of the sub-model later. The content of the online inference log may include the input, output, confidence level of the sorting results, resource consumption, and image data of the corresponding sub-model. The content of optimizing the parameters of the sub-model may include but is not limited to adjusting the input parameters of the sub-model (for example, the batch size (batchsize) and image size of the training samples used for model optimization, etc.).

[0120] In some embodiments, the material sorting method may further include: in response to a received version update instruction, performing a hot update process on model parameters corresponding to the material sorting model to obtain a target material sorting model, so that the updated target material sorting model can meet the user's usage requirements. The version update instruction may be used to instruct a version rollback to a specified historical version of the material sorting model; alternatively, the version update instruction may be used to instruct a version upgrade to obtain the latest version of the material sorting model.

[0121] In some optional application scenarios, taking the material sorting task as ore sorting as an example, the process of performing material sorting by the material sorting method provided by the present disclosure may be as follows:

[0122] During the ore transmission process, an image of the material to be sorted is obtained and input into a pre-trained material sorting model to determine the target sorting result corresponding to the ore to be sorted. The material sorting model includes different multi-layer sub-models, which have the same functions and are independent of each other. High-level sub-models have higher resource usage and higher sorting accuracy than low-level sub-models. The lowest-level sub-model of the material sorting model is calculated based on traditional material sorting algorithms and has the highest execution efficiency. The other sub-models of the material sorting model are trained based on artificial intelligence algorithms, and the highest-level sub-model is the sub-model with the largest resource usage and the highest sorting accuracy among the multi-layer sub-models.

[0123] In the material sorting model, the lowest-level submodel is preferentially invoked to perform sorting processing on the image to be processed. In response to a first confidence level corresponding to the outputted first sorting result being greater than or equal to a first threshold, the first sorting result is used as a target sorting result corresponding to the ore to be sorted, and the target sorting result is output.

[0124] In response to the first confidence level corresponding to the output first sorting result being less than the first threshold, the second-layer sub-model is invoked to perform sorting processing on the image to be processed, thereby obtaining a second sorting result and a second confidence level corresponding to the second sorting result. In response to the second confidence level being greater than or equal to the second threshold, the second sorting result is used as the target sorting result corresponding to the ore to be sorted.

[0125] In response to the second confidence being less than the second threshold, each intermediate layer sub-model is called to perform sorting processing on the image to be processed, obtaining a third sorting result and a third confidence level corresponding to each intermediate layer sub-model. Furthermore, based on the confidence weight, the third sorting result, and the third confidence level corresponding to each intermediate layer sub-model, a comprehensive sorting result determined by the multiple intermediate layer sub-models and a corresponding comprehensive confidence level are determined. In response to the comprehensive confidence being greater than or equal to the third threshold, the comprehensive sorting result is used as the target sorting result.

[0126] In response to the comprehensive confidence being less than the third threshold, the highest-level sub-model is called to perform sorting processing on the image to be processed, and the obtained sorting result is used as the target sorting result.

[0127] According to the target sorting results, the ore to be sorted is sorted to improve the material sorting efficiency and ensure the sorting performance of the material sorting equipment.

[0128] In other application scenarios, the material sorting model provided by the present invention can achieve a balance between efficiency and effect. By combining dynamic calling strategies and intelligent optimization mechanisms, it can reduce the average inference delay by 30% to 50% under the premise of fixed hardware costs, while ensuring the accuracy of high-difficulty scenarios, thereby effectively improving material sorting efficiency and enhancing the sorting performance of material sorting equipment.

[0129] Based on the same inventive concept, the present disclosure also provides a material sorting device applied to material sorting equipment. Figure 5 As shown, the material sorting device 300 may include:

[0130] An acquisition module 310 is used to acquire an image of the material to be sorted.

[0131] A first processing module 320 is configured to input an image to be processed into a material sorting model, perform sorting processing on the image by invoking at least one sub-model in the material sorting model, and determine and output a target sorting result corresponding to the material to be sorted. The material sorting model includes different multi-layer sub-models, each of which has the same functions and is independent of each other. Higher-level sub-models consume more resources and have higher sorting accuracy than lower-level sub-models. The sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms.

[0132] The second processing module 330 is used to sort the materials to be sorted according to the target sorting result.

[0133] In some embodiments, the first processing module 320 may include: a first processing unit, used to input the image to be processed into the material sorting model, and perform sorting processing on the image to be processed by calling the lowest-level sub-model in the material sorting model to obtain a first sorting result and a first confidence level corresponding to the first sorting result, wherein the lowest-level sub-model is the sub-model with the highest execution efficiency in the multi-layer sub-model; a second processing unit, used to use the first sorting result as the target sorting result corresponding to the material to be sorted in response to the first confidence level being greater than or equal to a first threshold; and an output unit, used to output the target sorting result.

[0134] In some embodiments, the first processing module 320 may further include: a third processing unit for, in response to the first confidence being less than the first threshold, calling the second layer sub-model to perform sorting processing on the image to be processed, to obtain a second sorting result and a second confidence corresponding to the second sorting result; and a fourth processing unit for, in response to the second confidence being greater than or equal to the second threshold, using the second sorting result as the target sorting result corresponding to the material to be sorted.

[0135] In some embodiments, the material sorting model includes multiple intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models. The first processing module 320 may further include: a fifth processing unit, for calling other intermediate layer sub-models layer by layer to perform sorting processing on the image to be processed in response to the second confidence being less than the second threshold, and determining the corresponding candidate sorting results and the confidence of the sorting results; a first execution unit, for using the candidate sorting result as the target sorting result corresponding to the material to be sorted if there is a sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model; a second execution unit, for calling the highest layer sub-model to perform sorting processing on the image to be processed in response to the completion of calling all intermediate layer sub-models and the absence of a sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, and using the obtained sorting result as the target sorting result, wherein the highest layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

[0136] In some embodiments, the fifth processing unit may include: a control unit, used to call the current intermediate layer sub-model to perform sorting processing on the processed image to obtain candidate sorting results and the confidence of the candidate sorting results; an acquisition unit, used to obtain the confidence of the previous candidate sorting result, and the previous candidate sorting result is the sorting result obtained by the previous intermediate layer sub-model to perform sorting processing on the processed image; a confidence determination unit, used to determine the sorting result confidence of the candidate sorting result based on the first confidence weight corresponding to the previous intermediate layer sub-model, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate layer sub-model and the confidence of the candidate sorting result.

[0137] In some embodiments, the material sorting model includes multiple intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models; the first processing module 320 may also include: a sixth processing unit for calling each intermediate layer sub-model to perform sorting processing on the image to be processed in response to the second confidence being less than the second threshold, and obtaining a third sorting result and a corresponding third confidence corresponding to each intermediate layer sub-model; a seventh processing unit for determining the comprehensive sorting result and the corresponding comprehensive confidence determined by the multiple intermediate layer sub-models based on the confidence weight, the corresponding third sorting result and the corresponding third confidence corresponding to each intermediate layer sub-model; an eighth processing unit for taking the comprehensive sorting result as the target sorting result in response to the comprehensive confidence being greater than or equal to the third threshold.

[0138] In some embodiments, the first processing module 320 may also include: a ninth processing unit for calling the highest-level sub-model to perform sorting processing on the image to be processed in response to the comprehensive confidence being less than the third threshold, and using the obtained sorting result as the target sorting result, wherein the highest-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

[0139] In some embodiments, the first processing module 320 may include: an input unit for inputting the image to be processed into the material sorting model; a first calling unit for, in response to sufficient remaining idle resources, calling the highest-level sub-model in the material sorting model to sort the image to be processed, and using the obtained sorting result as the target sorting result, wherein the highest-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy in the multi-layer sub-model; a second calling unit for, in response to insufficient remaining idle resources, calling each sub-model to sort the image to be processed based on the hierarchical order of each sub-model in the material sorting model, until a sorting result with a confidence level greater than or equal to the corresponding threshold is obtained, and used as the target sorting result corresponding to the material to be sorted.

[0140] In some embodiments, in the material sorting model, the lowest level sub-model is calculated based on a traditional material sorting algorithm, and other sub-models are different from the lowest level sub-model framework and are trained based on an artificial intelligence algorithm.

[0141] In some embodiments, the material sorting device 300 may further include: an acquisition module for acquiring actual sorting results of the material to be sorted; and an optimization module for optimizing the material sorting model based on a comparison result between the actual sorting results and the target sorting results.

[0142] In some embodiments, the material sorting device 300 may further include: an update module, configured to perform a hot update process on the model parameters corresponding to the material sorting model in response to a received version update instruction, to obtain a target material sorting model.

[0143] Regarding the image detection device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0144] Based on the same inventive concept, an embodiment of the present disclosure provides a material sorting device. Figure 6As shown, the material sorting device includes: one or more processors 410, a memory 420, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the material sorting device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple material sorting devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 410 is taken as an example.

[0145] Processor 410 may be a central processing unit, a network processor, or a combination thereof. Processor 410 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0146] The memory 420 stores instructions that can be executed by at least one processor 410, so as to enable at least one processor 410 to implement the material sorting method shown in the above embodiment.

[0147] The memory 420 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the material sorting device, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 420 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the material sorting device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The memory 420 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 420 may also include a combination of the above types of memory.

[0149] The material sorting device further includes an input device 430 and an output device 440. The processor 410, the memory 420, the input device 430 and the output device 440 can be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0150] Input device 430 can receive input numeric or character information and generate key input signals related to user settings and function control of the material sorting equipment. Examples include a touch screen, keypad, mouse, trackpad, touchpad, indicator stick, one or more mouse buttons, trackball, joystick, etc. Output device 440 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). Such display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, monitors, and plasma displays. In some optional embodiments, the display device may be a touch screen.

[0151] Based on the same inventive concept, the present disclosure further provides a computer-readable storage medium, which stores the following program, which is used to execute the material sorting method of any of the aforementioned embodiments.

[0152] This disclosure uses specific terms to describe the embodiments of the present disclosure. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic associated with at least one embodiment of the present disclosure. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or more in different places in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of the present disclosure may be appropriately combined.

[0153] In the context of this disclosure, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0154] Similarly, it should be noted that, in order to simplify the presentation of this disclosure and thereby facilitate understanding of one or more application embodiments, the foregoing descriptions of the embodiments of this disclosure sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not mean that the disclosed subject matter requires more features than the claimed features. In fact, the features of an embodiment may be fewer than all the features of a single disclosed embodiment.

[0155] The basic concepts have been described above. It will be apparent to those skilled in the art that the above disclosure is merely illustrative and does not constitute a limitation of the present disclosure. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and revisions to the present disclosure. Such modifications, improvements, and revisions are suggested in the present disclosure and remain within the spirit and scope of the embodiments of the present disclosure.

Claims

1. A material sorting method, characterized in that: Applied to material sorting equipment, the method includes: Acquire the image of the material to be sorted; Input the image to be processed into a material sorting model, sort the image to be processed by calling at least one layer of sub-model in the material sorting model, and determine and output a target sorting result corresponding to the material to be sorted, including: based on the hierarchical order of each sub-model in the material sorting model, respectively calling each sub-model to sort the image to be processed until a sorting result with a confidence level greater than or equal to a corresponding threshold is obtained, and the result is used as the target sorting result corresponding to the material to be sorted, wherein the material sorting model includes different multi-layer sub-models, the multi-layer sub-models have the same functions and are independent of each other, the high-level sub-models have larger resource usage and higher sorting accuracy than the low-level sub-models, the lowest-level sub-model is the sub-model with the highest execution efficiency among the multi-layer sub-models, the sub-models are calculated based on a traditional material sorting algorithm or trained based on an artificial intelligence algorithm, in the material sorting model, the lowest-level sub-model is calculated based on a traditional material sorting algorithm, and the other sub-models have a different framework from the lowest-level sub-model and are trained based on an artificial intelligence algorithm; The materials to be sorted are sorted according to the target sorting result.

2. The material sorting method according to claim 1, characterized in that: Inputting the image to be processed into the material sorting model, performing sorting processing on the image to be processed by calling at least one sub-model in the material sorting model, and determining a target sorting result corresponding to the material to be sorted, includes: Inputting the image to be processed into the material sorting model, performing sorting processing on the image to be processed by calling the lowest level sub-model in the material sorting model, and obtaining a first sorting result and a first confidence level corresponding to the first sorting result; In response to the first confidence being greater than or equal to a first threshold, taking the first sorting result as the target sorting result corresponding to the material to be sorted; Output the target sorting result.

3. The material sorting method according to claim 2, characterized in that: The step of inputting the image to be processed into a material sorting model, performing sorting processing on the image to be processed by calling at least one sub-model in the material sorting model, and determining a target sorting result corresponding to the material to be sorted further includes: In response to the first confidence level being less than the first threshold, calling the second layer sub-model to perform sorting processing on the image to be processed to obtain a second sorting result and a second confidence level corresponding to the second sorting result; In response to the second confidence being greater than or equal to a second threshold, the second sorting result is used as the target sorting result corresponding to the material to be sorted.

4. The material sorting method according to claim 3, characterized in that: The material sorting model includes a plurality of intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the plurality of intermediate layer sub-models; inputting the image to be processed into the material sorting model, performing sorting processing on the image to be processed by calling at least one layer sub-model in the material sorting model, and determining a target sorting result corresponding to the material to be sorted further includes: In response to the second confidence being less than the second threshold, calling other intermediate layer sub-models layer by layer to perform sorting processing on the image to be processed, and determining corresponding candidate sorting results and confidences of the sorting results; If there is a sorting result confidence that is greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, the candidate sorting result is used as the target sorting result corresponding to the material to be sorted; If in response to the completion of calling all the intermediate layer sub-models and there is no sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, the highest layer sub-model is called to sort the image to be processed, and the obtained sorting result is used as the target sorting result, wherein the highest layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

5. The material sorting method according to claim 4, characterized in that: The step of calling other intermediate layer sub-models layer by layer to sort the image to be processed and determining the corresponding candidate sorting results and the confidence of the sorting results includes: Calling the current middle layer sub-model to perform sorting processing on the image to be processed, and obtaining candidate sorting results and confidence levels of the candidate sorting results; Obtaining the confidence of a previous candidate sorting result, where the previous candidate sorting result is a sorting result obtained by a previous intermediate layer sub-model performing sorting processing on the image to be processed; The sorting result confidence of the candidate sorting result is determined according to the first confidence weight corresponding to the previous intermediate layer sub-model, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate layer sub-model and the confidence of the candidate sorting result.

6. The material sorting method according to claim 3, characterized in that: The material sorting model includes a plurality of intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the plurality of intermediate layer sub-models; inputting the image to be processed into the material sorting model, performing sorting processing on the image to be processed by calling at least one layer sub-model in the material sorting model, and determining a target sorting result corresponding to the material to be sorted further includes: In response to the second confidence level being less than the second threshold, calling each of the intermediate layer sub-models to perform sorting processing on the image to be processed, and obtaining a third sorting result and a corresponding third confidence level corresponding to each of the intermediate layer sub-models; Determining a comprehensive sorting result determined by the plurality of the intermediate layer sub-models and a corresponding comprehensive confidence based on the confidence weight corresponding to each of the intermediate layer sub-models, the corresponding third sorting result, and the corresponding third confidence; In response to the comprehensive confidence being greater than or equal to a third threshold, the comprehensive sorting result is used as the target sorting result.

7. The material sorting method according to claim 6, characterized in that: The step of inputting the image to be processed into a material sorting model, performing sorting processing on the image to be processed by calling at least one sub-model in the material sorting model, and determining a target sorting result corresponding to the material to be sorted further includes: In response to the comprehensive confidence being less than the third threshold, the highest-level sub-model is called to perform sorting on the image to be processed, and the obtained sorting result is used as the target sorting result, wherein the highest-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

8. The material sorting method according to claim 1, characterized in that: Inputting the image to be processed into the material sorting model, performing sorting processing on the image to be processed by calling at least one sub-model in the material sorting model, and determining a target sorting result corresponding to the material to be sorted, includes: Inputting the image to be processed into the material sorting model; In response to the remaining amount of idle resources being sufficient, sorting the image to be processed by calling the highest-level sub-model in the material sorting model, and using the obtained sorting result as the target sorting result, wherein the highest-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models; In response to insufficient remaining idle resources, based on the hierarchical order of each sub-model in the material sorting model, each sub-model is called to sort the image to be processed until a sorting result with a confidence level greater than or equal to the corresponding threshold is obtained, and is used as the target sorting result corresponding to the material to be sorted.

9. The material sorting method according to claim 1, characterized in that: The method further comprises: Obtaining actual sorting results of the material to be sorted; Based on the comparison result between the actual sorting result and the target sorting result, the material sorting model is optimized.

10. The material sorting method according to claim 1 or 9, characterized in that: The method further comprises: In response to the received version update instruction, a hot update process is performed on the model parameters corresponding to the material sorting model to obtain a target material sorting model.

11. A material sorting device, characterized in that: Applied to material sorting equipment, the device includes: An acquisition module, used for acquiring an image of the material to be sorted to be processed; A first processing module is configured to input the image to be processed into a material sorting model, perform sorting processing on the image to be processed by calling at least one layer of sub-models in the material sorting model, and determine and output a target sorting result corresponding to the material to be sorted, including: based on the hierarchical order of each sub-model in the material sorting model, respectively calling each sub-model to perform sorting processing on the image to be processed until a sorting result with a confidence level greater than or equal to a corresponding threshold is obtained, and the result is used as the target sorting result corresponding to the material to be sorted, wherein the material sorting model includes different multi-layer sub-models, the multi-layer sub-models have the same functions and are independent of each other, the high-level sub-models have larger resource usage and higher sorting accuracy than the low-level sub-models, the lowest-level sub-model is the sub-model with the highest execution efficiency among the multi-layer sub-models, the sub-models are calculated based on a traditional material sorting algorithm or trained based on an artificial intelligence algorithm, and in the material sorting model, the lowest-level sub-model is calculated based on a traditional material sorting algorithm, and the other sub-models have a different framework from the lowest-level sub-model and are trained based on an artificial intelligence algorithm; The second processing module is used to sort the materials to be sorted according to the target sorting result.

12. A material sorting device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor performs the material sorting method according to any one of claims 1 to 10 by executing the computer instructions.

13. A computer-readable storage medium storing the following program, wherein the program is used to perform the material sorting method according to any one of claims 1 to 10.

Images