Material sorting method, material sorting device, material sorting equipment and storage medium

Abstract

The invention relates to the field of material sorting, in particular to a material sorting method, a material sorting device, material sorting equipment and a storage medium. The material sorting method comprises the following steps: acquiring a to-be-processed image of a to-be-sorted material; the to-be-processed image is input into a material sorting model, sorting processing is conducted on the to-be-processed image by calling at least one layer of sub-model in the material sorting model, a target sorting result corresponding to the to-be-sorted material is determined and output, the material sorting model comprises multiple different layers of sub-models, and the multiple layers of sub-models are the same in function and are mutually independent; the resource occupation amount of the high-level sub-model is larger than that of the low-level sub-model, the sorting accuracy is higher, and the sub-model is obtained through calculation based on a traditional material sorting algorithm or obtained through training based on an artificial intelligence algorithm; and sorting the to-be-sorted materials according to the target sorting result. The accuracy and reliability of the target sorting result can be guaranteed, then the material sorting efficiency can be improved, and the performance of the material sorting equipment is guaranteed.

Description

Technical Field

[0001] The present disclosure relates to the field of material sorting, and in particular to a material sorting method, a material sorting device, a material sorting equipment and a computer-readable storage medium. Background Art

[0002] In the process of material sorting, the currently transmitted materials are detected, identified and sorted by means of image acquisition. In the related technology, the material sorting system deployed on the material sorting equipment processes the image to be processed, determines the category of the material in the image to be processed, and then sorts the material in a targeted manner. Among them, the material sorting system includes an algorithm for material sorting. Among them, the algorithm can be a traditional material sorting algorithm or an algorithm based on the framework of an artificial intelligence (AI) model.

[0003] However, if the algorithm is a traditional algorithm for material sorting, although the execution efficiency is high, it will be limited by the execution logic designed by the expert system, which will affect the sorting accuracy. However, if the algorithm is based on the AI ​​model framework, although the accuracy is high, the execution efficiency is low and it is limited by the computing power of the equipment. Summary of the invention

[0004] In order to overcome the problems existing in the related art, an exemplary embodiment of the present disclosure provides a material sorting method, which is applied to material sorting equipment, and the method includes: obtaining a to-be-processed image of the to-be-sorted material; inputting the to-be-processed image into a material sorting model, and performing sorting processing on the to-be-processed image by calling at least one layer of sub-model in the material sorting model, and determining and outputting a target sorting result corresponding to the to-be-sorted material, wherein the material sorting model includes different multi-layer sub-models, and the multi-layer sub-models have the same functions and are independent of each other. The high-level sub-models have larger resource occupancy and higher sorting accuracy than the low-level sub-models, and the sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms; and sorting the to-be-sorted material according to the target sorting result.

[0005] In some embodiments, the image to be processed is input into a material sorting model, and the image to be processed is sorted by calling at least one layer of sub-model in the material sorting model to determine the target sorting result corresponding to the material to be sorted, including: inputting the image to be processed into the material sorting model, and sorting the image to be processed by calling the lowest layer of sub-model in the material sorting model to obtain a first sorting result and a first confidence level corresponding to the first sorting result; in response to the first confidence level being greater than or equal to a first threshold value, the first sorting result is used as the target sorting result corresponding to the material to be sorted; and the target sorting result is output.

[0006] In some embodiments, the image to be processed is input into a material sorting model, and at least one layer of material sorting sub-models in the material sorting model is called to perform sorting processing on the image to be processed to determine the target sorting result corresponding to the material to be sorted. It further includes: in response to the first confidence level being less than the first threshold, the second layer sub-model is called to perform sorting processing on the image to be processed, and a second sorting result and a second confidence level corresponding to the second sorting result are obtained, where the lowest layer sub-model is the sub-model with the highest execution efficiency among the multi-layer sub-models; in response to the second confidence level being greater than or equal to the second threshold, the second sorting result is used as the target sorting result corresponding to the material to be sorted.

[0007] In some embodiments, the material sorting model includes multiple intermediate layer sub-models, where the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models; the image to be processed is input into the material sorting model, and at least one layer of sub-models in the material sorting model is called to perform sorting processing on the image to be processed to determine the target sorting result corresponding to the material to be sorted. It further includes: in response to the second confidence level being less than the second threshold, other intermediate layer sub-models are called layer by layer to perform sorting processing on the image to be processed, and corresponding candidate sorting results and sorting result confidence levels are determined; if there is a sorting result confidence level greater than or equal to the confidence level threshold corresponding to the intermediate layer sub-model, the candidate sorting result is used as the target sorting result corresponding to the material to be sorted; if in response to all intermediate layer sub-models being called and there is no sorting result confidence level greater than or equal to the confidence level threshold corresponding to the intermediate layer sub-model, the highest layer sub-model is called to perform sorting processing on the image to be processed, and the obtained sorting result is used as the target sorting result, where the highest layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

[0008] In some embodiments, calling other intermediate layer sub-models layer by layer to perform sorting processing on the image to be processed and determining corresponding candidate sorting results and sorting result confidence levels includes: calling the current intermediate layer sub-model to perform sorting processing on the image to be processed to obtain a candidate sorting result and the confidence level of the candidate sorting result; obtaining the confidence level of the previous candidate sorting result, where the previous candidate sorting result is the sorting result obtained by the previous intermediate layer sub-model performing sorting processing on the image to be processed; according to the first confidence level weight corresponding to the previous intermediate layer sub-model, the confidence level of the previous candidate sorting result, the second confidence level weight corresponding to the current intermediate layer sub-model, and the confidence level of the candidate sorting result, the sorting result confidence level of the candidate sorting result is determined.

[0009] In some embodiments, the material sorting model includes multiple intermediate layer sub-models, where the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models; inputting the image to be processed into the material sorting model, and performing sorting processing on the image to be processed by calling at least one layer of sub-models in the material sorting model to determine the target sorting result corresponding to the material to be sorted, further including: in response to the second confidence level being less than the second threshold, respectively calling each intermediate layer sub-model to perform sorting processing on the image to be processed, obtaining the third sorting result corresponding to each intermediate layer sub-model and the corresponding third confidence level; based on the confidence level weight, the corresponding third sorting result, and the corresponding third confidence level corresponding to each intermediate layer sub-model, determining the comprehensive sorting result determined by the multiple intermediate layer sub-models and the corresponding comprehensive confidence level; in response to the comprehensive confidence level being greater than or equal to the third threshold, taking the comprehensive sorting result as the target sorting result.

[0010] In some embodiments, inputting the image to be processed into the material sorting model, and performing sorting processing on the image to be processed by calling at least one layer of sub-models in the material sorting model to determine the target sorting result corresponding to the material to be sorted, further including: in response to the comprehensive confidence level being less than the third threshold, calling the highest layer sub-model to perform sorting processing on the image to be processed, and taking the obtained sorting result as the target sorting result, where the highest layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy rate among the multiple layer sub-models.

[0011] In some embodiments, inputting the image to be processed into the material sorting model, and performing sorting processing on the image to be processed by calling at least one layer of sub-models in the material sorting model to determine the target sorting result corresponding to the material to be sorted, including: inputting the image to be processed into the material sorting model; in response to the sufficient remaining amount of idle resources, performing sorting processing on the image to be processed by calling the highest layer sub-model in the material sorting model, and taking the obtained sorting result as the target sorting result, where the highest layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy rate among the multiple layer sub-models; in response to the insufficient remaining amount of idle resources, respectively calling each sub-model to perform sorting processing on the image to be processed based on the hierarchical order of each sub-model in the material sorting model until the obtained sorting result with a confidence level greater than or equal to the corresponding threshold is obtained, and taking it as the target sorting result corresponding to the material to be sorted.

[0012] In some embodiments, in the material sorting model, the lowest layer sub-model is calculated based on a traditional material sorting algorithm, and other sub-models have different frameworks from the lowest layer sub-model and are trained based on artificial intelligence algorithms.

[0013] In some embodiments, the method further includes: obtaining the actual sorting result of the material to be sorted; optimizing the material sorting model based on the comparison result between the actual sorting result and the target sorting result.

[0014] In some embodiments, the method further includes: in response to a received version update instruction, performing a hot update process on the model parameters corresponding to the material sorting model to obtain a target material sorting model.

[0015] In a second aspect, the present disclosure also provides a material sorting device, which is applied to a material sorting device. The device includes: an acquisition module, configured to acquire a to-be-processed image of the material to be sorted; a first processing module, configured to input the to-be-processed image into the material sorting model, and perform a sorting process on the to-be-processed image by invoking at least one layer of sub-models in the material sorting model to determine and output a target sorting result corresponding to the material to be sorted. The material sorting model includes different multi-layers of sub-models, the functions of the multi-layers of sub-models are the same and independent of each other, the high-level sub-models occupy more resources and have a higher sorting accuracy than the low-level sub-models, and the sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms; a second processing module, configured to sort the material to be sorted according to the target sorting result.

[0016] In a third aspect, the present disclosure also provides a material sorting device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the material sorting method provided in any one of the above aspects.

[0017] In a fourth aspect, the present disclosure also provides a computer-readable storage medium, which stores the following program, and the program is used to perform the material sorting method provided in any one of the above aspects.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.

[0019] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: According to the material sorting method provided by the present disclosure, by calling at least one layer of sub-models in the material sorting model to perform sorting processing on the image to be processed, the accuracy and reliability of the target sorting result can be ensured. Furthermore, when sorting the materials to be sorted, the efficiency of material sorting can be improved, and the performance of the material sorting equipment can be ensured. Moreover, the material sorting model includes both sub-models calculated based on traditional material sorting algorithms and sub-models trained based on artificial intelligence algorithms. Each sub-model is independent. Therefore, when performing material sorting, the advantages of traditional algorithms and artificial intelligence can be combined, which can improve the flexibility and execution efficiency of material sorting while ensuring the accuracy of material sorting, and is also helpful for expanding and maintaining the material sorting model. Since the high-level sub-models occupy more resources and have higher sorting accuracy than the low-level sub-models, through multiple layers of sub-models, especially high-level sub-models, finer feature differences can be captured, thereby realizing fine sorting of materials and providing the accuracy of material sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By describing the exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, the present disclosure can be better understood. In the drawings:

[0021] Figure 1 is a schematic structural diagram of a material sorting device shown according to an exemplary embodiment of the present disclosure;

[0022] Figure 2 is a schematic flowchart of a material sorting method shown according to an exemplary embodiment of the present disclosure;

[0023] Figure 3 is a schematic flowchart of another material sorting method shown according to an exemplary embodiment of the present disclosure;

[0024] Figure 4 is a schematic flowchart of yet another material sorting method shown according to an exemplary embodiment of the present disclosure;

[0025] Figure 5 is a schematic block diagram of the structure of a material sorting device shown according to an exemplary embodiment of the present disclosure;

[0026] Figure 6 is a schematic block diagram of the structure of a material sorting device shown according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Specific embodiments of the present disclosure will be described below. It should be noted that in the process of the specific description of these embodiments, for the sake of concise description, this specification may not describe all features of the actual embodiments in detail. It should be understood that in the actual implementation process of any embodiment, just as in the process of any engineering project or design project, in order to achieve the specific goals of the developer and to meet system-related or business-related restrictions, various specific decisions are often made, and these will also change from one embodiment to another. In addition, it should also be understood that although the efforts made in this development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present disclosure, some design, manufacturing or production changes based on the technical content disclosed in the present disclosure are only conventional technical means and should not be understood as the content of the present disclosure being insufficient.

[0028] Unless otherwise defined, technical terms or scientific terms used in the present disclosure should have the ordinary meaning understood by those of ordinary skill in the art within the technical field to which the present disclosure belongs. The "first", "second" and similar terms used in the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. Words such as "a" or "an" do not denote a quantity limitation, but mean that there is at least one. Words such as "comprising" or "including" mean that the elements or objects appearing before "comprising" or "including" cover the elements or objects listed after "comprising" or "including" and their equivalent elements, and do not exclude other elements or objects. The terms "connected" or "coupled" and the like are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0029] In the related art, based on a material sorting system deployed in a material sorting device, an image to be processed is processed to determine the category of the material corresponding to the image to be processed, and then the material is sorted specifically. Among them, the material sorting system includes an algorithm for material sorting. Among them, the algorithm can be a traditional material sorting algorithm or an algorithm based on an AI model as a framework.

[0030] However, if the algorithm is a traditional algorithm for material sorting, although the execution efficiency is high, it will be limited by the execution logic designed by the expert system, which will affect the sorting accuracy, and the generalization ability is also poor. However, if the algorithm is an algorithm based on an AI model as a framework, although the generalization ability is strong, the adaptation ability is strong, and the reliability is high, the execution efficiency is low and it is limited by the device computing power.

[0031] To solve the above problems, an exemplary embodiment of the present disclosure provides a material sorting method that can be applied to a material sorting device. As Figure 1As shown, the material sorting device 100 may include a feeding mechanism 110, a conveying mechanism 120, a detection mechanism 130, and a sorting device 140. The feeding mechanism 110 is used to feed the material to be sorted into the conveying mechanism 120. The conveying mechanism 120 may be a structure such as a conveyor belt or a chute, and is used to convey the material to be sorted fed by the feeding mechanism 110. The detection mechanism 130 is used to detect the material conveyed on the conveying mechanism 120 to detect whether the material is the material to be removed; the material to be removed refers to the material separated by the sorting device. The material to be removed may be the required material or the non-required material, as long as the sorting of the material can be achieved. The sorting device 140 is used to remove the material to be removed. The material sorting device 100 can be used for sorting tasks such as ore sorting, plastic bottle sorting, or other material sorting, depending on the actual application requirements.

[0032] As Figure 2 shown, the material sorting method may include:

[0033] Step S210, obtaining a to-be-processed image of the material to be sorted.

[0034] The to-be-processed image can be understood as an image containing the material to be sorted. The to-be-processed image can be obtained during the process of the conveying mechanism of the material sorting device conveying the material. By obtaining the to-be-processed image, the material category in the to-be-processed image can be specifically identified and determined. Subsequently, when the sorting mechanism of the material sorting device performs material sorting, the material sorting efficiency can be improved.

[0035] Step S220, inputting the to-be-processed image into the material sorting model, and performing sorting processing on the to-be-processed image by calling at least one layer of sub-model in the material sorting model to determine and output the target sorting result corresponding to the material to be sorted.

[0036] The material sorting model can be understood as a model that is pre-deployed in the material sorting equipment and used for material sorting. Among them, the material sorting model can include different multi-layer sub-models. The functions of the multi-layer sub-models are the same and independent of each other. The higher-level sub-models have a larger resource occupancy and a higher sorting accuracy than the lower-level sub-models. The sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms. That is, in this material sorting model, there are multiple different sub-models that can be used to perform material sorting, and the multiple sub-models are distributed at different levels. In the actual process of material sorting, each sub-model can perform material sorting processing independently. The higher the level of the sub-model, the greater the resource occupancy required for the sub-model to perform material sorting processing, and the higher the sorting accuracy. In the material sorting model, some sub-models can be calculated based on traditional material sorting algorithms, and some sub-models can be trained based on artificial intelligence algorithms. Traditional material sorting algorithms can include, but are not limited to, algorithms for material sorting based on feature extraction algorithms, optical sorting algorithms, or other algorithms.

[0037] To determine the target sorting result corresponding to the material to be sorted in the image to be processed, after obtaining the image to be processed, the image to be processed is input into the material sorting model, and at least one layer of sub-model in the material sorting model is called to perform sorting processing on the image to be processed, so as to determine the target sorting result corresponding to the material to be sorted through at least one layer of sub-model. Then, after obtaining the target sorting result corresponding to the material to be sorted, the target sorting result is output, so that the material to be sorted can be sorted specifically according to the target sorting result subsequently.

[0038] In some examples, at least one layer of sub-model can be called to perform sorting processing on the image to be processed according to the hierarchical structure of the sub-models in the material sorting model. For example, the lowest-level sub-model can be called to process the image to be processed first. If the confidence level output by the lowest-level sub-model is relatively high, the sorting result output by the lowest-level sub-model can be used as the target sorting result. At this time, only one layer of sub-model in the material sorting model needs to be called to determine the target sorting result. If the confidence level output by the lowest-level sub-model is relatively high, other layers of sub-models are continuously called to process the image to be processed until the target sorting result is determined. At this time, at least two layers of sub-models in the material sorting model need to be called to determine the target sorting result.

[0039] In some other examples, at least one layer of sub-models can be called according to the remaining amount of idle resources of the material sorting device to perform sorting processing on the image to be processed. For example, if the remaining amount of idle resources is sufficient, the highest layer of sub-model in the material sorting model can be selected to process the image to be processed to obtain the most accurate target analysis result. If the remaining amount of idle resources is insufficient, at least one low-level sub-model is called to process the image to be processed to obtain the required target sorting result.

[0040] Step S230, sort the materials to be sorted according to the target sorting result.

[0041] Through the target sorting result, the category corresponding to the material to be sorted can be determined, and then the material to be sorted can be sorted according to this category to meet the requirements of material sorting. For example, if the material to be sorted is ore, the target sorting result can be used to determine whether the ore is concentrate or waste ore, and then the ore can be sorted specifically. Another example is that if the material to be sorted is a plastic bottle, the target sorting result can be used to determine whether the plastic bottle is a bottle to be retained or a discarded bottle, and then the plastic bottle can be sorted specifically.

[0042] According to the material sorting method provided by the present disclosure, by calling at least one layer of sub-models in the material sorting model to perform sorting processing on the image to be processed, the accuracy and reliability of the target sorting result can be guaranteed. Furthermore, when sorting the materials to be sorted, the efficiency of material sorting can be improved and the performance of the material sorting device can be guaranteed. Moreover, the material sorting model includes both sub-models calculated based on traditional material sorting algorithms and sub-models trained based on artificial intelligence algorithms. Each sub-model is independent. Therefore, when performing material sorting, the advantages of traditional algorithms and artificial intelligence can be combined, which can improve the flexibility and execution efficiency of material sorting while ensuring the accuracy of material sorting, and is also helpful for expanding and maintaining the material sorting model. Since the high-level sub-models occupy more resources and have higher sorting accuracy than the low-level sub-models, through multiple layers of sub-models, especially high-level sub-models, finer feature differences can be captured, and then fine sorting of materials can be realized, thereby providing the accuracy of material sorting.

[0043] In some embodiments, as Figure 3 shown, the above step S220 may include:

[0044] Step S221, input the image to be processed into the material sorting model, and call the lowest layer of sub-model in the material sorting model to perform sorting processing on the image to be processed to obtain a first sorting result and a first confidence level corresponding to the first sorting result.

[0045] In the material sorting model, the lower the sub - model level, the higher the execution efficiency of the model, that is, the lower the delay during the sorting process. Therefore, the lowest - level sub - model has the highest execution efficiency among the multi - level sub - models.

[0046] After obtaining the image to be processed, input it into the material sorting model, and preferentially call the lowest - level sub - model in the material sorting model to perform sorting processing on the image to be processed, so as to improve the sorting efficiency, and then obtain the first sorting result and the first confidence level corresponding to the first sorting result. Among them, the first sorting result can be understood as the sorting result identified after the lowest - level sub - model performs sorting processing on the image to be processed.

[0047] Step S222, in response to the first confidence level being greater than or equal to the first threshold, take the first sorting result as the target sorting result corresponding to the material to be sorted.

[0048] The first threshold can be understood as the minimum confidence level value used to measure that the sorting result output by the lowest - level sub - model is reliable. For example, the first threshold can be 0.95, and the specific value can be determined according to requirements and is not limited here.

[0049] In response to the first confidence level being greater than or equal to the first threshold, it indicates that the sorting result obtained by the lowest - level sub - model for sorting the image to be processed is reliable. Therefore, the first sorting result can be taken as the target sorting result corresponding to the material to be sorted.

[0050] Step S223, output the target sorting result.

[0051] Output the target sorting result to clarify the sorting type corresponding to the material to be sorted, which is convenient for subsequent targeted sorting.

[0052] Determine the target sorting result according to the above method. By preferentially calling the lowest - level sub - model to classify the image to be processed, the efficiency of determining the target sorting result can be maximally improved, and the delay can be reduced, which helps to speed up the subsequent sorting efficiency of the material to be sorted and improve the performance of the material sorting equipment.

[0053] In some other embodiments, as Figure 3 shown, the above - mentioned step S220 may further include:

[0054] Step S224, in response to the first confidence level being less than the first threshold, call the second - level sub - model to perform sorting processing on the image to be processed, and obtain the second sorting result and the second confidence level corresponding to the second sorting result.

[0055] In response to the first confidence level being less than the first threshold, it is characterized that the sorting result of the image to be processed by the lowest-level sub-model is unreliable, and the first sorting result belongs to an invalid result. Therefore, to determine the target sorting result corresponding to the material to be sorted, the second-level sub-model is called to perform sorting processing on the image to be processed, so as to determine the sorting category corresponding to the material to be sorted through the second-level sub-model, thereby obtaining the second sorting result and the second confidence level corresponding to the second sorting result. Among them, the level of the second-level sub-model is higher than that of the lowest-level sub-model. Therefore, compared with the lowest-level sub-model, the second-level sub-model has a large resource occupancy and a high sorting accuracy, but has a certain delay compared with the lowest-level sub-model. The second sorting result can be understood as the sorting result identified after the second-level sub-model performs sorting processing on the image to be processed.

[0056] Step S225, in response to the second confidence level being greater than or equal to the second threshold, the second sorting result is used as the target sorting result corresponding to the material to be sorted.

[0057] The second threshold can be understood as the minimum confidence level value used to measure that the sorting result output by the second-level sub-model is reliable. For example, the second threshold can be 0.95, and the specific value can be determined according to requirements, and the second threshold can be the same as the first threshold or different from the first threshold, which is not limited here.

[0058] In response to the second confidence level being greater than or equal to the second threshold, it is characterized that the sorting result of the image to be processed by the second-level sub-model is reliable. Therefore, the second sorting result can be used as the target sorting result corresponding to the material to be sorted.

[0059] Determining the target sorting result according to the above method can continue to call sub-models of other levels to perform sorting processing on the image to be processed when the sorting result output by the lowest-level sub-model is unreliable, which can meet the processing requirements for determining the target sorting result and ensure the accuracy and reliability of the finally obtained target sorting result.

[0060] In some other embodiments, the material sorting model includes multiple intermediate-level sub-models. Among them, the second-level sub-model is the lowest-level sub-model among the multiple intermediate-level sub-models. As Figure 3 shown, the above step S220 may further include:

[0061] Step S226, in response to the second confidence level being less than the second threshold, layer by layer call other intermediate-level sub-models to perform sorting processing on the image to be processed, and determine the corresponding candidate sorting results and sorting result confidence levels.

[0062] In response to the second confidence level being less than the second threshold, it indicates that the sorting result of the image to be processed by the second-layer sub-model is unreliable, and the second sorting result belongs to an invalid result. Therefore, to determine the target sorting result corresponding to the material to be sorted, other intermediate-layer sub-models are called layer by layer to perform sorting processing on the image to be processed, and the candidate sorting results obtained after each called intermediate-layer sub-model performs sorting processing on the image to be processed and the corresponding sorting result confidence levels are determined.

[0063] In some examples, step S226 described above may include the following steps:

[0064] Step a1, call the current intermediate-layer sub-model to perform sorting processing on the image to be processed, and obtain the candidate sorting result and the confidence level of the candidate sorting result;

[0065] Step a2, obtain the confidence level of the previous candidate sorting result;

[0066] Step a3, determine the sorting result confidence level of the candidate sorting result according to the first confidence weight corresponding to the previous intermediate-layer sub-model, the confidence level of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate-layer sub-model, and the confidence level of the candidate sorting result.

[0067] Specifically, for the current intermediate-layer sub-model, after calling the current intermediate-layer sub-model to perform sorting processing on the image to be processed, the candidate sorting result and the confidence level of the candidate sorting result are obtained.

[0068] Since the current intermediate-layer sub-model is located above the previous intermediate-layer sub-model, in terms of sorting efficiency and sorting effect, it can be relatively superior to the previous intermediate-layer sub-model. However, in the actual process of calling sub-models, the previous intermediate-layer sub-model is called first, and the current intermediate-layer sub-model is called for sorting processing only when the sorting result confidence level corresponding to the candidate sorting result determined by the previous intermediate-layer sub-model is less than the confidence threshold of the corresponding intermediate-layer sub-model.

[0069] Therefore, to make the result output by the current intermediate-layer sub-model more reliable, the confidence level of the previous candidate sorting result is obtained, where the previous candidate sorting result is the sorting result obtained by the previous intermediate-layer sub-model performing sorting processing on the image to be processed.

[0070] According to the first confidence weight corresponding to the previous intermediate layer sub-model, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate layer sub-model, and the confidence of the candidate sorting result, jointly determine the sorting result confidence of the candidate sorting result corresponding to the current intermediate layer sub-model, so that the determination of the sorting result confidence can refer to the sorting result determination situation of the previous intermediate layer sub-model, thereby making the obtained sorting result confidence more reliable and helping to improve the robustness and decision-making quality of the material sorting model. Among them, the sum of the first confidence weight corresponding to the previous intermediate layer sub-model and the second confidence weight corresponding to the current intermediate layer sub-model is 1, and the specific weight distribution can be determined according to the reliability of the actual model, which is not limited here.

[0071] In some examples, it can be through weighted average or a more complex fusion function to fuse the first confidence weight corresponding to the previous intermediate layer sub-model, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate layer sub-model, and the confidence of the candidate sorting result, so as to obtain the sorting result confidence that can characterize the candidate sorting result. For example, taking weighted average as an example, if the first confidence weight corresponding to the previous intermediate layer sub-model is 0.7 and the first confidence weight of the current intermediate layer sub-model is 0.3, then the sorting result confidence of the candidate sorting result = the confidence of the previous candidate sorting result * 0.7 + the confidence of the candidate sorting result * 0.3.

[0072] In other examples, the confidence of the candidate sorting result determined by the current layer sub-model can be directly used as the sorting result confidence corresponding to the candidate sorting result, which helps to improve the determination efficiency and facilitate the quick determination of the target sorting result.

[0073] Step S227, if there is a sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, then use the candidate sorting result as the target sorting result corresponding to the material to be sorted.

[0074] If during the process of layer-by-layer call, there is a sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, it indicates that the candidate sorting result detected by this intermediate layer sub-model is reliable. Therefore, the candidate sorting result detected by this intermediate layer sub-model can be directly used as the target sorting result corresponding to the material to be sorted, without the need to continue calling the next intermediate layer sub-model for processing, which helps to avoid resource waste.

[0075] Step S228, if in response to the completion of the call of all intermediate layer sub-models and there is no sorting result confidence greater than or equal to the confidence threshold of the corresponding intermediate layer sub-model, then call the top layer sub-model to perform sorting processing on the image to be processed, and use the obtained sorting result as the target sorting result.

[0076] If until the highest intermediate sub-model is called, there is no confidence level of the sorting result that is greater than or equal to the confidence threshold of the corresponding intermediate sub-model, then the multiple intermediate sub-models in the material sorting model are not suitable for sorting the image to be sorted. Therefore, the highest sub-model is called to sort the image to be processed, and the obtained sorting result is used as the target sorting result to ensure the accuracy of the target sorting result. Among them, the highest sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models. The sorting result determined by the highest sub-model can be assumed to be the most accurate and reliable by default.

[0077] By determining the target sorting result in the above manner, the image to be processed can be sorted by calling the intermediate layer sub-model layer by layer, which can improve the flexibility and execution efficiency of material sorting while ensuring the accuracy of material sorting. Moreover, since the high-level sub-model can capture more subtle feature differences, it can ensure that the image to be processed can be sorted more finely, so that the final target sorting result is more accurate and reliable.

[0078] In some further embodiments, the material sorting model includes a plurality of intermediate layer sub-models, wherein the second layer sub-model is the lowest layer sub-model among the plurality of intermediate layer sub-models, such as Figure 3 As shown, the above step S220 may further include:

[0079] Step S229, in response to the second confidence being less than the second threshold, each intermediate layer sub-model is called to perform sorting processing on the image to be processed, and a third sorting result and a corresponding third confidence corresponding to each intermediate layer sub-model are obtained.

[0080] In response to the second confidence being less than the second threshold, it is characterized that the result of sorting the image to be processed by the second layer sub-model is unreliable, and the second sorting result is an invalid result.

[0081] In the material sorting model, the execution efficiency of sub-models at the same level is relatively close. However, since the highest-level sub-model is the sub-model with the highest accuracy, its execution efficiency is the lowest compared to other sub-models. Therefore, in order to ensure the efficiency of determining the target sorting result and reduce the delay as much as possible, each intermediate-level sub-model is called separately to perform sorting processing on the image to be processed, and the third sorting result and the corresponding third confidence level corresponding to each intermediate-level sub-model are obtained, so that the target sorting result can be determined based on all the third sorting results and the corresponding third confidence levels in the future.

[0082] Step S2210: Based on the confidence weights, corresponding third sorting results, and corresponding third confidence levels of each intermediate-layer sub-model, determine the comprehensive sorting result and the corresponding comprehensive confidence level determined by multiple intermediate-layer sub-models.

[0083] The comprehensive performances of multiple intermediate-layer sub-models are relatively close. Therefore, to make the sorting results determined by invoking intermediate-layer sub-models more reliable and exclude the situation where some intermediate-layer sub-models have abnormal identifications due to failures during image processing, based on the confidence weights, corresponding third sorting results, and corresponding third confidence levels of each intermediate-layer sub-model, through the method of weighted summation, determine the comprehensive sorting result and the corresponding comprehensive confidence level jointly determined by multiple intermediate-layer sub-models, so as to represent the sorting results corresponding to multiple intermediate-layer sub-models through the comprehensive sorting result. For example, taking two intermediate-layer sub-models as an example, including the second-layer sub-model and the third-layer sub-model. The third confidence weight corresponding to the second-layer sub-model is 0.7, and the third confidence weight corresponding to the third-layer sub-model is 0.3. Then the comprehensive confidence level = the third confidence level corresponding to the second-layer sub-model * 0.7 + the third confidence level corresponding to the third-layer sub-model * 0.3.

[0084] Step S2211: In response to the comprehensive confidence level being greater than or equal to the third threshold, use the comprehensive sorting result as the target sorting result.

[0085] The third threshold can be understood as the minimum confidence value used to measure that the sorting results output by multiple intermediate-layer sub-models are reliable. For example, the third threshold can be 0.8, and the specific value can be determined according to requirements and is not limited here.

[0086] In response to the comprehensive confidence level being greater than or equal to the third threshold, it indicates that the sorting result obtained by sorting the image to be processed through multiple intermediate-layer sub-models is reliable. Therefore, the comprehensive sorting result can be used as the target sorting result corresponding to the material to be sorted.

[0087] Determining the target sorting result according to the above method can reduce the latency as much as possible and improve the determination efficiency of the target sorting result, thereby contributing to promoting the execution process of material sorting.

[0088] In some other embodiments, as Figure 3 shown, the above step S220 may further include:

[0089] Step S2212: In response to the comprehensive confidence level being less than the third threshold, invoke the top-layer sub-model to perform sorting processing on the image to be processed, and use the obtained sorting result as the target sorting result.

[0090] In response to the comprehensive confidence being less than the third threshold, it indicates that the sorting result obtained by processing the image to be processed through multiple intermediate layer sub-models is still an unreliable result. Therefore, the highest layer sub-model is called to sort the image to be processed, and the obtained sorting result is used as the target sorting result to ensure the accuracy of the target sorting result. Among them, the highest layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models. The sorting result determined by the highest layer sub-model can be defaulted to be the most accurate and reliable.

[0091] Determining the target sorting result according to the above method can ensure that the target sorting result can be output smoothly, the material sorting task can be carried out in an orderly manner, and the materials to be sorted can be sorted normally.

[0092] In some other embodiments, in order to save the remaining idle resources, some of the lowest intermediate layer sub-models can be called first to jointly determine the comprehensive sorting results and the corresponding comprehensive confidence. If the comprehensive confidence at this time is greater than or equal to the third confidence, the comprehensive sorting results jointly determined by some of the lowest intermediate layer sub-models are directly used as the target sorting results. If the comprehensive confidence at this time is less than the third confidence, other intermediate layer sub-models are called to jointly determine the comprehensive sorting results and the corresponding comprehensive confidence, so as to reduce the probability of the highest layer sub-model being called as much as possible. Among them, for this situation, the confidence weight corresponding to each intermediate layer sub-model is determined based on the corresponding performance index and the number of intermediate layer sub-models called, and can be dynamically adjusted according to the actual calling situation to ensure that the value range corresponding to the determined comprehensive confidence is in [0,1].

[0093] In some application scenarios, at least one or more of the first confidence level, the second confidence level, and the third confidence level may be determined by random forest, gradient boosting tree, entropy, or Bayesian error estimation, and the specific determination method may be determined based on the characteristics and requirements of the corresponding sub-model.

[0094] In some application scenarios, the distributed stream processing platform (Apache Kafka) can be used to control the material sorting model call or switch sub-models to sort the images to be processed. The distributed stream processing platform can support decision trees or reinforcement learning (DQN) to generate management rules for calling or switching sub-models to reduce manual intervention and enhance the flexibility of sub-model calls, thereby providing more intelligent and efficient model management and decision support for material sorting models.

[0095] In some embodiments, Figure 4 As shown, the above step S220 may further include:

[0096] Step S2211, input the image to be processed into the material sorting model.

[0097] Step S2212: In response to sufficient remaining idle resources, sort the image to be processed by invoking the top-level sub-model in the material sorting model, and use the obtained sorting result as the target sorting result.

[0098] In response to sufficient remaining idle resources, it indicates that when sorting the image to be processed, it can be unrestricted by the inability to determine the target sorting result due to insufficient resources. Therefore, the top-level sub-model in the material sorting model can be directly invoked to sort the image to be processed, enabling the top-level sub-model to make full use of the remaining idle resources, thereby obtaining an accurate and reliable target sorting result to ensure the efficiency and accuracy of subsequent material sorting. Among them, the top-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-level sub-models.

[0099] In some application scenarios, the remaining amount of idle resources can be monitored through Prometheus (an open-source monitoring tool) to ensure the timeliness of determining the remaining idle resources.

[0100] Step S2213: In response to insufficient remaining idle resources, sort the image to be processed by invoking each sub-model based on the hierarchical order of the sub-models in the material sorting model until a sorting result with a confidence level greater than or equal to the corresponding threshold is obtained and used as the target sorting result for the material to be sorted.

[0101] In response to insufficient remaining idle resources, it indicates that directly using the top-level sub-model to sort the image to be processed will affect the sorting performance of the material sorting equipment. Therefore, to reduce latency, each sub-model is sequentially invoked to sort the image to be processed based on the hierarchical order of the sub-models in the material sorting model. If the confidence level corresponding to the sorting result determined by a certain sub-model is greater than or equal to the sorting result of the corresponding threshold, it indicates that the sorting result determined by this sub-model is reliable. Furthermore, this sorting result can be used as the target sorting result for the material to be sorted to save resources and avoid resource waste.

[0102] In some embodiments, the hierarchical order can be from low level to high level. Since the resource occupancy required by the low-level sub-models is relatively small, preferentially invoking the low-level sub-models to sort the image to be processed can ensure that there are sufficient resources available during the sorting process, thereby accelerating the determination efficiency of the target sorting result and reducing latency.

[0103] In some other embodiments, the hierarchical order may be from high level to low level, so as to ensure the determination efficiency of the target sorting result as much as possible, reduce the number of times the material sorting model calls sub-models, and simplify the management logic of the material sorting model.

[0104] In some examples, the material sorting model can be run by calling any or a combination of the remaining idle resources of a Graphics Processing Unit (GPU) and a Central Processing Unit (CPU), which can be specifically determined according to the hardware configuration requirements in the material sorting device and will not be limited here. For example, if according to the configuration requirements of the GPU, it can be determined that the GPU can independently undertake the operation of the material sorting model, then during the operation of the material sorting model, the remaining idle resources on the GPU can be called to support the operation of each sub-model in the material sorting model. If according to the configuration requirements of the GPU, it is determined that the GPU cannot independently undertake the operation of the material sorting model, then the remaining idle resources on the GPU and the CPU can be called to jointly support the operation of the material sorting model. If according to the configuration requirements of the GPU, it can be determined that the GPU cannot undertake the operation of the material sorting model, then during the operation of the material sorting model, the remaining idle resources on the CPU can be called to support the operation of each sub-model in the material sorting model.

[0105] In some embodiments, in the material sorting model, the lowest-level sub-model can be calculated based on traditional material sorting algorithms, and other sub-models have different frameworks from the lowest-level sub-model and are trained based on artificial intelligence algorithms. By deploying the material sorting model in this way, when sorting the image to be processed, the sub-model calculated based on traditional material sorting algorithms can be preferentially used for sorting to quickly determine the target sorting result, reduce latency, and improve execution efficiency. Moreover, by combining traditional material sorting algorithms and artificial intelligence algorithms to deploy multiple sub-models, the advantages of traditional material sorting algorithms and artificial intelligence algorithms can be fully utilized to achieve more efficient and accurate material sorting.

[0106] In some examples, the material sorting model can include three layers of sub-models, and the sub-models of each layer are divided based on efficiency (latency, resource occupancy) and effect (accuracy, recall rate). For example, the lowest-level sub-model is the one with the highest efficiency but lower effect compared to the sub-models of other layers. The highest-level sub-model is the one with the lowest efficiency but relatively the best effect. The number of intermediate-level sub-models can be greater than or equal to one. The number of intermediate-level sub-models can be determined according to the task requirements or deployment requirements to be executed.

[0107] In some application scenarios, taking the number of intermediate layer sub-models as two as an example, the hierarchical architecture of the material sorting model can include: the lowest layer sub-model can be a sub-model calculated based on traditional material sorting algorithms, the second layer sub-model is a lightweight model trained based on artificial intelligence algorithms (such as a model trained with MobileNet as the framework); the third layer sub-model is a balanced model trained based on artificial intelligence algorithms (such as a model trained with ResNet-50 as the framework), and the highest layer sub-model is a high-precision guarantee model trained based on artificial intelligence algorithms (such as a model trained with ResNet-152 as the framework). The second layer sub-model and the third layer sub-model are intermediate layer sub-models. Among them, the training methods of the sub-models in the material sorting model are all different. For example, the lowest layer sub-model can be obtained by performing feature extraction and analysis training based on the prior knowledge provided by the "expert system". The lightweight model can adopt the method of knowledge distillation, using a teacher model with a large training volume to transfer the knowledge learned by the teacher model to a student model with a small training volume for training, so as to obtain the required sub-model. The balanced model can be trained by adopting data augmentation (for example, data augmentation is performed through processing methods such as random masking or MixUp). The high-precision guarantee model can be obtained by training with all data, and the model size of this high-precision guarantee model is the largest.

[0108] In some other application scenarios, the lowest sub-layer model of the material sorting model is a sub-model calculated based on traditional material sorting algorithms, with a latency less than 1 millisecond (ms) and a resource occupancy less than 100 MB of memory. The position order of the second layer sub-model and the third layer sub-model can be determined according to the actual processing efficiency and effect. For example, if the latency of sub-model a is less than 50 ms and the resource occupancy is less than 500 MB, and the latency of sub-model b is between 50 - 200 ms and the resource occupancy is greater than 500 MB, then sub-model a is the second layer sub-model and sub-model b is the third layer sub-model. The highest layer sub-model has the highest accuracy but the largest resource occupancy.

[0109] It should be noted that the order and model type of the sub-models in the second layer and above of the material sorting model are not fixed and can be determined according to the actual model size, task processing efficiency, and task processing effect. The sub-models included in the material sorting models corresponding to different materials can be different or the same, depending on the actual training results.

[0110] In some application scenarios, multiple sub-models in the material sorting model are trained independently. Taking one of the sub-models as an example, when training the sub-model, the training sample expressions used for training can be to label the complexity label and domain label of the input data through a pre-set rule engine or a lightweight model (FastText). Among them, the complexity label can include: highly complex, moderately complex, or lowly complex. The domain label can be determined according to the actual material sorting task. For example, if it is ore sorting, the domain label can include ore identification and classification. If it is plastic bottle sorting, the domain label can include plastic bottle identification and classification.

[0111] When determining the training degree of the sub-model, it can be determined through a confidence scoring system. The confidence scoring system can include, but is not limited to, determining the convergence situation of the sub-model by performing any one or more of the classification task and the regression task, so as to determine the training degree of the sub-model. For example, when performing the classification task, the probability entropy can be calculated through normalization processing (such as using the Softmax function for processing), and the confidence can be calibrated through the Platt Scaling (a method of converting the output of a classification model into a class probability distribution), so as to determine the convergence situation of the sub-model. When performing the regression task, the uncertainty can be estimated through the prediction variance or Monte Carlo Dropout (MC Dropout), so as to determine the convergence situation of the sub-model.

[0112] In some other application scenarios, an APM tool (Datadog) can be pre-deployed to monitor the processing delay of each sub-model in real time. If the delay duration is greater than the delay duration threshold of the corresponding layer, the layer where the sub-model is located is adjusted so that the deployment of the sub-model meets the deployment requirements that the multi-layer sub-models in the material sorting model can meet the requirements that the higher-level sub-models have a larger resource occupancy and a higher sorting accuracy than the lower-level sub-models.

[0113] Step S2214, output the target sorting result.

[0114] In some embodiments, the material sorting method may further include:

[0115] Step b1, obtain the actual sorting result of the material to be sorted;

[0116] Step b2, optimize the material sorting model based on the comparison result between the actual sorting result and the target sorting result.

[0117] Specifically, to ensure the performance of the material sorting model, after determining the target sorting result output by the material sorting model, the target sorting result can be compared with the actual sorting result of the material to be sorted. If the two are the same, it means that the performance of the material sorting model is good and no optimization is required. However, if there is a difference between the two, it means that the performance of the material sorting model may be abnormal and needs to be optimized.

[0118] In some application scenarios, in order to avoid over-optimization due to errors, when it is determined that the performance of the material sorting model may be abnormal, the number of abnormalities in the material sorting model is monitored, and when the number is greater than the threshold, the material sorting model is optimized to ensure the performance of the material sorting model.

[0119] In other application scenarios, during the material sorting process, data can be collected based on the processing status of the sub-model, and then the parameters of the sub-model can be optimized in a targeted manner based on the collected data results, so that the sub-model can meet the corresponding hierarchical deployment requirements and ensure the performance of the material sorting model. Among them, data collection can be performed by recording online reasoning logs so that incremental training data sets can be constructed for the optimization of the sub-model later. Among them, the content of the online reasoning log may include the input, output, confidence of the sorting results, resource consumption, and image data of the corresponding sub-model. The content of optimizing the parameters of the sub-model may include, but is not limited to, adjusting the input parameters of the sub-model (for example, the batch size (batchsize) of the training samples used for model optimization and the image size, etc.).

[0120] In some embodiments, the material sorting method may further include: in response to the received version update instruction, hot updating the model parameters corresponding to the material sorting model to obtain the target material sorting model, so that the updated target material sorting model can meet the user's usage requirements. The version update instruction can be used to instruct a version rollback to return to a specified historical version of the material sorting model; or, the version update instruction can be used to instruct a version upgrade to obtain the latest version of the material sorting model.

[0121] In some optional application scenarios, taking the material sorting task as ore sorting as an example, the process of performing material sorting by the material sorting method provided by the present disclosure may be as follows:

[0122] During the process of transporting ores, a to-be-processed image of the material to be sorted is obtained and input into a pre-trained material sorting model to determine the target sorting result corresponding to the ore to be sorted. Among them, the material sorting model includes different multi-layer sub-models. The multi-layer sub-models have the same function and are independent of each other. The high-level sub-models have a larger resource occupancy and a higher sorting accuracy than the low-level sub-models. The lowest-level sub-model of the material sorting model is calculated based on a traditional material sorting algorithm and has the highest execution efficiency. The other layer sub-models of the material sorting model are trained based on artificial intelligence algorithms, and the highest-level sub-model has the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

[0123] In the material sorting model, the lowest-level sub-model is preferentially called to perform sorting processing on the to-be-processed image. In response to the first confidence level corresponding to the output first sorting result being greater than or equal to the first threshold, the first sorting result is used as the target sorting result corresponding to the ore to be sorted, and the target sorting result is output.

[0124] In response to the first confidence level corresponding to the output first sorting result being less than the first threshold, the second-layer sub-model is called to perform sorting processing on the to-be-processed image, obtaining a second sorting result and a second confidence level corresponding to the second sorting result. In response to the second confidence level being greater than or equal to the second threshold, the second sorting result is used as the target sorting result corresponding to the ore to be sorted.

[0125] In response to the second confidence level being less than the second threshold, each intermediate-layer sub-model is called to perform sorting processing on the to-be-processed image, obtaining a third sorting result and a corresponding third confidence level for each intermediate-layer sub-model. Then, based on the confidence weight corresponding to each intermediate-layer sub-model, the corresponding third sorting result, and the corresponding third confidence level, a comprehensive sorting result determined by multiple intermediate-layer sub-models and the corresponding comprehensive confidence level are determined. In response to the comprehensive confidence level being greater than or equal to the third threshold, the comprehensive sorting result is used as the target sorting result.

[0126] In response to the comprehensive confidence level being less than the third threshold, the highest-level sub-model is called to perform sorting processing on the to-be-processed image, and the obtained sorting result is used as the target sorting result.

[0127] According to the target sorting result, the ore to be sorted is sorted to improve the material sorting efficiency and ensure the sorting performance of the material sorting equipment.

[0128] In some other application scenarios, the material sorting model provided by the present disclosure can achieve a balance between efficiency and effectiveness. By combining the dynamic call strategy and the intelligent optimization mechanism, it is possible to reduce the average inference latency by 30% - 50% on the premise of a fixed hardware cost, while ensuring the accuracy rate in high-difficulty scenarios, thereby effectively improving the material sorting efficiency and enhancing the sorting performance of the material sorting equipment.

[0129] Based on the same inventive concept, the present disclosure also provides a material sorting device applied to a material sorting equipment. As Figure 5 shown, the material sorting device 300 may include:

[0130] An acquisition module 310, configured to acquire a to-be-processed image of the material to be sorted;

[0131] A first processing module 320, configured to input the to-be-processed image into the material sorting model, perform sorting processing on the to-be-processed image by invoking at least one layer of sub-models in the material sorting model, and determine and output a target sorting result corresponding to the material to be sorted. Among them, the material sorting model includes different multi-layers of sub-models, the functions of the multi-layers of sub-models are the same and independent of each other, the high-level sub-models occupy more resources and have a higher sorting accuracy rate than the low-level sub-models, and the sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms;

[0132] A second processing module 330, configured to sort the material to be sorted according to the target sorting result.

[0133] In some embodiments, the first processing module 320 may include: a first processing unit, configured to input the to-be-processed image into the material sorting model, perform sorting processing on the to-be-processed image by invoking the lowest-level sub-model in the material sorting model, and obtain a first sorting result and a first confidence level corresponding to the first sorting result. Among them, the lowest-level sub-model is the sub-model with the highest execution efficiency among the multi-layers of sub-models; a second processing unit, configured to, in response to the first confidence level being greater than or equal to a first threshold, use the first sorting result as the target sorting result corresponding to the material to be sorted; an output unit, configured to output the target sorting result.

[0134] In some embodiments, the first processing module 320 may further include: a third processing unit, configured to, in response to the first confidence level being less than the first threshold, invoke the second-layer sub-model to perform sorting processing on the to-be-processed image, and obtain a second sorting result and a second confidence level corresponding to the second sorting result; a fourth processing unit, configured to, in response to the second confidence level being greater than or equal to a second threshold, use the second sorting result as the target sorting result corresponding to the material to be sorted.

[0135] In some embodiments, the material sorting model includes multiple intermediate layer sub-models. Among them, the second-layer sub-model is the lowest-layer sub-model among the multiple intermediate layer sub-models. The first processing module 320 may further include: a fifth processing unit, configured to, in response to the second confidence level being less than the second threshold, layer by layer call other intermediate layer sub-models to perform sorting processing on the image to be processed, and determine the corresponding candidate sorting result and the confidence level of the sorting result; a first execution unit, configured to, if there is a confidence level of the sorting result greater than or equal to the confidence level threshold of the corresponding intermediate layer sub-model, use the candidate sorting result as the target sorting result corresponding to the material to be sorted; a second execution unit, configured to, if in response to all intermediate layer sub-models being called and there is no confidence level of the sorting result greater than or equal to the confidence level threshold of the corresponding intermediate layer sub-model, call the top-layer sub-model to perform sorting processing on the image to be processed, and use the obtained sorting result as the target sorting result, where the top-layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

[0136] In some embodiments, the fifth processing unit may include: a control unit, configured to call the current intermediate layer sub-model to perform sorting processing on the image to be processed to obtain the candidate sorting result and the confidence level of the candidate sorting result; an acquisition unit, configured to acquire the confidence level of the previous candidate sorting result, where the previous candidate sorting result is the sorting result obtained by the previous intermediate layer sub-model performing sorting processing on the image to be processed; a confidence level determination unit, configured to determine the confidence level of the sorting result of the candidate sorting result according to the first confidence level weight corresponding to the previous intermediate layer sub-model, the confidence level of the previous candidate sorting result, the second confidence level weight corresponding to the current intermediate layer sub-model, and the confidence level of the candidate sorting result.

[0137] In some embodiments, the material sorting model includes multiple intermediate layer sub-models. Among them, the second-layer sub-model is the lowest-layer sub-model among the multiple intermediate layer sub-models; the first processing module 320 may further include: a sixth processing unit, configured to, in response to the second confidence level being less than the second threshold, respectively call each intermediate layer sub-model to perform sorting processing on the image to be processed to obtain the third sorting result corresponding to each intermediate layer sub-model and the corresponding third confidence level; a seventh processing unit, configured to determine the comprehensive sorting result determined by the multiple intermediate layer sub-models and the corresponding comprehensive confidence level based on the confidence level weight corresponding to each intermediate layer sub-model, the corresponding third sorting result, and the corresponding third confidence level; an eighth processing unit, configured to, in response to the comprehensive confidence level being greater than or equal to the third threshold, use the comprehensive sorting result as the target sorting result.

[0138] In some embodiments, the first processing module 320 may further include: a ninth processing unit, configured to, in response to the comprehensive confidence level being less than a third threshold, call the top-level sub-model to perform sorting processing on the image to be processed, and use the obtained sorting result as the target sorting result, where the top-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-level sub-models.

[0139] In some embodiments, the first processing module 320 may include: an input unit, configured to input the image to be processed into the material sorting model; a first calling unit, configured to, in response to sufficient remaining idle resources, call the top-level sub-model in the material sorting model to perform sorting processing on the image to be processed, and use the obtained sorting result as the target sorting result, where the top-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-level sub-models; a second calling unit, configured to, in response to insufficient remaining idle resources, call each sub-model in the material sorting model to perform sorting processing on the image to be processed based on the hierarchical order of each sub-model in the material sorting model, until a sorting result with a confidence level greater than or equal to the corresponding threshold is obtained, and use it as the target sorting result corresponding to the material to be sorted.

[0140] In some embodiments, in the material sorting model, the lowest-level sub-model is calculated based on a traditional material sorting algorithm, and the other sub-models have different frameworks from the lowest-level sub-model and are trained based on an artificial intelligence algorithm.

[0141] In some embodiments, the material sorting device 300 may further include: an acquisition module, configured to acquire the actual sorting result of the material to be sorted; an optimization module, configured to optimize the material sorting model based on the comparison result between the actual sorting result and the target sorting result.

[0142] In some embodiments, the material sorting device 300 may further include: an update module, configured to, in response to a received version update instruction, perform hot update processing on the model parameters corresponding to the material sorting model to obtain a target material sorting model.

[0143] Regarding the image detection device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0144] Based on the same inventive concept, an embodiment of the present disclosure provides a material sorting device. As Figure 6As shown, the material sorting device includes: one or more processors 410, a memory 420, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the material sorting device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if needed, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple material sorting devices can be connected, and each device provides some necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system). Figure 6 In [the figure], a processor 410 is taken as an example.

[0145] The processor 410 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 410 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field-programmable gate array, a generic array logic, or any combination thereof.

[0146] Among them, the memory 420 stores instructions executable by at least one processor 410, so that at least one processor 410 executes the material sorting method shown in the above embodiments.

[0147] The memory 420 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the material sorting device, etc. In addition, the memory 420 can include a high-speed random access memory and can also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 420 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the material sorting device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The memory 420 can include a volatile memory, such as a random access memory; the memory can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive; the memory 420 can also include a combination of the above types of memories.

[0149] The material sorting device further includes an input device 430 and an output device 440. The processor 410, the memory 420, the input device 430, and the output device 440 may be connected through a bus or other means. Figure 6 Taking the connection through the bus as an example.

[0150] The input device 430 can receive input digital or character information and generate key signal inputs related to the user settings and function controls of the material sorting device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 440 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor), etc. The above display device includes, but is not limited to, a liquid crystal display, a light-emitting diode, a display, and a plasma display. In some alternative embodiments, the display device may be a touch screen.

[0151] Based on the same inventive concept, the present disclosure also provides a computer-readable storage medium storing the following program, and the program is used to execute the material sorting method of any of the foregoing embodiments.

[0152] The present disclosure uses specific terms to describe the embodiments of the present disclosure. Such as "an embodiment", "one embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of the present disclosure. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of the present disclosure can be appropriately combined.

[0153] In the context of the present disclosure, unless the context clearly indicates an exceptional situation, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0154] Similarly, it should be noted that, in order to simplify the expression of the present disclosure and thus help the understanding of one or more embodiments of the application, in the foregoing description of the embodiments of the present disclosure, sometimes multiple features are merged into one embodiment, drawing, or description thereof. However, this disclosure method does not mean that the features required by the object of the present disclosure are more than the features required to be protected. In fact, the features of the embodiment are less than all the features of the above-disclosed single embodiment.

[0155] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is only an example and does not constitute a limitation to the present disclosure. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to the present disclosure. Such modifications, improvements, and corrections are proposed in the present disclosure, so such modifications, improvements, and corrections still fall within the spirit and scope of the embodiments of the present disclosure.

Claims

1. A method for sorting materials, characterized in that, Applied to a material sorting device, the method includes: Obtaining a to-be-processed image of the material to be sorted; Inputting the to-be-processed image into a material sorting model, and performing sorting processing on the to-be-processed image by invoking at least one layer of sub-models in the material sorting model to determine and output a target sorting result corresponding to the material to be sorted. Among them, the material sorting model includes different multi-layers of sub-models. The functions of the multi-layers of sub-models are the same and independent of each other. The sub-models at higher levels occupy more resources and have higher sorting accuracy than those at lower levels. The sub-models are calculated based on traditional material sorting algorithms or trained based on artificial intelligence algorithms; Sorting the material to be sorted according to the target sorting result.

2. The method for sorting materials according to claim 1, characterized in that, The step of inputting the to-be-processed image into a material sorting model and performing sorting processing on the to-be-processed image by invoking at least one layer of sub-models in the material sorting model to determine the target sorting result corresponding to the material to be sorted includes: Inputting the to-be-processed image into the material sorting model, and performing sorting processing on the to-be-processed image by invoking the lowest-layer sub-model in the material sorting model to obtain a first sorting result and a first confidence level corresponding to the first sorting result. Among them, the lowest-layer sub-model is the sub-model with the highest execution efficiency among the multi-layers of sub-models; In response to the first confidence level being greater than or equal to a first threshold, taking the first sorting result as the target sorting result corresponding to the material to be sorted; Outputting the target sorting result.

3. The method for sorting materials according to claim 2, characterized in that, The step of inputting the to-be-processed image into a material sorting model and performing sorting processing on the to-be-processed image by invoking at least one layer of material sorting sub-models in the material sorting model to determine the target sorting result corresponding to the material to be sorted further includes: In response to the first confidence level being less than the first threshold, invoking the second-layer sub-model to perform sorting processing on the to-be-processed image to obtain a second sorting result and a second confidence level corresponding to the second sorting result; In response to the second confidence level being greater than or equal to a second threshold, taking the second sorting result as the target sorting result corresponding to the material to be sorted.

4. The method for sorting materials according to claim 3, characterized in that, The material sorting model includes multiple intermediate-layer sub-models. Among them, the second-layer sub-model is the lowest-layer sub-model among the multiple intermediate-layer sub-models. The step of inputting the to-be-processed image into a material sorting model and performing sorting processing on the to-be-processed image by invoking at least one layer of sub-models in the material sorting model to determine the target sorting result corresponding to the material to be sorted further includes: In response to the second confidence level being less than the second threshold, layer by layer invoking other intermediate-layer sub-models to perform sorting processing on the to-be-processed image, and determining corresponding candidate sorting results and sorting result confidence levels; If there is a sorting result confidence level greater than or equal to the confidence level threshold of the corresponding intermediate-layer sub-model, taking the candidate sorting result as the target sorting result corresponding to the material to be sorted; If, in response to all the intermediate layer sub-models having been called, and there is no sorting result confidence greater than or equal to the confidence threshold corresponding to the intermediate layer sub-model, the top layer sub-model is called to perform sorting processing on the to-be-processed image, and the obtained sorting result is used as the target sorting result, where the top layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

5. The method for sorting materials according to claim 4, characterized in that, The step of sequentially calling other intermediate layer sub-models to perform sorting processing on the to-be-processed image and determining the corresponding candidate sorting results and sorting result confidences includes: Calling the current intermediate layer sub-model to perform sorting processing on the to-be-processed image to obtain candidate sorting results and the confidence of the candidate sorting results; Obtaining the confidence of the previous candidate sorting result, where the previous candidate sorting result is the sorting result obtained by the previous intermediate layer sub-model performing sorting processing on the to-be-processed image; Determining the sorting result confidence of the candidate sorting result according to the first confidence weight corresponding to the previous intermediate layer sub-model, the confidence of the previous candidate sorting result, the second confidence weight corresponding to the current intermediate layer sub-model, and the confidence of the candidate sorting result.

6. The method for sorting materials according to claim 3, characterized in that, The material sorting model includes multiple intermediate layer sub-models, where the second layer sub-model is the lowest layer sub-model among the multiple intermediate layer sub-models; the step of inputting the to-be-processed image into the material sorting model and determining the target sorting result corresponding to the to-be-sorted material by calling at least one layer of sub-models in the material sorting model to perform sorting processing on the to-be-processed image further includes: In response to the second confidence being less than the second threshold, each of the intermediate layer sub-models is called to perform sorting processing on the to-be-processed image to obtain a third sorting result and a corresponding third confidence for each of the intermediate layer sub-models; Based on the confidence weight corresponding to each intermediate layer sub-model, the corresponding third sorting result, and the corresponding third confidence, determining the comprehensive sorting result and the corresponding comprehensive confidence determined by the multiple intermediate layer sub-models; In response to the comprehensive confidence being greater than or equal to the third threshold, the comprehensive sorting result is used as the target sorting result.

7. The method for sorting materials according to claim 6, characterized in that, The step of inputting the to-be-processed image into the material sorting model and determining the target sorting result corresponding to the to-be-sorted material by calling at least one layer of sub-models in the material sorting model to perform sorting processing on the to-be-processed image further includes: In response to the comprehensive confidence being less than the third threshold, the top layer sub-model is called to perform sorting processing on the to-be-processed image, and the obtained sorting result is used as the target sorting result, where the top layer sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-layer sub-models.

8. The method for sorting materials according to claim 1, characterized in that, The step of inputting the to-be-processed image into the material sorting model and determining the target sorting result corresponding to the to-be-sorted material by calling at least one layer of sub-models in the material sorting model to perform sorting processing on the to-be-processed image includes: Inputting the to-be-processed image into the material sorting model; In response to sufficient remaining idle resources, the highest-level sub-model in the material sorting model is called to perform sorting processing on the to-be-processed image, and the obtained sorting result is used as the target sorting result, where the highest-level sub-model is the sub-model with the largest resource occupancy and the highest sorting accuracy among the multi-level sub-models; In response to insufficient remaining idle resources, based on the hierarchical order of the sub-models in the material sorting model, each sub-model is called to perform sorting processing on the to-be-processed image until a sorting result with a confidence level greater than or equal to the corresponding threshold is obtained and used as the target sorting result corresponding to the material to be sorted.

9. The method for sorting materials according to claim 1, characterized in that, In the material sorting model, the lowest-level sub-model is calculated based on a traditional material sorting algorithm, and the other sub-models have different frameworks from the lowest-level sub-model and are trained based on an artificial intelligence algorithm.

10. The material sorting method according to claim 1, wherein The method further includes: Obtaining the actual sorting result of the material to be sorted; Optimizing the material sorting model based on the comparison result between the actual sorting result and the target sorting result.

11. The material sorting method according to claim 1 or 10, wherein The method further includes: In response to a received version update instruction, performing hot update processing on the model parameters corresponding to the material sorting model to obtain a target material sorting model.

12. A material sorting device, wherein Applied to a material sorting device, the device includes: An acquisition module, configured to acquire a to-be-processed image of a material to be sorted; A first processing module, configured to input the to-be-processed image into a material sorting model, perform sorting processing on the to-be-processed image by calling at least one layer of sub-models in the material sorting model, and determine and output a target sorting result corresponding to the material to be sorted, where the material sorting model includes different multi-level sub-models, the multi-level sub-models have the same function and are independent of each other, the high-level sub-models have a larger resource occupancy and a higher sorting accuracy than the low-level sub-models, and the sub-models are calculated based on a traditional material sorting algorithm or trained based on an artificial intelligence algorithm; A second processing module, configured to sort the material to be sorted according to the target sorting result.

13. A material sorting equipment, wherein Includes: A memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor performs the computer instructions to perform the material sorting method according to any one of claims 1-11.

14. A computer-readable storage medium storing the following program for performing the material sorting method according to any one of claims 1 - 11.

Images