Nut sorting method and device, electronic equipment and storage medium
The dynamic images of nuts are collected by multi-camera and sorted using convolutional neural network and Mamba model, which solves the problem of low detection accuracy of existing equipment and achieves more efficient and accurate nut sorting.
Patent Information
- Application Number
- CN202411302049.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing nut sorting equipment relies on static image analysis and lacks dynamic image sequence processing capabilities, resulting in low detection accuracy.
Multiple cameras are used to collect images of nuts from different angles, extract image features through convolutional neural networks, and combine features from different angles into sequence features, use pre-trained Mamba models for prediction, and combine adaptive adjustment mechanism to realize nuts accurate sorting.
It improves the accuracy and robustness of nut sorting, reduces misjudgment caused by factors such as occlusion and reflection, enhances the accuracy and stability of identification, and adapts to the nut needs of different batches and specifications through real-time feedback mechanism.
Smart Images

Figure CN119016358A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and more specifically, to a method, device, electronic device and storage medium for sorting nuts. Background Art
[0002] With the continuous improvement of industrial automation, higher requirements are put forward for product quality inspection and sorting. The traditional manual inspection method is not only inefficient, but also easily affected by subjective factors, resulting in unstable inspection results. Therefore, it is particularly important to develop an automated system that can efficiently and accurately sort the quality of ceramic tubes.
[0003] There are some screening devices for nuts on the market, but most of these devices rely on static image analysis and lack the ability to process dynamic image sequences, resulting in limited detection accuracy. As can be seen from the above, how to improve the accuracy of nut sorting remains to be solved. Summary of the invention
[0004] The present application provides a nut sorting method, device, electronic device and storage medium, which can solve the problem of low accuracy of nut sorting in the related art. The technical solution is as follows:
[0005] According to one aspect of the present application, a method for sorting nuts includes: obtaining original images of nuts to be identified at different angles based on multiple cameras installed on a sorting line, preprocessing the collected original images, and using a convolutional neural network to extract image features of the preprocessed original images; combining all image features of each nut to be identified into a sequence image feature, calling a pre-trained Mamba model, and inputting the sequence image features into the Mamba model for prediction, wherein each time point corresponds to the features of an image or a group of images; obtaining a corresponding result representation, and when the nuts are sent to a feeding area on the sorting line, controlling different collection mechanisms to collect the nuts in the feeding area corresponding to the sorting line based on the result representation, wherein the result representation includes good products, defective products and bad products.
[0006] In an exemplary embodiment, before the Mamba model is used for prediction, the method further includes inputting the sequence image features corresponding to the test nut into the Mamba model to obtain the corresponding result representation; using the result representation as the prediction result to obtain the true result corresponding to the test nut; matching the predicted result with the true result, and if there is a mismatch, adjusting the parameters of the Mamba model.
[0007] In an exemplary embodiment, in the process of extracting image features of the preprocessed original image using a convolutional neural network, the method also includes: setting multiple levels in the convolutional neural network, each level is responsible for extracting features of different scales, the shallow convolution layer extracts low-level features, and the deep convolution layer extracts higher-level abstract features, and the corresponding multi-scale image features are obtained based on the original image; or, in the convolutional neural network, using convolution kernels of different sizes to perform convolution operations on the original image to obtain corresponding multi-scale image features, wherein small convolution kernels capture detail features and large convolution kernels capture macro features; arranging the multi-scale image features into a sequence in chronological order to obtain corresponding sequence image features, wherein each time point corresponds to a feature of a viewing angle or a time point.
[0008] In an exemplary embodiment, in the process of inputting the sequence image features into the Mamba model for prediction, the method also includes: obtaining a probability distribution generated by the Mamba model, wherein the probability distribution corresponds to a classification label, and the classification label includes good products, defective products, and bad products; associating the classification label with the probability distribution to obtain a probability value corresponding to each classification label; and determining a corresponding result representation for the classification label with the largest probability value in the probability distribution.
[0009] In an exemplary embodiment, in the process of determining the corresponding result representation of the classification label with the largest probability value in the probability distribution, the method also includes: determining the classification label with the largest probability value in the probability distribution as the target label; calling the preset probability threshold of the target label; judging whether the probability threshold corresponding to the target label is greater than the probability threshold, and if so, determining the target label as the result representation.
[0010] In an exemplary embodiment, after determining the corresponding result representation, the method further includes: determining a set of defective result representations as defective products based on all the result representations; counting all the defective result representations in the set of defective result representations according to defective types; and providing feedback on the defective type corresponding to the count value reaching a preset value.
[0011] In an exemplary embodiment, the method also includes: the Mamba model also includes an adaptive adjustment mechanism. When a new nut is detected, a corresponding new original image is obtained, and the new original image is preprocessed, a convolutional neural network is used to extract key features of the new original image, all image features of each ceramic tube are combined into a sequence, and the serialized key features are input into the Mamba model. The Mamba model receives the new nut sample and updates its internal parameters through a gradient descent algorithm.
[0012] According to one aspect of the present application, a nut sorting device includes:
[0013] The original image acquisition module is used to acquire original images of nuts to be identified at different angles based on multiple cameras installed on the sorting line, preprocess the collected original images, and use a convolutional neural network to extract image features of the preprocessed original images;
[0014] A prediction module combines all image features of each nut to be identified into a sequence image feature, calls a pre-trained Mamba model, and inputs the sequence image features into the Mamba model for prediction, wherein each time point corresponds to a feature of an image or a group of images;
[0015] The collection module obtains the corresponding result representation. When the nuts are sent to the unloading area on the sorting line, different collection mechanisms are controlled based on the result representation to collect the nuts in the unloading area corresponding to the sorting line, wherein the result representation includes good products, defective products and bad products.
[0016] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the nut sorting method as described above.
[0017] According to one aspect of the present application, a storage medium stores computer-readable instructions thereon, and the computer-readable instructions are executed by one or more processors to implement the nut sorting method as described above.
[0018] According to one aspect of the present application, a computer program product includes computer-readable instructions, which are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the nut sorting method as described above.
[0019] The beneficial effects of the technical solution provided by this application are:
[0020] By collecting images from different angles using multiple cameras, comprehensive information about the nuts can be obtained, reducing misjudgments caused by occlusion, reflection and other factors, thereby improving sorting accuracy.
[0021] Combining image features from different angles into sequence features can make full use of multi-view information and improve recognition accuracy and robustness; at the same time, sequence features can also capture subtle differences in nuts at different perspectives, further improving sorting accuracy.
[0022] For example, the Mamba model can learn complex mapping relationships through training with a large amount of data, thereby accurately judging the classification results of nuts; compared with traditional classification methods, deep learning models have higher classification accuracy.
[0023] During the sorting process, the model can be fine-tuned or optimized based on real-time feedback results to meet the needs of sorting nuts of different batches and specifications. This real-time feedback mechanism helps to further improve the accuracy and stability of sorting. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.
[0025] Figure 1 It is a schematic diagram of the implementation environment involved in this application;
[0026] Figure 2 is a flow chart of a method for sorting nuts according to an exemplary embodiment;
[0027] Figure 3 is a flow chart from S001 to S003 in a method for sorting nuts according to an exemplary embodiment;
[0028] Figure 4 is a flow chart of S111 to S113 in a method for sorting nuts according to an exemplary embodiment;
[0029] Figure 5 is a flow chart of S114 to S116 in a method for sorting nuts according to an exemplary embodiment;
[0030] Figure 6 is a flow chart of S117 to S119 in a method for sorting nuts according to an exemplary embodiment;
[0031] Figure 7 is a flow chart of S121 to S123 in a method for sorting nuts according to an exemplary embodiment;
[0032] Figure 8 It is a structural block diagram of a nut sorting device according to an exemplary embodiment. DETAILED DESCRIPTION
[0033] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
[0034] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.
[0035] Among them, Mamba in the embodiment of the present application can process token sequences of up to one million levels, which is crucial for tasks that require long-term memory; Mamba's reasoning speed is extremely fast, 5 times faster than the traditional Transformer model, and can process more data in a shorter time; Mamba's scalability in sequence length is linear, that is, as the sequence length increases, the performance of the model will not be significantly affected. By solving the quadratic computational complexity problem that exists in traditional Transformers when processing long sequences, Mamba achieves high computing efficiency.
[0036] Figure 1 The schematic diagram of the implementation environment involved in a nut sorting method is shown in FIG. The implementation environment includes a collection end and a service end.
[0037] See also Figure 2 The present application embodiment provides a method for sorting nuts, which is applicable to electronic equipment, which may be Figure 1 The server side in the implementation environment is shown.
[0038] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.
[0039] like Figure 2 As shown, the method may include the following steps:
[0040] S100 obtains original images of nuts to be identified at different angles based on multiple cameras installed on the sorting line, preprocesses the collected original images, and uses a convolutional neural network to extract image features of the preprocessed original images.
[0041] Among them, using multiple cameras installed on the sorting line to obtain the original images of the nuts to be identified from different angles can ensure that comprehensive information about the nuts is obtained and reduce misjudgment caused by a single perspective; and the collected original images are preprocessed, including denoising, contrast enhancement, image correction, etc., to improve image quality; finally, a convolutional neural network is used to extract features from the preprocessed images. With its powerful feature extraction capabilities, the convolutional neural network can automatically learn high-level features from images that are useful for classification.
[0042] S110, combining all image features of each nut to be identified into a sequence image feature, calling a pre-trained Mamba model, and inputting the sequence image feature into the Mamba model for prediction.
[0043] Among them, each time point corresponds to the features of an image or a group of images. Combining image features from different angles into sequence features can make full use of multi-view information and improve recognition accuracy and robustness.
[0044] At the same time, the sequence features can also capture the subtle differences of nuts from different perspectives, further improving the accuracy of sorting; the Mamba model can learn complex mapping relationships through training with a large amount of data, so as to accurately judge the classification results of nuts. The Mamba model has higher classification accuracy and stronger generalization ability.
[0045] S120, obtaining a corresponding result representation, and when the nuts are sent to a feeding area on the sorting line, controlling different collecting mechanisms to collect the nuts in a feeding area corresponding to the sorting line based on the result representation.
[0046] The result characterization includes good products, inferior products and defective products.
[0047] In the above process, by collecting images of the nuts from multiple angles, we can have a more comprehensive understanding of the state of the nuts, thereby more accurately judging their quality; at the same time, the automated sorting process reduces dependence on manual inspection and improves production efficiency; through the prediction of the machine learning model, it is possible to effectively distinguish between good, defective and bad products to ensure the quality of the final product.
[0048] In addition, it should be pointed out that before using the Mamba model for prediction, during the training process of the Mamba model, it is necessary to adjust the parameters according to the prediction results of the Mamba model, such as Figure 3 As shown, the specific process also includes:
[0049] S001, inputting the sequence image features corresponding to the test nut into the Mamba model to obtain the corresponding result representation.
[0050] Among them, a part of nuts is selected as the test set, and the sequence image features of these test nuts are input into the Mamba model to obtain the predicted result representation (good products, defective products, and bad products).
[0051] S002, using the result characterization as a prediction result to obtain a true result corresponding to the test nut.
[0052] Among them, for each nut in the test set, its true result representation is manually labeled or determined through other reliable means. This step is to provide a standard answer for the model's prediction results for comparison.
[0053] S003, matching the predicted result with the actual result, if they do not match, adjusting the parameters of the Mamba model.
[0054] The predicted results of the model are compared with the actual results. If the predicted results are consistent with the actual results, it means that the model performs well on this part of the data. If they are inconsistent, it means that there is an error in the model. For mismatches, it is necessary to analyze the cause of the error and adjust the parameters of the Mamba model accordingly. The adjustment method can be retraining the model, modifying the model architecture, adjusting hyperparameters, etc.
[0055] By comparing the predicted results with the actual results, we can verify whether the model can achieve the expected performance level in actual applications, so that it can more accurately identify the quality status of nuts when actually deployed. In addition, by adjusting the model parameters according to the test results, the accuracy of sorting can be improved. Through continuous iterative optimization, the probability of the model misjudging the quality status of nuts can be reduced, thereby ensuring product quality control on the production line.
[0056] At the same time, in the process of using convolutional neural networks to extract image features of the original image after preprocessing, such as Figure 4 As shown, the method also includes:
[0057] S111, multiple layers are set in the convolutional neural network, each layer is responsible for extracting features of different scales, and corresponding multi-scale image features are obtained based on the original image.
[0058] Among them, shallow convolution layers extract low-level features, and deep convolution layers extract higher-level abstract features; multiple levels are mainly used to extract basic features of images, such as low-level features such as edges and textures, which are usually the basic building blocks of images; as the levels deepen, the network begins to learn higher-level abstract features, such as more complex structures such as the shape of the nut and defect patterns.
[0059] S112, or, in a convolutional neural network, using convolution kernels of different sizes to perform convolution operations on the original image to obtain corresponding multi-size image features.
[0060] Among them, small convolution kernels capture detailed features, such as local defects or small structures; large convolution kernels can capture features in a larger range, such as overall shape or distribution patterns, which helps to understand the overall structure of the image.
[0061] S113, arranging the multi-scale image features into a sequence in chronological order, and obtaining corresponding sequence image features.
[0062] Among them, the features extracted by convolution kernels of different levels or different sizes are integrated to form a multi-scale image feature representation, which are arranged in chronological order to form a sequence image feature, and each time point corresponds to a perspective or a feature of a time point.
[0063] Through the above process, multi-scale feature extraction can be performed so that the network can capture both detail information and global information in the image, thereby improving the richness of feature expression; the convolutional neural network can not only capture subtle differences in nut images, but also understand their overall shape and structure, thereby making more accurate judgments during the sorting process.
[0064] For the Mamba model in the embodiment of the present application, the Mamba model also includes an adaptive adjustment mechanism. When a new nut is detected, the corresponding new original image is obtained, and the new original image is preprocessed. The key features of the new original image are extracted using a convolutional neural network, and all image features of each ceramic tube are combined into a sequence. The serialized key features are input into the Mamba model. The Mamba model receives the new nut sample and updates its internal parameters through a gradient descent algorithm.
[0065] The adaptive adjustment mechanism enables the Mamba model to continuously learn new samples, thereby gradually optimizing its performance. By updating the Mamba model parameters online, it can quickly adapt to new nut types or quality changes, ensuring the continuous and efficient operation of the sorting system. For nut identification, when the system detects new nuts, it can immediately process them and feedback the results, enhancing the real-time performance of the system. Through the adaptive adjustment mechanism, the Mamba model can continuously optimize itself to cope with the changes in various nut samples that may occur in the production environment, thereby ensuring the stability and accuracy of the sorting process.
[0066] In the process of Mamba model prediction, if Figure 5 As shown, the method also includes:
[0067] S114, obtaining the probability distribution generated by the Mamba model.
[0068] Among them, the probability distribution corresponds to the classification label, and the classification label includes good products, defective products, and bad products;
[0069] S115, associating the classification label with the probability distribution, and obtaining the probability value corresponding to each classification label.
[0070] S116, determining the corresponding result representation for the classification label with the largest probability value in the probability distribution.
[0071] By selecting the classification label with the highest probability value as the final result, it is possible to clearly indicate which category the input sample belongs to, thereby achieving quality judgment of the nuts. In addition to giving the classification result, the probability distribution also provides the probability value of the model for the classification result; outputting the classification result in the form of probability helps the subsequent decision-making process.
[0072] For example, on an automated production line, different processing flows can be set up based on different probability thresholds. For example, high-probability “good products” can go directly to the next process, while lower-probability products require manual re-inspection.
[0073] It should be pointed out that in the process of determining the corresponding result representation based on the classification label with the largest probability value, it is necessary to make a further determination, such as Figure 6 As shown, the specific process includes:
[0074] S117, determining the classification label with the largest probability value in the probability distribution as the target label.
[0075] S118, retrieving the preset probability threshold of the target tag.
[0076] Among them, the probability threshold corresponding to each classification label is different. The system can be preset or the corresponding numerical adjustment can be made later.
[0077] S119, determining whether the probability threshold corresponding to the target tag is greater than the probability threshold, and if so, determining the target tag as the result representation.
[0078] By setting a probability threshold, we can filter out those classification results that the model is not very sure of, thereby improving the overall classification accuracy. If the actual maximum probability value of a sample is lower than the set threshold, this indicates that the model is not sure enough about the classification of the sample. In this case, not taking it as the final result can avoid misjudgment due to model uncertainty. In some cases, if the model fails to reach the threshold, it can trigger further inspection procedures or manual review to ensure product quality.
[0079] Finally, in the process of nut sorting, each time the corresponding characterization results are obtained, such as Figure 7 As shown, the following steps are also performed:
[0080] S121, determining a set of defective result representations as defective products based on all the result representations.
[0081] S122, counting all the bad result representations in the bad result representation set according to the bad type.
[0082] Among them, by classifying and counting defective products, the main types of problems in the production process can be quickly identified; for example, if the number of nuts with "unqualified size" is found to exceed the preset value, then this may be due to problems with the production equipment or process.
[0083] S123, feedback is given on the defect type corresponding to the count value reaching the preset value.
[0084] When the number of a certain type of defective product reaches a preset value, the system will automatically provide feedback to the relevant departments so that the problem can be noticed and handled in a timely manner, which helps prevent a large number of defective products from flowing into the next production link or final product. Through the feedback mechanism, relevant departments can be prompted to take preventive measures, such as adjusting production processes, calibrating equipment, strengthening employee training, etc., thereby reducing the number of defective products.
[0085] The following is an embodiment of the device of the present application, which can be used to perform the nut sorting method involved in the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the nut sorting method involved in the present application.
[0086] See also Figure 8 In the embodiments of the present application, a nut sorting device is provided, including but not limited to:
[0087] The original image acquisition module 200 is used to acquire original images of nuts to be identified at different angles based on multiple cameras installed on the sorting line, preprocess the collected original images, and use a convolutional neural network to extract image features of the preprocessed original images;
[0088] The prediction module 210 combines all the image features of each nut to be identified into a sequence image feature, and calls a pre-trained Mamba model to input the sequence image features into the Mamba model for prediction, wherein each time point corresponds to a feature of an image or a group of images;
[0089] The collection module 220 obtains the corresponding result representation. When the nuts are sent to the unloading area on the sorting line, different collection mechanisms are controlled based on the result representation to collect the nuts in the unloading area corresponding to the sorting line, wherein the result representation includes good products, defective products and bad products.
[0090] In an exemplary embodiment, the device further includes but is not limited to:
[0091] A training result representation acquisition module is used to input the sequence image features corresponding to the test nut into the Mamba model to obtain the corresponding result representation;
[0092] A real result acquisition module uses the result representation as a prediction result to obtain the real result corresponding to the test nut;
[0093] The matching module is used to match the predicted results with the actual results. If they do not match, the parameters of the Mamba model are adjusted.
[0094] In an exemplary embodiment, the device further includes but is not limited to:
[0095] The multi-scale image feature acquisition module sets multiple layers in the convolutional neural network. Each layer is responsible for extracting features of different scales. The shallow convolution layer extracts low-level features, and the deep convolution layer extracts more advanced abstract features. It is used to obtain the corresponding multi-scale image features based on the original image.
[0096] Or, in a convolutional neural network, convolution operations are performed on the original image using convolution kernels of different sizes to obtain corresponding multi-scale image features, where small convolution kernels capture detail features and large convolution kernels capture macro features;
[0097] The sequence image feature acquisition module arranges the multi-scale image features into a sequence in chronological order to obtain the corresponding sequence image features, wherein each time point corresponds to a viewing angle or a feature of a time point.
[0098] In an exemplary embodiment, the device further includes but is not limited to:
[0099] The probability distribution acquisition module is used to obtain the probability distribution generated by the Mamba model, where the probability distribution corresponds to the classification label, and the classification label includes good products, defective products, and bad products;
[0100] The probability value acquisition module associates the classification label with the probability distribution to obtain the probability value corresponding to each classification label;
[0101] The result representation determination module uses the classification label with the largest probability value in the probability distribution to determine the corresponding result representation.
[0102] In an exemplary embodiment, the device further includes but is not limited to:
[0103] A target label determination module determines the classification label with the largest probability value in the probability distribution as the target label;
[0104] A probability threshold calling module is used to call the preset probability threshold of the target tag;
[0105] The judgment module is used to judge whether the probability threshold corresponding to the target label is greater than the probability threshold. If so, the target label is determined as the result representation.
[0106] In an exemplary embodiment, the device further includes but is not limited to:
[0107] A bad result representation set determination module, based on a bad result representation set used to determine a bad product among all the result representations;
[0108] A counting module, used for counting all the bad result representations in the bad result representation set according to the bad type;
[0109] The feedback module is used to provide feedback on the defect type corresponding to the count value reaching the preset value.
[0110] In an exemplary embodiment, the device further includes but is not limited to:
[0111] The Mamba model also includes an adaptive adjustment mechanism. When a new nut is detected, the corresponding new original image is obtained and preprocessed. The key features of the new original image are extracted using a convolutional neural network. All image features of each ceramic tube are combined into a sequence, and the serialized key features are input into the Mamba model. The Mamba model receives the new nut sample and updates its internal parameters through the gradient descent algorithm.
[0112] It should be noted that the nut sorting device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example to illustrate the sorting of nuts. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the nut sorting device will be divided into different functional modules to complete all or part of the functions described above.
[0113] In addition, the nut sorting device and the nut sorting method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.
[0114] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the nut sorting method as described above.
[0115] In addition, a storage medium is provided in an embodiment of the present application, on which computer-readable instructions are stored. The computer-readable instructions are executed by one or more processors to implement the nut sorting method as described above.
[0116] A computer program product is provided in an embodiment of the present application. The computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the nut sorting method as described above.
[0117] Compared with related technologies, multiple cameras are used to capture images of the nuts to be identified from different angles to ensure that the information of the nuts is fully captured; different levels of CNN are used to extract multi-scale features from low-level to high-level to enhance the model's ability to recognize different types of defects; the image features of each nut obtained from different perspectives or time points are combined into a sequence to provide continuous time dimension information for the Mamba model.
[0118] The pre-trained Mamba model is used to predict the features of the sequence images to obtain the classification label of each nut (good, defective or bad). At the same time, the probability distribution output by the Mamba model is used to determine the final classification result. At the same time, a threshold can be set to filter out prediction results with high uncertainty.
[0119] By comparing the predicted results of the test nuts with the actual results, the Mamba model is fine-tuned to ensure the accuracy of the model; the types of defective products are counted and counted. When the number of defective products of a certain type exceeds the preset threshold, an alarm is issued to the relevant personnel or system so that timely measures can be taken.
[0120] Finally, the Mamba model has the ability to self-learn and can adapt to the changing data distribution by receiving new sample data and updating its own parameters using the gradient descent algorithm.
[0121] In summary, the present application scheme realizes the automation of nut sorting, improves sorting efficiency and accuracy, and also provides a feedback mechanism for quality control and the adaptive optimization function of the model.
[0122] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0123] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for sorting nuts, characterized in that: include: Based on multiple cameras installed on the sorting line, original images of nuts to be identified at different angles are obtained, the collected original images are preprocessed, and the image features of the preprocessed original images are extracted using a convolutional neural network; Combining all image features of each nut to be identified into a sequence image feature, calling a pre-trained Mamba model, and inputting the sequence image features into the Mamba model for prediction, wherein each time point corresponds to a feature of an image or a group of images; Obtain corresponding result representations, and when the nuts are sent to the unloading area on the sorting line, control different collecting mechanisms based on the result representations to collect the nuts in the unloading area corresponding to the sorting line, wherein the result representations include good products, defective products and bad products.
2. The method according to claim 1, characterized in that Before using the Mamba model for prediction, the method further includes: Input the sequence image features corresponding to the test nut into the Mamba model to obtain the corresponding result representation; Using the result characterization as a prediction result, obtaining a true result corresponding to the test nut; The predicted results are matched with the actual results. If they do not match, the parameters of the Mamba model are adjusted.
3. The method according to claim 2, characterized in that In the process of extracting image features of the preprocessed original image using the convolutional neural network, the method further includes: Multiple layers are set in the convolutional neural network, each layer is responsible for extracting features of different scales, the shallow convolution layer extracts low-level features, and the deep convolution layer extracts more advanced abstract features, and the corresponding multi-scale image features are obtained based on the original image; Or, in a convolutional neural network, convolution kernels of different sizes are used to perform convolution operations on the original image to obtain corresponding multi-scale image features, where small convolution kernels capture detail features and large convolution kernels capture macro features; The multi-scale image features are arranged into a sequence in time order to obtain corresponding sequence image features, wherein each time point corresponds to a viewing angle or a feature of a time point.
4. The method according to claim 2, characterized in that In the process of inputting the sequence image features into the Mamba model for prediction, the method further includes: Obtain the probability distribution generated by the Mamba model, wherein the probability distribution corresponds to a classification label, and the classification label includes good products, defective products, and bad products; Associating the classification label with the probability distribution to obtain a probability value corresponding to each classification label; The classification label with the largest probability value in the probability distribution is determined to correspond to the result representation.
5. The method according to claim 1, characterized in that In the process of determining the corresponding result representation of the classification label with the largest probability value in the probability distribution, the method further includes: Determine the classification label with the largest probability value in the probability distribution as the target label; Retrieve the preset probability threshold of the target tag; It is determined whether the probability threshold corresponding to the target tag is greater than the probability threshold. If so, the target tag is determined as the result representation.
6. The method according to claim 1, characterized in that After determining the corresponding result representation, the method further includes: A set of defective result representations determined as defective products based on all of the result representations; Count all the bad result representations in the bad result representation set according to the bad type; When the count value reaches the preset value, the corresponding defect type will be fed back.
7. The method according to claim 1, characterized in that The method also includes: The Mamba model also includes an adaptive adjustment mechanism. When a new nut is detected, a corresponding new original image is obtained, and the new original image is preprocessed. A convolutional neural network is used to extract key features of the new original image, and all image features of each ceramic tube are combined into a sequence. The serialized key features are input into the Mamba model. The Mamba model receives the new nut sample and updates its internal parameters through a gradient descent algorithm.
8. A nut sorting device, characterized in that: include: The original image acquisition module is used to acquire original images of nuts to be identified at different angles based on multiple cameras installed on the sorting line, preprocess the collected original images, and use a convolutional neural network to extract image features of the preprocessed original images; A prediction module combines all image features of each nut to be identified into a sequence image feature, calls a pre-trained Mamba model, and inputs the sequence image features into the Mamba model for prediction, wherein each time point corresponds to a feature of an image or a group of images; The collection module obtains the corresponding result representation. When the nuts are sent to the unloading area on the sorting line, different collection mechanisms are controlled based on the result representation to collect the nuts in the unloading area corresponding to the sorting line, wherein the result representation includes good products, defective products and bad products.
9. An electronic device, characterized in that: include: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the method for sorting nuts according to any one of claims 1 to 7.
10. A storage medium having computer-readable instructions stored thereon, characterized in that: The computer-readable instructions are executed by one or more processors to implement the nut sorting method according to any one of claims 1 to 7.
