Model searching method for sorting machine, sorting method and electronic equipment

Abstract

The invention provides a model searching method for a sorting machine, a sorting method and electronic equipment, and belongs to the field of data processing.The model searching method comprises the steps that searching sampling is conducted on a preset searching space, and a neural network structure and a hyper-parameter and an optimization algorithm corresponding to the neural network structure are obtained; selecting candidate network structures from the neural network structures according to the scale values of the neural network structures; based on the hyper-parameters of the candidate network structures and an optimization algorithm, performing optimization evaluation on the candidate network structures by using the plurality of data sets of different materials to obtain performance indexes of the candidate network structures; and determining an optimal network structure from the candidate network structures according to the performance indexes. Therefore, the optimal network structure is a model which has small scale values such as calculated amount and parameter amount and has optimal recognition performance on the multiple materials of the sorting machine, so that the optimal network structure has wide adaptability and has higher recognition precision and higher reasoning speed.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a model searching method, a sorting method and an electronic device for a sorting machine. Background Art

[0002] Color sorters, also known as sorting machines, are important equipment widely used in agriculture, food processing, mineral resources and other industries. Their core function is to achieve efficient and automated sorting and screening of materials by automatically identifying and classifying physical properties such as color, texture, and contour. Traditional color sorters mainly rely on preset rules and manually constructed feature extraction mechanisms. Although this approach is practical, it is limited by its inherent pattern matching capabilities and is difficult to adapt to the complex and variable nature of material characteristics in terms of type and form.

[0003] Recently, deep learning technology has shown excellent performance in the field of image processing, and neural network models have been widely used in image classification and object detection tasks due to their powerful learning and generalization capabilities. However, the neural network models currently used in sorting machines cannot take into account both inference speed and accuracy. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a model search method, a sorting method and an electronic device for a sorting machine, so as to achieve both the inference speed and the accuracy of the neural network model applied to the sorting machine.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a model search method for a sorting machine, the method comprising:

[0007] Searching and sampling a preset search space to obtain search results; wherein the search results include a plurality of neural network structures and hyperparameters and optimization algorithms corresponding to each of the neural network structures;

[0008] Calculating a scale value of the neural network structure when performing a forward inference, and selecting a candidate network structure from each of the neural network structures according to the scale value;

[0009] For each candidate network structure, based on the hyperparameters and optimization algorithm of the candidate network structure, multiple data sets are used to optimize and evaluate the candidate network structure to obtain performance indicators of the candidate network structure; wherein each data set is a data set of a material of the sorting machine;

[0010] According to the performance indicators, an optimal network structure is determined from the candidate network structures.

[0011] Optionally, the step of optimizing and evaluating the candidate network structure using multiple data sets based on the hyperparameters and optimization algorithm of the candidate network structure to obtain the performance index of the candidate network structure includes:

[0012] For each of the data sets, according to the settings of the hyperparameters of the candidate network structure, using the data set and the optimization algorithm of the candidate network structure, iteratively training the candidate network structure, and evaluating the intermediate model obtained from each round of iterative training to obtain the performance value of the candidate network structure on the data set;

[0013] According to the performance value of the candidate network structure on each of the data sets, a performance indicator of the candidate network structure is obtained.

[0014] Optionally, the data set includes a training set and a test set, and the hyperparameters include model structure parameters and training strategies;

[0015] The step of iteratively training the candidate network structure according to the setting of the hyperparameters of the candidate network structure and using the optimization algorithm of the data set and the candidate network structure for each data set, and evaluating the intermediate model obtained from each round of iterative training to obtain the performance value of the candidate network structure on the data set comprises:

[0016] According to the model structure parameters and the candidate network structure, an initial model to be trained is obtained;

[0017] According to the training strategy and the optimization algorithm of the candidate network structure, using the training set, performing a round of iterative training on the model to be trained to obtain an intermediate model;

[0018] Using the test set, performing performance evaluation on the intermediate model to obtain model performance of the intermediate model;

[0019] Updating the number of iterations, and determining whether the number of iterations is equal to a preset iteration threshold;

[0020] If not, use the intermediate model as the next model to be trained, and return to the step of executing the optimization algorithm according to the training strategy and the candidate network structure, using the training set, performing a round of iterative training on the model to be trained, and obtaining an intermediate model;

[0021] If so, the performance value of the candidate network structure on the data set is obtained according to the performance of each model.

[0022] Optionally, the step of obtaining the performance value of the candidate network structure on the data set according to the performance of each model includes:

[0023] From the model performances, the optimal model performance is selected as the performance value of the candidate network structure on the data set.

[0024] Optionally, the step of obtaining a performance indicator of the candidate network structure according to the performance value of the candidate network structure on each of the data sets includes:

[0025] An average value is taken for the performance values ​​of the candidate network structure on each of the data sets, and the average value is used as the performance indicator of the candidate network structure.

[0026] Optionally, the scale value includes a calculation amount and a parameter amount;

[0027] The step of selecting a candidate network structure from each of the neural network structures according to the scale value comprises:

[0028] For each of the neural network structures, when the computational amount of the neural network structure is less than a preset first threshold and the parameter amount is less than a preset second threshold, the neural network structure is taken as a candidate network structure.

[0029] Optionally, the step of performing search sampling on a preset search space to obtain search results includes:

[0030] A grid search algorithm is used to search and sample a preset search space to obtain search results; wherein the search space includes multiple convolution kernels, activation functions, the number of convolution kernels of the convolution layer, the configuration method of the convolution module, hyperparameters and optimization algorithms.

[0031] Optionally, before the step of searching and sampling the preset search space to obtain search results, the method further includes:

[0032] Acquire original images of various materials of the sorting machine, determine the sorting category of the original images according to the material sorting standard, and annotate the original images according to the sorting category to obtain annotated images;

[0033] Performing data augmentation and normalization operations on each of the labeled images to obtain multiple sample images;

[0034] For each type of the material, each sample image of the material is taken as a data set, and each sample image in the data set is divided into a training set and a test set according to a preset ratio.

[0035] In a second aspect, the present invention provides a sorting method, which is applied to a control device of a sorting machine, wherein the control device is communicatively connected with a sorting spray valve of the sorting machine, and the method comprises:

[0036] Input the currently acquired material image into a preset detection model, and generate a detection result using the detection model; wherein the detection model is a model obtained by training the searched optimal network structure, and the optimal network structure is obtained by using the model search method for the sorting machine as described in the first aspect;

[0037] Based on the detection result, the sorting spray valve is controlled to sort the materials in the sorting machine.

[0038] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the model search method for a sorting machine as described in the first aspect, or the sorting method as described in the second aspect.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the model search method for a sorting machine as described in the first aspect, or the sorting method as described in the second aspect is implemented.

[0040] The model search method, sorting method and electronic device for a sorting machine provided by an embodiment of the present invention include: searching and sampling a preset search space to obtain search results, the search results include multiple neural network structures and hyperparameters and optimization algorithms corresponding to each neural network structure; selecting a candidate network structure from each neural network structure according to the scale value of the neural network structure when performing a forward reasoning; for each candidate network structure, based on the hyperparameters and optimization algorithms of the candidate network structure, using multiple data sets to optimize and evaluate the candidate network structure respectively, to obtain the performance index of the candidate network structure, each data set is a data set of a material of the sorting machine; according to the performance index, the optimal network structure is determined from each candidate network structure. In this way, the optimal network structure obtained is a model with small scale values ​​such as calculation amount and parameter amount, and the best recognition performance for multiple materials of the sorting machine, so that the optimal network structure has wide adaptability and achieves the effect of having higher recognition accuracy and faster reasoning speed. In addition, by automatically searching for neural network structures, the cost of manual design is greatly reduced.

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 A block diagram of a sorting system provided by an embodiment of the present invention is shown.

[0044] Figure 2 The structure diagram of the sorting machine provided by the embodiment of the present invention is shown.

[0045] Figure 3 Shows Figure 2 Enlarged view of part A.

[0046] Figure 4 A block diagram of an electronic device provided by an embodiment of the present invention is shown.

[0047] Figure 5 One of the flow charts of the model search method for a sorting machine provided by an embodiment of the present invention is shown.

[0048] Figure 6 Shows Figure 5 Schematic diagram of the process flow of some sub-steps of step 15.

[0049] Figure 7 Shows Figure 6 A flowchart of some sub-steps of step 151.

[0050] Figure 8 The second flowchart of the model search method for a sorting machine provided in an embodiment of the present invention is shown.

[0051] Fig. 9 A schematic flow chart of a sorting method provided in an embodiment of the present invention is shown.

[0052] Fig.10 A structural diagram of the network structure obtained using the model search method for the sorting machine is shown.

[0053] Fig.11 A block diagram of a model search device provided by an embodiment of the present invention is shown.

[0054] Explanation of the reference numerals: 10 - sorting system; 110 - model search device; 120 - sorting machine; 121 - machine body; 122 - sorting spray valve; 123 - control device; 124 - camera; 125 - good product chamber; 126 - waste chamber; 127 - unloading channel; 20 - electronic equipment; 210 - memory; 220 - processor; 230 - communication module; 30 - model search device; 310 - search module; 320 - first selection module; 330 - performance evaluation module; 340 - second selection module. DETAILED DESCRIPTION

[0055] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0057] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0058] The model search method and sorting method for the sorting machine 120 provided in the embodiment of the present invention can be applied to Figure 1 In the sorting system 10 shown, reference is made to Figures 1 to 3 The sorting system 10 includes a model search device 110 and a sorting machine 120. The sorting machine 120 includes a body 121, a sorting spray valve 122, a control device 123 and a camera 124. The control device 123 is respectively communicatively connected with the sorting spray valve 122, the camera 124 and the model search device 110, wherein the communication connection can be achieved through data line connection, network, wireless communication and the like.

[0059] A plurality of material discharge channels 127 corresponding to the cameras 124 are provided inside the body 121 of the sorting machine 120, a good material chamber 125 is provided below the material discharge channel 127, a waste material chamber 126 is provided on one side of the good material chamber 125, a material drop zone is provided between the material drop port of the material discharge channel 127 and the good material chamber 125, a sorting spray valve 122 is provided in the material drop zone in the body 121, and the camera 124 is provided in the body 121, and the camera 124 faces the corresponding material discharge channel 127.

[0060] The model search device 110 is used to implement the model search method for the sorting machine 120 provided in an embodiment of the present invention, and to train the optimal network structure obtained by the model search method, and to install the trained detection model to the control device 123 of the sorting machine 120.

[0061] The camera 124 is used to photograph the corresponding material discharge channel 127 to obtain a material image, and transmit the material image to the control device 123 in real time.

[0062] The control device 123 is used to implement the sorting method provided in the embodiment of the present invention to control the sorting spray valve 122 to work and sort the materials in the sorting machine 120 .

[0063] Among them, the model search device 110 can be but is not limited to: a personal computer, an independent server or a server cluster, and the control device 123 can be but is not limited to: a single chip microcomputer (MCU), a field programmable gate array (Field Programmable Gate Array, FPGA), a programmable logic controller (PLC), a computer or any embedded device.

[0064] Please refer to Figure 4 , is a block diagram of an electronic device 20, which may be Figure 1 The model search device 110 in the sorting system 10 shown may also be Figure 1 The control device 123 in the sorting system 10 shown. The electronic device 20 includes a memory 210, a processor 220 and a communication module 230. The memory 210, the processor 220 and the communication module 230 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0065] The memory 210 is used to store programs or data, and can be, but not limited to, a random access memory 210, a read-only memory 210, a programmable read-only memory 210, an erasable read-only memory 210, an electrically erasable read-only memory 210, and the like.

[0066] The processor 220 is used to read / write data or programs stored in the memory 210 and execute corresponding functions. For example, Figure 1 In the sorting system 10 shown, the processor 220 of the model search device 110 executes a computer program stored in the memory 210 to implement a model search method for the sorter 120, or the processor 220 of the control device 123 executes a computer program stored in the memory 210 to implement a sorting method.

[0067] The communication module 230 is used to establish a communication connection between the electronic device 20 and other communication terminals, and to send and receive data. Figure 1 In the sorting system 10 shown, the communication module 230 of the model search device 110 sends and receives data with the control device 123 through the network, and the control device 123 can send and receive data with the camera 124 and the sorting spray valve 122 through the data line.

[0068] It should be understood that Figure 4 The structure shown is only a schematic diagram of the structure of the electronic device 20. The electronic device 20 may also include Figure 4 More or fewer components as shown, or with Figure 4 Different configurations shown. Figure 4 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0069] Please refer to Figure 5 The embodiment of the present invention provides a model search method for a sorting machine 120, comprising steps 11 to 17, in which Figures 1 to 3 In the sorting system 10 shown, the model search device 110 can Figure 2 The structure shown implements step 11 to step 17 when the processor 220 executes the computer program stored in the memory 210.

[0070] Step 11, search and sample the preset search space to obtain search results.

[0071] The search results include multiple neural network structures and the hyperparameters and optimization algorithms corresponding to each neural network structure.

[0072] Step 13, calculate the scale value of the neural network structure when performing a forward reasoning, and select a candidate network structure from each neural network structure based on the scale value.

[0073] Step 15, for each candidate network structure, based on the hyperparameters and optimization algorithm of the candidate network structure, multiple data sets are used to optimize and evaluate the candidate network structure to obtain the performance indicators of the candidate network structure.

[0074] Each data set is a data set of a material of the sorting machine 120 .

[0075] Step 17: Determine the optimal network structure from the candidate network structures according to the performance indicators.

[0076] It should be noted that the search space includes any information related to building a neural network model, such as the convolution kernel, activation function, attention network, number of convolution kernels, configuration of convolution modules, hyperparameters, and optimization algorithm. The scale value may include any one or more information that affects the model inference speed, such as the amount of calculation and the amount of parameters.

[0077] For example, in combination Figure 1 For the sorting system 10 shown, developers can prepare data sets in advance, such as controlling the camera 124 to take pictures of the materials in the unloading channel 127 of the sorting machine 120, obtain material images, and construct a data set for each material through data annotation. Among them, there can be 8 kinds of materials, such as peppers, peppers, Chinese medicine, sponge titanium, shrimps, ore, tea and X-ray macadamia nuts, or 6 kinds of materials, or 9 kinds of materials, and the number of material types is not limited. At the same time, the search space is pre-constructed. Thus, the search space and the data set are input into the model search device 110 to prepare the data.

[0078] After the search space and data set are prepared, the model search device 110 searches and samples the search space to obtain all possible neural network structures and the hyperparameters and optimization algorithms corresponding to each neural network structure. Next, the model search device 110 allows each neural network structure to perform a forward inference and calculates the scale value of the forward inference, and then deletes the neural network structure whose scale value does not reach the expected value according to the scale value to obtain the candidate network structure.

[0079] Then, the model search device 110 optimizes and evaluates each candidate network structure based on the hyperparameters and optimization algorithm of the candidate network structure using multiple pre-selected data sets to obtain the performance index of the candidate network structure. Furthermore, the model search device 110 determines the optimal network structure from each candidate network structure based on the performance index.

[0080] In the above-mentioned model search method for the sorting machine 120, a variety of neural network structures are first generated by automatic search, which greatly reduces the cost of manual design. Then, the neural network structure is screened according to the scale value, and the optimization evaluation of each selected candidate network structure is performed to screen out the optimal network structure with small scale values ​​such as the amount of calculation and the amount of parameters, and the best recognition performance for multiple materials of the sorting machine 120, so that the optimal network structure has wide adaptability and achieves the effect of having higher recognition accuracy and faster reasoning speed.

[0081] Among them, in order to implement the above steps 11 to 17, a search space can be pre-constructed according to needs. The search space may include a variety of convolution kernels, activation functions, the number of convolution kernels of the convolution layer, the configuration method of the convolution module, hyperparameters, optimization algorithms and any other factors required to build a neural network structure.

[0082] For example, the search space includes design options for convolutional layers, configuration methods for convolutional modules, parameter options for parameters such as the number of channels, convolution kernel size, stride, etc. of each module, as well as hyperparameters and optimization algorithms.

[0083] The design options for convolutional layers can include various kernel sizes, various activation functions, and various numbers of kernels.

[0084] For example, the convolution kernel size of each convolution layer can be selected from the following three options: 1×1, 3×3, and 5×5. The optional activation functions of each convolution layer include ReLU function, ReLU6 function, Leaky ReLU function, and HardSwish function. The number of convolution kernels in each convolution layer can be selected from 16, 32, 64, 128, and 256.

[0085] The configuration of the convolution module can include options such as traditional modules, advanced modules, and attention mechanisms.

[0086] The traditional module is a combination of traditional convolution layer + pooling layer + activation function.

[0087] Advanced modules, that is, you can choose more advanced structures, such as depth-separable convolution modules and inverted residual modules.

[0088] Attention mechanism, that is, an attention mechanism module can be randomly added to each convolution module to improve the feature extraction ability of the model.

[0089] The parameter options of each module, that is, the multiple values ​​of various parameters, such as the number of channels, convolution kernel size, stride and other parameters of each module can be flexibly selected within the preset range.

[0090] The hyperparameters and optimization algorithm options are the same as above and will not be repeated here.

[0091] Furthermore, in step 11, a variety of search sampling methods can be used to sample the search space to obtain all possible search results. For example, a brute force search algorithm or a random search algorithm can be used, and the above two methods are only examples, and their implementation methods are not limited.

[0092] In order to obtain all possible neural network structures as much as possible, a grid search is introduced in the search sampling in step 11. For example, in step 11, a grid search algorithm is used to search and sample a preset search space to obtain search results.

[0093] By using grid search, all possible neural network structures are obtained by exhaustively traversing various types of options in the search space.

[0094] After all possible neural network structures are obtained in the above manner, in step 13, the model search device 110 first allows each neural network structure to perform forward reasoning, that is, an input is given to the neural network structure, and an output is generated using the neural network structure, thereby calculating the scale value of the neural network structure in the process, and the scale value at least includes the calculation amount and the parameter amount.

[0095] After obtaining the scale value of each neural network structure, in order to ensure that the reasoning speed meets the expected requirements, in step 13, the neural network structures whose scale values ​​exceed the expected value can be deleted, and the remaining neural network structures can be used as candidate neural networks.

[0096] By way of example, the process of selecting a candidate network structure in step 13 can be implemented as follows: for each neural network structure, when the computational amount of the neural network structure is less than a preset first threshold and the parameter amount is less than a preset second threshold, the neural network structure is selected as a candidate network structure.

[0097] The first threshold represents the upper limit of the amount of computation, and the second threshold represents the upper limit of the amount of parameters. Moreover, the specific values ​​of the first threshold and the second threshold can be adjusted according to the required reasoning speed and classification requirements. For example, the first threshold can be 2MFLOPs, and the second threshold can be 0.05M. The first threshold can also be 3MFLOPs, and the second threshold can also be 0.06M. The specific values ​​of the two are not limited.

[0098] After all candidate network structures are selected, the method of obtaining the performance index of the candidate network structure in step 15 can be flexibly selected. For example, a data set can be arbitrarily selected to train the candidate network structure, and the performance index of the trained model can be used as the performance index of the candidate network structure. The performance index of the candidate network structure can also be determined by using preset rules. The above two methods are only examples, and the implementation method is not limited.

[0099] In order to make the performance indicators reflect the generalization and performance of the candidate network structure, so that the optimal network structure can not only take into account the inference speed and accuracy, but also have a wide range of applications in the sorting field, that is, have a higher generalization ability, the training evaluation results of multiple data sets are introduced in step 15 to determine the concept of the final performance of the candidate network structure. Figure 6 , step 15 can be implemented as steps 151 to 153.

[0100] Step 151, for each data set, according to the setting of the hyperparameters of the candidate network structure, using the optimization algorithm of the data set and the candidate network structure, iteratively train the candidate network structure, and evaluate the intermediate model obtained from each round of iterative training to obtain the performance value of the candidate network structure on the data set.

[0101] Step 153, obtaining the performance index of the candidate network structure according to the performance value of the candidate network structure on each data set.

[0102] Among them, the data set includes training set and test set, and the hyperparameters include model structure parameters and training strategies.

[0103] In steps 151 to 153, for each candidate network structure, each data set is used to train and evaluate the candidate network structure to obtain the performance value of the candidate network structure on each data set, and then the performance index of the candidate network structure is determined by combining the various performance values ​​of the candidate network structure, so that the performance index can reflect the reasoning precision (i.e., accuracy) and generalization ability of the candidate network structure.

[0104] Assume that the candidate network structures include network structure A, and there are 8 data sets, which are the data sets corresponding to pepper, pepper, Chinese medicine, sponge titanium, shrimp, ore, tea and X-ray macadamia. For network structure A, the data sets of pepper, pepper, Chinese medicine, sponge titanium, shrimp, ore, tea and macadamia are used to train and evaluate network structure A, and the performance value of network structure A on the pepper data set is 1, the performance value of network structure A on the pepper data set is 2, the performance value of network structure A on the Chinese medicine data set is 3, the performance value of network structure A on the sponge titanium data set is 4, the performance value of network structure A on the shrimp data set is 5, the performance value of network structure A on the ore data set is 6, the performance value of network structure A on the tea data set is 7, and the performance value of network structure A on the macadamia data set is 8. Then, according to the performance value 1, performance value 2, performance value 3, performance value 4, performance value 5, performance value 6, performance value 7 and performance value 8, the performance index of network structure A is obtained.

[0105] Among them, in step 151, the method of obtaining the performance value of the candidate network structure on the data set can be flexibly selected, for example, the evaluation result of any intermediate model can be selected as the performance value of the candidate network structure on the data set, or the evaluation result of the intermediate model that meets the preset conditions can be selected as the performance value of the candidate network structure on the data set. The above two methods are only examples, and their implementation methods are not limited.

[0106] In order to make the performance value of the candidate network structure on the data set more representative, in step 151, the intermediate model obtained after each round of training is evaluated to obtain the model performance of the intermediate model, and the performance value of the candidate network structure on the data set is determined by the model performance of each intermediate model. Figure 7 , step 151 may include steps 1511 to 1516.

[0107] Step 1511, according to the model structure parameters and the candidate network structure, obtain the initial model to be trained.

[0108] Step 1512, according to the training strategy and the optimization algorithm of the candidate network structure, use the training set to perform a round of iterative training on the training model to obtain an intermediate model.

[0109] It should be noted that the iterative training is carried out according to the batch size, number of iterations and other information specified by the training strategy, and during the training process, the training error (i.e., loss value) of the model is dynamically calculated, and the weight parameters of the model are optimized using the back propagation algorithm based on the optimization algorithm to obtain the intermediate model. Among them, an intermediate model is obtained after each round of iteration, and the network structure of each intermediate model is the same, only the model parameters are different.

[0110] Therefore, we pay attention to the recognition accuracy and real-time indicators of the model to adjust the parameters to ensure that the model can adapt to the special needs of the color sorter task.

[0111] In addition, during the training process, if the model is detected to be overfitting or underfitting, the regularization parameters are modified or the early stopping strategy is implemented to prevent the model from overfitting or underfitting and to improve the generalization ability of the model.

[0112] Step 1513, using the test set, performs performance evaluation on the intermediate model to obtain the model performance of the intermediate model.

[0113] Step 1514, update the number of iterations, and determine whether the number of iterations is equal to a preset iteration threshold. If not, execute step 1515, and if so, execute step 1516.

[0114] Step 1515, taking the intermediate model as the next model to be trained.

[0115] In addition, in order to obtain the best performance of the model as much as possible, after obtaining the performance value of the intermediate model, in step 1515, the learning rate of the intermediate model is adjusted according to the performance value. For example, when the model is underfitting or underfitting, the learning rate is increased, and when the model does not converge, the learning rate is reduced. Then, the intermediate model after adjusting the learning rate is used as the next model to be trained, so as to carry out the next round of iterative training and evaluation.

[0116] Step 1516, based on the performance of each model, obtain the performance value of the candidate network structure on the data set.

[0117] Here, any model performance can be selected as the performance value of the candidate network structure on the data set, or the model performance that meets the set conditions can be selected as the performance value of the candidate network structure on the data set. The above method is only an example, and this embodiment does not limit the method for determining the performance value of the candidate network structure on the data set.

[0118] In order to enable the performance value to characterize the optimal detection capability of the candidate network structure for the material corresponding to the data set, in step 1516, the optimal model performance is selected from the performances of each model as the performance value of the candidate network structure on the data set.

[0119] After the performance values ​​of the candidate network structure on each data set are determined through the above steps 1511 to 1516, any performance value can be selected in step 153 as the performance indicator of the candidate network structure. In step 153, the performance of the candidate network structure on the data set can also be scored according to the preset first scoring rule and the performance value of the candidate network structure on the data set, and then the scores are summed to obtain the final performance score of the candidate network structure. The implementation method of obtaining the performance indicator of the candidate network structure is not limited.

[0120] In order to enable the performance indicator to characterize the optimal detection ability and generalization ability of the candidate network structure, in step 153, the performance values ​​of the candidate network structure on each data set are averaged, and the average value is used as the performance indicator of the candidate network structure.

[0121] The performance value may include any one or more of any other performance indicators such as recognition accuracy, inference time, and computing resource consumption. The average value of each performance value is taken as the final performance indicator of the candidate network structure.

[0122] After the performance indicators of each candidate network structure are determined through the above steps 151 to 153 and related sub-steps, in step 17, when the performance indicator is one item or one score, the candidate network structure with the best indicator or the highest score can be selected as the optimal network structure.

[0123] In the case of multiple performance indicators, each indicator of the candidate network structure can be scored according to the preset second scoring rule (specifying the scoring rule and scoring weight of each indicator) and the performance indicator of the candidate network structure, and then the score values ​​of each indicator are summed according to the scoring weight to obtain the final performance score of the candidate network structure. Thus, the candidate network structure with the highest performance score is selected as the optimal network structure. Alternatively, the candidate network structure with the highest one of the indicators can be selected as the optimal network structure. This embodiment does not limit its implementation method.

[0124] In addition, in order to improve the ability of the optimal network structure finally selected to identify different feature differences of the same material (such as insect eyes in melon seeds, yellow spots in melon seeds, and shriveled melon seeds) and ensure the detection accuracy of the optimal network structure, the concept of data enhancement and sub-category standards is introduced in the construction of the data set. Figure 8 The model search method for a sorting machine provided by an embodiment of the present invention may include steps 21 to 25 of constructing a data set.

[0125] Step 21, obtaining the original image of each material of the sorting machine, determining the sorting category of the original image according to the material sorting standard, and marking the original image according to the sorting category to obtain a marked image.

[0126] During the annotation process, the original images are annotated into different subcategories, such as insect eyes, yellow spots, etc., and the specific features of each original image are recorded. At the same time, high-precision annotation tools are used in the annotation process to ensure that the annotation information of each image is accurate.

[0127] Step 23, performing data enhancement and normalization operations on each labeled image to obtain multiple sample images.

[0128] Here, the data enhancement operation includes any one or several of fine-grained enhancement, geometric transformation, color adjustment, and noise addition.

[0129] Fine-grained enhancement of annotated images involves applying fine-grained data augmentation techniques to simulate different lighting conditions, dirt, or defects for each annotated image.

[0130] The geometric transformation of the annotated image includes: rotating, scaling, cropping and other operations on the annotated image to simulate different shooting angles and material position changes.

[0131] The color adjustment of the annotated image includes: adjusting the color and contrast of the annotated image to simulate the color change and lighting difference in actual use.

[0132] Noise addition to the annotated image includes: adding random noise or blur to the annotated image to enhance the recognition ability of the model under noise interference.

[0133] After obtaining more annotated images through data augmentation, the image data is normalized in terms of image size and pixel value to make the sample images consistent and meet the input requirements of model training.

[0134] Step 25: for each type of material, each sample image of the material is taken as a data set, and each sample image in the data set is divided into a training set and a test set according to a preset ratio.

[0135] Through the above steps 21 to 25, the sample images of the data set for each material are made consistent and diverse, and the annotations are made more refined, which helps the subsequent network structure to accurately identify subtle differences in material characteristics after training, so as to obtain more accurate evaluation results.

[0136] Based on the same concept as the model search method for the sorting machine described above, refer to Fig. 9 The present invention also provides a sorting method, which is applied to Figures 1 to 3 In the control device 123 of the sorting system 10 shown, the sorting method includes steps 31 to 33.

[0137] Step 31, input the currently acquired material image into a preset detection model, and generate a detection result using the detection model.

[0138] The detection model is a model obtained by training the searched optimal network structure, and the optimal network structure is obtained by using the model search method for the sorting machine 120 provided above.

[0139] Step 33, based on the detection result, control the sorting spray valve to sort the materials in the sorting machine.

[0140] In the above-mentioned sorting method, since the detection model has higher detection accuracy (ie, accuracy) and faster reasoning speed, the sorting performance and sorting speed of the sorting machine are greatly improved, and the sorting performance is greatly improved.

[0141] In order to verify the effect of the model search method and sorting method for the sorting machine provided above, based on the data sets and search spaces of eight materials including pepper, pepper, Chinese medicine, sponge titanium, shrimp, ore, tea, and X-ray macadamia, the model search method provided above was used in the model construction of the sorting machine, and finally the following results were obtained: Fig.10The optimal network structure shown in Figure 1. The detection model trained with the optimal network structure is deployed on the control device of the sorting machine. The control device sorts peppers, peppers, Chinese medicine, sponge titanium, shrimp, ore, tea, and macadamia nuts based on the detection model. Based on the sorting results, the performance of the detection model is compared with that of MobileNetV3, which is currently the most commonly used in the industry. The accuracy comparison results are shown in Table 1, and the comparison results of model size, parameter quantity, and computational complexity are shown in Table 2.

[0142] Table 1

[0143]

[0144]

[0145] Table 2

[0146]

[0147] As can be seen from Table 1, compared with the commonly used MobileNetV3 model in the industry, the accuracy of the detection model obtained by the model search method for the sorting machine 120 provided by the implementation of the present invention is significantly higher. As can be seen from Table 2, compared with the commonly used MobileNetV3 model in the industry, the model size, calculation amount and parameter amount of the detection model obtained by the model search method for the sorting machine 120 provided by the implementation of the present invention are greatly reduced, and the detection model obtained by the model search method for the sorting machine 120 is more lightweight.

[0148] Based on the same concept as the model search method for the sorting machine 120 described above, refer to Fig.11 The embodiment of the present invention further provides a model search device 30 for a sorting machine 120 , including a search module 310 , a first selection module 320 , a performance evaluation module 330 and a second selection module 340 .

[0149] The search module 310 is used to search and sample the preset search space to obtain search results, wherein the search results include multiple neural network structures and hyperparameters and optimization algorithms corresponding to each neural network structure.

[0150] The first selection module 320 is used to calculate the scale value of the neural network structure when performing a forward reasoning, and select a candidate network structure from each neural network structure according to the scale value.

[0151] The performance evaluation module 330 is used to optimize and evaluate each candidate network structure based on the hyperparameters and optimization algorithm of the candidate network structure using multiple data sets to obtain performance indicators of the candidate network structure. Each data set is a data set of a material of the sorting machine.

[0152] The second selection module 340 is used to determine the optimal network structure from the candidate network structures according to the performance index.

[0153] The model search device 30 for the sorting machine automatically searches and generates a variety of neural network structures through the synergistic effect of the search module 310, the first selection module 320, the performance evaluation module 330 and the second selection module 340, which greatly reduces the cost of manual design. Then, the neural network structure is screened according to the scale value, and the optimization evaluation of each selected candidate network structure is performed to select the optimal network structure with small scale values ​​such as the amount of calculation and the amount of parameters, and the best recognition performance for multiple materials of the sorting machine, so that the optimal network structure has wide adaptability and achieves the effect of having higher recognition accuracy and faster reasoning speed.

[0154] For the specific implementation and effect of the model search device 30, please refer to the above description of the implementation of the model search method for the sorting machine, such as, for the specific implementation and effect of the search module 310, please refer to the description of the relevant content of step 11 above, for the specific implementation and effect of the first selection module 320, please refer to the description of the relevant content of step 13 above, for the specific implementation and effect of the performance evaluation module 330, please refer to the description of the relevant content of step 15 above, for the specific implementation and effect of the second selection module 340, please refer to the description of the relevant content of step 17 above, and no further details will be given here.

[0155] An embodiment of the present invention further provides a sorting device, including a detection module and a sorting module.

[0156] The detection module is used to input the currently acquired material image into a preset detection model and generate a detection result using the detection model.

[0157] Among them, the detection model is a model obtained after training the searched optimal network structure, and the optimal network structure is obtained by using the model search method for the sorting machine provided above.

[0158] The sorting module is used to control the sorting spray valve to sort the materials in the sorting machine based on the detection results.

[0159] In the above-mentioned sorting device, since the detection model has higher detection accuracy (ie, accuracy) and faster reasoning speed, the sorting performance and sorting speed of the sorting machine are greatly improved, and the sorting performance is greatly improved.

[0160] In addition, each module in the above-mentioned model search device 30 and the sorting device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor 220 in the electronic device 20 in the form of hardware, or can be stored in the memory 210 of the electronic device 20 in the form of software, so that the processor 220 calls and executes the operations corresponding to the above-mentioned modules to implement the model search method for the sorting machine provided above, or the sorting method provided above.

[0161] Based on the same inventive concept as the above-mentioned model search method, an embodiment of the present invention also provides an electronic device 20, including a processor 220 and a memory 210, the memory 210 stores a computer program that can be executed by the processor 220, and the processor 220 can execute the computer program to implement the model search method proposed in the embodiment of the present invention, or the sorting method provided above.

[0162] The embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by the processor 220, the model search method proposed in the embodiment of the present invention or the sorting method provided above is implemented.

[0163] In summary, the model search method, sorting method and electronic device for a sorting machine provided by the embodiments of the present invention automatically generate the optimal neural network architecture by defining the search space and automatically exploring and optimizing the network structure using methods such as grid search.

[0164] It has at least the following beneficial effects:

[0165] (1) Automatically search and optimize network structure, significantly reducing the degree of human involvement, which not only improves design efficiency, but also reduces dependence on expert experience, making the neural network design process more automated and systematic;

[0166] (ii) Strictly limit the computational complexity and number of parameters of the model during the search process to ensure that the resulting network structure can both meet the high-precision requirements of the sorting machine and significantly improve the inference speed;

[0167] (III) During the network search process, a large amount of image data collected in actual machine environments was collected and prepared, covering a variety of different types of materials. In the training and evaluation process, the performance of the searched network structure under the fine-grained features of these different materials was fully considered to select the final optimal network structure, thereby ensuring that the designed network can accurately extract and identify these features, so that the model has stability and wide adaptability, can handle a variety of material features, and is suitable for different application scenarios;

[0168] (iv) The application scope of the sorting machine has been enhanced, so that the equipment can cope with a wider range of material types and characteristic changes, reducing the debugging and adjustment work caused by differences in material characteristics;

[0169] (V) By strictly limiting the amount of calculation and parameters, lightweight model design is achieved, the number of network parameters and calculation is reduced, and by optimizing the network structure, the dependence on hardware resources is reduced, ensuring that the model can run efficiently on embedded devices.

[0170] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0171] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0172] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory 210 (ROM, Read-Only Memory), random access memory 210 (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A model search method for a sorting machine, characterized in that: The method comprises: Searching and sampling a preset search space to obtain search results; wherein the search results include a plurality of neural network structures and hyperparameters and optimization algorithms corresponding to each of the neural network structures; Calculating a scale value of the neural network structure when performing a forward inference, and selecting a candidate network structure from each of the neural network structures according to the scale value; For each candidate network structure, based on the hyperparameters and optimization algorithm of the candidate network structure, multiple data sets are used to optimize and evaluate the candidate network structure to obtain performance indicators of the candidate network structure; wherein each data set is a data set of a material of the sorting machine; According to the performance indicators, an optimal network structure is determined from the candidate network structures.

2. The model search method for a sorting machine according to claim 1, characterized in that: The step of optimizing and evaluating the candidate network structure using multiple data sets based on the hyperparameters and optimization algorithm of the candidate network structure to obtain the performance index of the candidate network structure includes: For each of the data sets, according to the settings of the hyperparameters of the candidate network structure, using the data set and the optimization algorithm of the candidate network structure, iteratively training the candidate network structure, and evaluating the intermediate model obtained from each round of iterative training to obtain the performance value of the candidate network structure on the data set; According to the performance value of the candidate network structure on each of the data sets, a performance indicator of the candidate network structure is obtained.

3. The model search method for a sorting machine according to claim 2, characterized in that: The data set includes a training set and a test set, and the hyperparameters include model structure parameters and training strategies; The step of iteratively training the candidate network structure according to the setting of the hyperparameters of the candidate network structure and using the optimization algorithm of the data set and the candidate network structure for each data set, and evaluating the intermediate model obtained from each round of iterative training to obtain the performance value of the candidate network structure on the data set comprises: According to the model structure parameters and the candidate network structure, an initial model to be trained is obtained; According to the training strategy and the optimization algorithm of the candidate network structure, using the training set, performing a round of iterative training on the model to be trained to obtain an intermediate model; Using the test set, performing performance evaluation on the intermediate model to obtain model performance of the intermediate model; Updating the number of iterations, and determining whether the number of iterations is equal to a preset iteration threshold; If not, use the intermediate model as the next model to be trained, and return to the step of executing the optimization algorithm according to the training strategy and the candidate network structure, using the training set, performing a round of iterative training on the model to be trained, and obtaining an intermediate model; If so, the performance value of the candidate network structure on the data set is obtained according to the performance of each model.

4. The model search method for a sorting machine according to claim 3, characterized in that: The step of obtaining the performance value of the candidate network structure on the data set according to the performance of each model includes: From the model performances, the optimal model performance is selected as the performance value of the candidate network structure on the data set.

5. The model search method for a sorting machine according to claim 2, characterized in that: The step of obtaining the performance index of the candidate network structure according to the performance value of the candidate network structure on each of the data sets includes: An average value is taken for the performance values ​​of the candidate network structure on each of the data sets, and the average value is used as the performance indicator of the candidate network structure.

6. The model search method for a sorting machine according to any one of claims 1 to 5, characterized in that: The scale value includes the amount of calculation and the amount of parameters; The step of selecting a candidate network structure from each of the neural network structures according to the scale value comprises: For each of the neural network structures, when the computational amount of the neural network structure is less than a preset first threshold and the parameter amount is less than a preset second threshold, the neural network structure is taken as a candidate network structure.

7. The model search method for a sorting machine according to any one of claims 1 to 5, characterized in that: The step of performing search sampling on the preset search space to obtain search results includes: A grid search algorithm is used to search and sample a preset search space to obtain search results; wherein the search space includes multiple convolution kernels, activation functions, the number of convolution kernels of the convolution layer, the configuration method of the convolution module, hyperparameters and optimization algorithms.

8. The model search method for a sorting machine according to any one of claims 1 to 5, characterized in that: Before the step of searching and sampling the preset search space to obtain search results, the method further includes: Acquire original images of various materials of the sorting machine, determine the sorting category of the original images according to the material sorting standard, and annotate the original images according to the sorting category to obtain annotated images; Performing data augmentation and normalization operations on each of the labeled images to obtain multiple sample images; For each type of the material, each sample image of the material is taken as a data set, and each sample image in the data set is divided into a training set and a test set according to a preset ratio.

9. A sorting method, characterized in that: A control device applied to a sorting machine, wherein the control device is communicatively connected with a sorting spray valve of the sorting machine, and the method comprises: Input the currently acquired material image into a preset detection model, and generate a detection result using the detection model; wherein the detection model is a model obtained by training the searched optimal network structure, and the optimal network structure is obtained by using the model search method for a sorting machine according to any one of claims 1 to 8; Based on the detection result, the sorting spray valve is controlled to sort the materials in the sorting machine.

10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the model search method for a sorting machine as described in any one of claims 1 to 8, or the sorting method as described in claim 9.

Images