Durian nondestructive testing method and device and storage medium

Abstract

The invention relates to the technical field of durian quality detection, in particular to a durian nondestructive detection method and device and a storage medium. According to the method, after a three-dimensional reconstruction body of a durian to be detected is obtained, maturity prediction and insect fruit prediction are performed by using a trained maturity classification model and an insect fruit identification model respectively, and a corresponding maturity classification result and an insect fruit identification result are obtained; wherein the maturity classification model is constructed based on a multi-scale VGG network, and training is carried out by utilizing a pre-acquired durian maturity data set; the insect and fruit recognition model is constructed based on a ResNet3D network, and training is carried out by using a pre-screened durian block data set. Compared with the prior art, the method has the advantages of improving the nondestructive testing efficiency and accuracy of the durian and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of durian quality detection, and in particular to a durian nondestructive detection method, device and storage medium. Background Art

[0002] Nondestructive testing refers to testing the internal and external quality of the object to be tested without destroying its integrity, and analyzing and evaluating the test results according to preset standards. For fruits with thicker shells such as durian, nondestructive testing technology is very important for its quality assessment.

[0003] After searching, Chinese patent CN117388292A discloses a method and system for non-destructive rapid detection and grading of the internal quality of durian. The method uses an X-ray detection source and an X-ray detection plate to obtain a projection image of the durian to be detected and transmits it back to a computing system for image processing. The three-dimensional image data of the durian sample is obtained by an FDK reconstruction algorithm, and the reconstruction result is decomposed according to a fixed step size to obtain a tomographic image. The tissues in the tomographic image are segmented using a segmentation algorithm based on the DeepLabV3 model, and finally the durian pulp, core, peel and cavity are accurately divided on the entire three-dimensional image data. The phenotypic traits of durian such as two-dimensional edible rate, number of pulp cells, fruit length, fruit volume, fruit width, pulp volume and cavity volume are calculated according to the segmentation results. Finally, the maturity of the durian to be detected is assigned a category according to the calculated phenotypic trait results, and the internal quality of the durian is regressed and analyzed in combination with the quality information, and the output result is a durian quality score. Although this scheme can realize non-destructive detection of durian maturity, the segmentation algorithm is used to divide the various parts of the durian, and obtaining the final durian maturity requires a complex calculation process, which reduces the detection efficiency. In addition, the use of a single dimension of phenotypic traits to judge the maturity of the durian has a certain impact on the accuracy of the test results. In addition, after searching, there is currently no non-destructive detection method specifically for whether durian is a worm-infested fruit. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a durian non-destructive testing method, device and storage medium, which can improve the efficiency and accuracy of durian non-destructive testing.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to a first aspect of the present invention, a durian nondestructive detection method is provided, comprising the following steps: obtaining a three-dimensional reconstruction of a durian to be detected; based on the three-dimensional reconstruction, respectively using a trained maturity classification model and an insect-fruit recognition model to perform maturity prediction and insect-fruit prediction, to obtain corresponding maturity classification results and insect-fruit recognition results; wherein the maturity classification model is constructed based on a multi-scale VGG network and is trained using a pre-acquired durian maturity data set; the insect-fruit recognition model is constructed based on a ResNet3D network and is trained using a pre-screened durian cut data set.

[0007] As an optimal technical solution, the maturity classification model includes a multi-scale VGG network, a threshold segmentation processing module and a classification fully connected layer. The multi-scale VGG network includes a first convolutional layer, an intermediate layer and an embedding layer connected in sequence. The trained maturity classification model is used to predict maturity. The specific process includes: inputting the three-dimensional reconstructed tomogram obtained according to the three-dimensional reconstructed body into the first convolutional layer to obtain a first intermediate feature, a second intermediate feature and a third intermediate feature respectively, and inputting the three-dimensional reconstructed tomogram into the threshold segmentation processing module to obtain a density feature; the first intermediate feature, the second intermediate feature and the third intermediate feature are respectively processed by the corresponding intermediate layers, and then spliced ​​in the embedding layer to obtain a shape feature; the shape feature and the density feature are spliced ​​and input into the classification fully connected layer, and the final maturity classification result is output.

[0008] As a preferred technical solution, the convolution kernel sizes of the first convolution layer include 3×3, 15×15 and 96×96, corresponding to the output of the first intermediate feature, the second intermediate feature and the third intermediate feature, respectively.

[0009] As a preferred technical solution, the training process of the maturity classification model specifically includes: performing image processing and analysis on the three-dimensional reconstructed body to obtain a three-dimensional reconstructed tomogram; inputting the three-dimensional reconstructed tomogram into the constructed maturity classification model, and training using the durian maturity data set and cross entropy loss value, wherein the cross entropy loss value is calculated using the shape features and density features extracted in each iteration.

[0010] As a preferred technical solution, the screening process of the durian slice data set specifically includes: using a sliding window of a preset size to traverse a preset three-dimensional reconstructed volume data set, and calculating the voxel value averages of the slices in the sliding window in turn; when the voxel value average meets a first preset condition, standard deviation screening is performed; when the screened standard deviation meets a second preset condition, the current slice is determined to be a durian slice for training; wherein, the first preset condition is that the voxel value average is between a first hyperparameter and a second hyperparameter, and the first hyperparameter and the second hyperparameter are used to determine that the slice is at the boundary of the durian shell; the second preset condition is that the standard deviation is greater than a third hyperparameter, and the third hyperparameter is used to determine that the slice is not distributed in the gaps or empty rooms of the durian shell.

[0011] As a preferred technical solution, the trained insect-fruit recognition model is used to predict insect-fruit, which specifically includes the following steps: S601, sliding window cutting is performed on the three-dimensional reconstructed body to screen the current cut block for insect-fruit prediction; S602, the screened current cut block is input into the trained insect-fruit recognition model to obtain a preliminary recognition result; S603, when the preliminary recognition result is good fruit, return to S601; S604, when the preliminary recognition result is insect-fruit, determine whether the current cut block is a vertex cut block, if so, return to step S601, otherwise execute S6 05, the vertex block is the last block of the current three-dimensional reconstruction window; S605, obtain other blocks whose Manhattan distance with the current block is 1, and input the trained insect fruit recognition model to obtain a second recognition result. If there is an insect fruit in the second recognition result, the final recognition result of the current block is an insect fruit, otherwise the final recognition result of the current block is a good fruit; S606, after the sliding window ends, obtain the number of insect fruits in the final recognition results of all blocks. If it is greater than a preset threshold, the final recognition result of the current durian to be detected is an insect fruit.

[0012] As a preferred technical solution, the process of acquiring the three-dimensional reconstruction body specifically includes: acquiring a projection image of the durian to be detected; and reconstructing the projection image into a three-dimensional reconstruction body using an FDK algorithm.

[0013] According to a second aspect of the present invention, a durian nondestructive testing device is provided, the device is used to implement the method, the device at least comprises a three-dimensional reconstruction module and a model prediction module, wherein: the three-dimensional reconstruction module is used to obtain a three-dimensional reconstruction body of the durian to be detected; the model prediction module is used to perform maturity prediction and insect-fruit prediction based on the three-dimensional reconstruction body, respectively using a trained maturity classification model and an insect-fruit recognition model, to obtain corresponding maturity classification results and insect-fruit recognition results; wherein the maturity classification model is constructed based on a multi-scale VGG network and is trained using a pre-acquired durian maturity data set; the insect-fruit recognition model is constructed based on a ResNet3D network and is trained using a pre-screened durian cut data set.

[0014] As a preferred technical solution, the device also includes an X-ray shooting module, which is used to obtain a projection image of the durian to be detected, and the projection image is input into the three-dimensional reconstruction module.

[0015] According to a third aspect of the present invention, there is provided a storage medium having a program stored thereon, wherein the program implements the method described above when executed.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] 1. After obtaining the three-dimensional reconstruction of the durian to be detected, the present invention uses the trained maturity classification model based on the multi-scale VGG network and the insect-fruit recognition model based on the ResNet3D network to perform maturity prediction and insect-fruit prediction. The corresponding maturity classification results and insect-fruit recognition results can be directly obtained through the three-dimensional reconstruction prediction, without the need to perform complex calculations based on the algorithm prediction results, and the two prediction processes can be performed simultaneously, which can fully improve the efficiency of non-destructive detection of durian;

[0018] 2. In the present invention, the maturity classification model includes a multi-scale VGG network, a threshold segmentation processing module and a classification fully connected layer. On the one hand, the multi-scale VGG network can extract features of different scales inside the durian. The second intermediate feature and the third intermediate feature are used to enable the model to better learn the Luokuo features of the durian pulp level, and the first intermediate feature is used to enable the model to learn the detailed texture of the durian pulp. After splicing, the shape features of the durian pulp are jointly characterized; on the other hand, the threshold segmentation processing module can extract the density features of the durian from the three-dimensional reconstructed tomogram, and the density features and the shape features are spliced ​​through the classification fully connected layer to obtain the final maturity classification result. The present invention judges the maturity of the durian by multi-dimensional factors such as shape and density, which can effectively improve the accuracy of durian maturity prediction and classification;

[0019] 3. The present invention uses a pre-screened durian cut data set to train the insect fruit recognition model. The screening process of the durian cut data set is realized by a sliding window cutting method based on a standard deviation threshold. The method is based on three-dimensional reconstructed volume data and tries to cut small pieces on the durian shell for insect bite feature recognition. This can avoid the problem that the final recognition result is difficult to count and solve the problem that too small features are difficult to learn, thereby effectively improving the accuracy of insect fruit recognition;

[0020] 4. When the present invention uses the insect fruit recognition model to predict insect fruits, when the preliminary recognition result is an insect fruit, it is necessary to determine whether the current block is a vertex block. The vertex block is the last block of the sliding window in the x, y, and z directions of the current three-dimensional reconstruction body. Combined with the joint prediction of the adjacent blocks in step S605, under the premise of setting the sliding window step size according to the block side length, it can avoid missing wormholes due to skipping vertex blocks, and further ensure the accuracy of the insect fruit prediction results;

[0021] 5. The method and device provided by the present invention can not only predict and classify the maturity of durian, but also identify insect-infested fruits, thereby improving the applicability of the method and device for non-destructive detection of durian. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an implementation flow of a method provided in an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a structure of a maturity classification model in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of insect bite characteristics provided in an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of an implementation flow of a sliding window slicing method in an embodiment of the present invention;

[0026] Figure 5 Schematic diagram of one structure of the insect fruit identification model in an embodiment of the present invention;

[0027] Figure 6 It is a schematic diagram of the implementation process of the block prediction result integration strategy in an embodiment of the present invention;

[0028] Figure 7 It is a schematic diagram of the overall mechanical framework of one type of assembly line of the durian detection system in an embodiment of the present invention;

[0029] Among them: 700, radiation-proof shell; 701, first photoelectric; 702, second photoelectric; 703, third photoelectric; 704, fourth photoelectric; 705, fifth photoelectric; 706, entrance conveyor belt; 707, first entrance radiation-proof door; 708, second entrance radiation-proof door; 709, internal roller; 710, first exit radiation-proof door; 711, second exit radiation-proof door; 712, exit conveyor belt; 713, durian turntable; 7131, linear drive motor; 7132, lifting cylinder; 7133, rotating skeleton; 7134, lifting thorn. DETAILED DESCRIPTION

[0030] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0031] Example:

[0032] In order to solve the problems in the prior art that the efficiency of durian nondestructive detection is low and the accuracy cannot be guaranteed, the present invention provides a durian nondestructive detection method. After obtaining a three-dimensional reconstruction of a durian to be detected, the method uses a trained maturity classification model and an insect-fruit recognition model to perform maturity prediction and insect-fruit prediction, respectively, to obtain corresponding maturity classification results and insect-fruit recognition results; wherein the maturity classification model is constructed based on a multi-scale VGG network and is trained using a pre-acquired durian maturity data set; the insect-fruit recognition model is constructed based on a ResNet3D network and is trained using a pre-screened durian cut data set.

[0033] Figure 1 One of the implementation processes of the aforementioned method is shown. Specifically, firstly, CT technology is used to realize three-dimensional reconstruction of the durian to be detected, and then the obtained three-dimensional reconstruction body is synchronously input into the trained maturity classification model and the insect-fruit recognition model to obtain the corresponding maturity classification result and the insect-fruit recognition result. Among them, the three-dimensional reconstruction of the durian to be detected using CT technology specifically includes: using an X-ray source and a detector to obtain a projection image of the durian to be detected; using the FDK algorithm to reconstruct the projection image into a three-dimensional voxel image (i.e. Figure 1 In actual application, 180 durian projection images can be taken within 6 seconds. In addition, both the maturity classification model and the insect-fruit recognition model use deep learning models as the main network. The maturity classification model is built based on the multi-scale VGG network, and the insect-fruit recognition model is built based on the ResNet3D network. The training and deployment of the models are completed offline. The parallel operation of the two models can maximize the operation efficiency.

[0034] Since CT technology can mainly obtain the spatial density information of the durian to be detected, it is considered to use the tomogram of the three-dimensional reconstruction of the durian as a sample, and train the deep model with the input of the two-dimensional grayscale image to classify the maturity of the durian. In the present embodiment, the maturity classification model of the durian includes a multi-scale VGG network, a threshold segmentation processing module and a classification full connection layer, and the multi-scale VGG network includes a first convolutional layer, an intermediate layer and an embedding layer connected in sequence. Therefore, the maturity classification model trained is used to predict maturity, and the specific process includes: inputting the three-dimensional reconstruction tomogram obtained according to the three-dimensional reconstruction into the first convolutional layer, respectively obtaining the first intermediate feature, the second intermediate feature and the third intermediate feature, and inputting the three-dimensional reconstruction tomogram into the threshold segmentation processing module to obtain the density feature; the first intermediate feature, the second intermediate feature and the third intermediate feature are respectively processed by the corresponding intermediate layer, and then spliced ​​in the embedding layer to obtain the shape feature; the shape feature and the density feature are spliced ​​and input into the classification full connection layer, and the final maturity classification result is output. The three-dimensional reconstruction tomogram can be obtained by decomposing the three-dimensional reconstruction fruit according to a fixed step size.

[0035] Figure 2 One of the structures of the maturity classification model based on the multi-scale VGG network is shown, and the input of the model is a three-dimensional reconstructed tomogram (image). Specifically: in the VGG network structure of multi-scale convolution, the first convolution layer can use convolution kernels of 3×3, 15×15 and 96×96 sizes, respectively, corresponding to the output of the first intermediate feature (397 dimensions), the second intermediate feature (27 dimensions) and the third intermediate feature (20 dimensions) to extract features of different scales. This is because the durian pulp cross-section size contained in the three-dimensional reconstructed tomogram of durian is inconsistent in different layers and different postures, which may be very small or very large. After observing a large amount of durian data, it was found that the maximum side length of the durian pulp cross-section is about 96 pixels, and the minimum is about 15 pixels. Therefore, the 15×15 and 96×96 convolution kernels can allow the model to better learn the contour features of the durian pulp level, while the 3×3 convolution kernel allows the model to learn the detailed texture of the durian pulp. Finally, the three scales of deep features are spliced ​​in the embedding layer (the output layer with the first channel dimension of each grid trunk being 1) so that they do not interfere with each other and jointly characterize the shape characteristics of the durian pulp.

[0036] Exemplarily, for a multi-scale VGG network, Figure 2In the example, the first intermediate feature (397 dimensions) passes through the following intermediate layers: 3×3 convolutional layer (3×3conv, 64), maximum pooling layer (max pool, / 2), 3×3 convolutional layer (3×3conv, 128), maximum pooling layer (max pool, / 2), 3×3 convolution layer (3×3conv, 256), maximum pooling layer (maxpool, / 2), 3×3 convolution layer (3×3conv, 512) and 3×3 convolution layer (3×3conv, 1024); the intermediate layers passed by the second intermediate feature (27 dimensions) include the activation layer (Relu), 3×3 convolution layer (3×3conv, 64), activation layer (Relu), 3×3 convolution layer (3×3conv, 128), activation layer (Relu) and 3×3 convolution layer (3×3conv, 256) connected in sequence; the intermediate layers passed by the third intermediate feature (20 dimensions) include the activation layer (Relu), 5×5 convolution layer (5×5conv, 32), activation layer (Relu), maximum pooling layer (max pool, / 2), 3×3 convolution layer (3×3conv, 64) and average pooling layer (avgpool, / 4) connected in sequence. Correspondingly, Figure 2 The embedding layer shown in the figure includes the 3×3 convolution layer (3×3conv, 1024) that the first intermediate feature passes through, the 3×3 convolution layer (3×3conv, 256) that the second intermediate feature passes through, and the average pooling layer (avg pool, / 4) that the third intermediate feature passes through. The output (1 dimension) of the first intermediate feature after passing through the corresponding embedding layer must also pass through a fully connected layer (fc, 512), and then concatenated with the output (1 dimension) of the second and third intermediate features after passing through the corresponding embedding layer. The concatenated features (832 dimensions) pass through a fully connected layer (fc, 10) to obtain a 10-dimensional shape feature. The input size in the figure represents the input dimension, and the output size represents the output dimension.

[0037] At the same time, for the threshold segmentation processing module and the classification fully connected layer, Figure 2In the , the number of voxels in each segment of the durian 3D reconstruction is counted by threshold segmentation (here there are 10 segments, so the output is a 10-dimensional vector), and then the Softmax operation can be performed. This method has been widely verified to have significant effects on classification problems. The purpose is to allow the model to extract density-related features, which are spliced ​​with the shape features extracted by the multi-scale VGG network in the penultimate layer (after splicing, 20-dimensional features are obtained), and then enter the final classification fully connected layer (fc, 4). It should be noted that during the training stage, the cross entropy of the shape features (10 dimensions) extracted by the multi-scale VGG network and the density features (10 dimensions) obtained by the threshold segmentation module is calculated as a regularization term and added to the training.

[0038] Optionally, the maturity of durian is divided into multiple levels: less than 6 mature, 6-7 mature, 8-9 mature, 10 mature and over-mature. This level division takes into account both the characteristics of CT technology that mainly obtains density features and the similarity of durian maturity levels.

[0039] Exemplarily, the maturity classification model proposed in this embodiment is actually tested using the Sanya Toucan Durian data set, and the test set classification accuracy can be obtained to be 92%. After the model is deployed, the average prediction time of each durian maturity is 120ms, which greatly improves the durian maturity prediction efficiency compared to traditional methods. Among them, the Sanya Toucan Durian data set is collected from domestic native durian, with an average age of 4 years. By August 1, 2024, a total of 80 domestic durians and 300 Thai durians were collected. Among them, domestic durian includes labels raw (2), 6 mature (7), 7 mature (11), 8 mature (8), 9 mature (32), 10 mature (7), insect fruit (20), and rotten (7). The training set and test set are not defined, and 10-fold cross validation is used for training and testing.

[0040] Next, the training and operation of the insect-fruit identification model are explained.

[0041] Based on the observation of 3D reconstruction data, it is found that when the size of the 3D reconstruction volume is (397, 397, 397), the features of the insect bite part usually do not exceed (11, 11, 11). Since features that are too small will make the model difficult to learn, and in practical applications, durian data often faces small sample situations, so the sliding window cutting method is adopted on the 3D reconstruction volume, and each cut is used as a sample for annotation and training.

[0042] Since the characteristics of insect bites are often distributed on the outer shell of durian, such as Figure 3 As shown, we try to cut as small pieces as possible from the shell, which can avoid the problem of difficult statistics of the final recognition results (mentioned below) and solve the problem of difficult feature learning.

[0043] In order to make the cut pieces distributed on the shell as much as possible, the present embodiment provides a sliding window cutting method based on standard deviation threshold, which uses a sliding window of preset size to traverse a preset three-dimensional reconstructed volume data set, and calculates the voxel value averages of the cut pieces in the sliding window in sequence; when the voxel value average meets the first preset condition, standard deviation screening is performed; when the screened standard deviation meets the second preset condition, the current cut piece is determined to be a durian cut piece for training; wherein the first preset condition is that the voxel value average is between the first hyperparameter and the second hyperparameter, and the first hyperparameter and the second hyperparameter are used to determine that the cut piece is at the boundary of the durian shell; the second preset condition is that the standard deviation is greater than the third hyperparameter, and the third hyperparameter is used to determine that the cut piece is not distributed in the gaps or empty rooms of the durian shell.

[0044] Figure 4 One implementation of the above sliding window slicing method is shown:

[0045] First, a sliding window of a fixed size (42, 42, 42) is used to traverse the entire 3D reconstructed volume data set, and the voxel value average (mb) of the cut blocks in the sliding window is calculated in turn; secondly, when the voxel value average is between 3 / 4 and 7 / 9 of the voxel value average (mv) of the entire data set (these two ratios are the first hyperparameter and the second hyperparameter, respectively, constituting the first preset condition), standard deviation screening is performed; finally, when the standard deviation (stdb) is greater than 45 (the third hyperparameter, constituting the second preset condition), it can be determined that the cut block is a cut block of the shell surface. The sliding window is performed according to the first preset condition and the second preset condition until the last cut block is completed.

[0046] The reasons for the selection of the above three hyperparameters are as follows: On the one hand, according to the statistical analysis of the voxel values ​​of the durian 3D reconstruction volume data set, the voxel values ​​of the durian distribution area in the entire matrix space are significantly higher than the background and noise distribution areas. Therefore, when the voxel value of the cut block is within a certain range of the average value of the entire voxel value, it can be ensured that the cut block is neither in an area full of flesh or shell, nor in an area full of background or noise, that is, at the boundary of the shell. According to a large number of data experiments, it is found that when the average value of the voxel value of the cut block is between 3 / 4 and 7 / 9, the cut block can completely cover all the shells without losing part of the surface, and at the same time, it can avoid being distributed in empty rooms or shell gaps as much as possible. On the other hand, in order to further reduce the distribution of the cut block in the empty room or shell gap, the characteristic of the small standard deviation of the voxels in the empty room or shell gap can be used to screen out the cut blocks with a standard deviation greater than 45, which can avoid the distribution of the cut block in the shell gap or empty room.

[0047] After the above two screenings, it can be basically ensured that the cut pieces are only distributed on the shell and can cover the entire durian shell without omission. It should be noted that with the difference of X-ray energy level and detector hardware conditions, the hyperparameters need to be adjusted accordingly.

[0048] Through the sliding window slicing method based on the standard deviation threshold, the slicing distributed in the durian shell can be screened out, and then the slicing is used to make a data set for training. This embodiment uses ResNet3D as the training model to obtain the final insect fruit recognition model. Figure 5 The figure shows one of the specific model structures. The input of the model is a three-dimensional reconstruction (image 3dblock), and the output is a score (Score). The model as a whole includes multiple convolutional layers (Conv3d), batch normalization layers (BatchNormalize), activation layers (Relu, Softmax), average pooling layers (Avg Pool3d), fully connected layers (FC) and stacked basic blocks (BasicBlock), among which, Figure 5 The right half of the diagram shows that each basic block performs multiple convolutions, batch normalization, and activation on the corresponding input image, and finally obtains the corresponding output image. The input size in the diagram represents the input dimension, and the output size represents the output dimension.

[0049] It should be noted that in the stage of worm fruit prediction, there is a problem of result statistics, because in actual application scenarios, it is difficult for the model to achieve 100% accuracy, so it is necessary to count the prediction results of the same durian cut. Otherwise, if an integrated strategy is adopted to judge the entire durian as a worm fruit when one cut is identified as a worm fruit, a large number of normal durians will be misidentified. Therefore, the worm fruit prediction process of this embodiment is implemented based on a cut prediction result integration strategy that takes result statistics into consideration, such as Figure 6 As shown in the figure, the integration strategy of the block prediction results can be summarized as the following process:

[0050] Step S601, using the aforementioned sliding window slicing method to perform sliding window slicing on the three-dimensional reconstructed body, and selecting the current slicing block for insect fruit prediction;

[0051] Step S602, inputting the selected current cut into the trained insect fruit recognition model to obtain a preliminary recognition result;

[0052] Step S603, when the preliminary recognition result is good, return to step S601;

[0053] Step S604, when the preliminary identification result is a worm fruit, determine whether the current cut is a vertex cut, if so, return to step S601, otherwise execute step S605;

[0054] Step S605, obtaining other cut pieces whose Manhattan distance to the current cut piece is 1, and inputting the trained insect fruit recognition model to obtain a second recognition result. If there is an insect fruit in the second recognition result, the final recognition result of the current cut piece is an insect fruit, otherwise the final recognition result of the current cut piece is a good fruit;

[0055] Step S606, after the sliding window ends, the number of fruit with infestation in the final identification results of all the cut pieces is counted. If the number is greater than a preset threshold, the final identification result of the current durian to be detected is a fruit with infestation.

[0056] In step S604, the vertex block refers to the last block of the sliding window in the x, y, and z directions of the three-dimensional reconstruction. According to the process of joint prediction of adjacent blocks in step S605, if the vertex block is a worm fruit, then the adjacent block before the vertex block must also be a worm fruit, because the sliding window step length is only 1 / 3 of the block side length. Therefore, the wormhole will not be missed due to skipping the vertex block. If the vertex block is a good fruit, it will not affect the final judgment. Combining the aforementioned block prediction result integration strategy for worm fruit identification can effectively ensure the accuracy of the identification result.

[0057] Furthermore, the present embodiment also provides a durian nondestructive testing device for implementing the aforementioned durian nondestructive prediction method. The device includes an industrial computer and a three-dimensional reconstruction module and a model prediction module deployed in the industrial computer, wherein: the three-dimensional reconstruction module is used to obtain a three-dimensional reconstruction body of the durian to be detected; the model prediction module is used to perform maturity prediction and insect fruit prediction based on the three-dimensional reconstruction body, respectively using the trained maturity classification model and insect fruit recognition model to obtain corresponding maturity classification results and insect fruit recognition results; wherein the maturity classification model is constructed based on a multi-scale VGG network and trained using a pre-acquired durian maturity data set; the insect fruit recognition model is constructed based on a ResNet3D network and trained using a pre-screened durian cut data set. The training and deployment of the model are completed offline, and the module deployed on the industrial computer only performs the prediction process. The specific training and prediction processes are basically consistent with the aforementioned method and will not be repeated here.

[0058] In actual application scenarios, the durian nondestructive testing device belongs to the motion control part of the entire durian testing system (which can be called a durian machine in practice). Figure 7As shown, the entire durian detection system can also include other motion control parts and mechanical architectures related to the motion control parts. Other motion control parts such as motion control cards, direct drive motors and 3 conveyor belt motors, the motion control card distributes and controls each lower-level actuator (direct drive motor, conveyor belt motor), and independently completes each preset action; the industrial computer communicates with the motion control card separately, and only sends instructions or monitoring status for executing preset actions. The mechanical architecture includes an imaging component (five photoelectric mechanisms in the figure, namely the first photoelectric 701, the second photoelectric 702, the third photoelectric 703, the fourth photoelectric 704 and the fifth photoelectric 705), an entrance conveyor belt 706, an entrance radiation protection door (2, respectively the first entrance radiation protection door 707 and the second entrance radiation protection door 708), an internal conveyor belt / internal roller 709, an exit radiation protection door (2, respectively the first exit radiation protection door 710 and the second exit radiation protection door 711), an exit conveyor belt 712 and a durian rotating table 713. The durian rotating platform 713 is composed of a linear drive motor 7131 (i.e., a direct drive motor), a lifting cylinder 7132, a rotating skeleton 7133, and a lifting thorn 7134. The third photoelectric 703, the internal roller 709, and the durian rotating platform 713 are all located inside the radiation-proof housing 700.

[0059] Durian detection system also includes an X-ray shooting module. The X-ray shooting module may include an Xray source and a detector, which is used to shoot the projection image of the durian to be detected. After shooting, it communicates with the industrial computer, and the projection image is input into the three-dimensional reconstruction module. The three-dimensional reconstruction module uses a three-dimensional image reconstruction method (such as FDK algorithm) to reconstruct the projection image into a three-dimensional reconstruction body. Optionally, an external trigger mode is used to complete the Xray shooting linkage. Specifically, the linear drive motor 7131, the motion control card, and the Xray detector form a shooting linkage, which is specifically implemented as the linear drive motor 7131 to complete accurate rotational motion, and the rotating skeleton 7133 sends a signal to the motion control card every 1° rotation, and the control relay completes the high-voltage trigger change of the Xray detector (i.e., ends an exposure shooting, and obtains an Xray projection image), and obtains 180 or 360 pictures after rotating 180° or 360°, and now the Xray shooting of the Xray projection image group of a durian is completed.

[0060] Based on this, the pipeline control process of the entire durian detection system is as follows:

[0061] Step 1: If the first photoelectric device 701 detects an object (i.e., the durian to be detected), the first entrance radiation protection door 707 and the entrance conveyor belt 706 are opened;

[0062] Step 2: If the second photoelectric device 702 detects an object and the third photoelectric device 703 detects an object, close the first entrance radiation protection door 707 and the entrance conveyor belt 706;

[0063] Step 3: If the second photoelectric device 702 detects an object and the third photoelectric device 703 does not detect an object, close the first entrance radiation protection door 707, open the second entrance radiation protection door 708, and open the entrance conveyor belt 706;

[0064] Step 4: If the third photoelectric device 703 detects an object, the internal roller 709 is stopped, the lifting cylinder 7132 is raised, and the rotation shooting process begins. After the shooting is completed, the lifting cylinder 7132 is lowered;

[0065] Step 5: If the third photoelectric device 703 detects an object and the fourth photoelectric device 704 does not detect an object, open the first exit radiation protection door 710 and open the internal roller 709;

[0066] Step 6: If the fourth photoelectric device 704 detects an object, the second exit radiation protection door 711 is opened, the first exit radiation protection door 710 is closed, and the exit conveyor belt 712 is opened;

[0067] Step 7: If the fifth photoelectric device 705 detects an object, the exit conveyor belt 712 is closed, and the second exit radiation protection door 711 is closed.

[0068] In the actual application scenario, durian is inspected with the pipeline architecture of the entire durian detection system as the background. The number of durians detected each time is 1, and there is no upper limit on the durian detection input. The construction of the durian dataset and the non-destructive testing process can be carried out simultaneously. Therefore, it can be considered that the total number of test sets of the durian dataset is the total number of durians selected for testing.

[0069] In one of the field tests, 20 durians were selected for test verification, including 5 worm-eaten fruits and 15 good fruits. Among the 15 good fruits, 2 were less than 6 ripe, 5 were 6-7 ripe, 5 were 8-9 ripe, and 3 were 10 ripe and over-ripe. Each durian entered the durian detection system through the pipeline in turn. The final maturity classification results of the durian were 2 less than 6 ripe, 5 were 6-7 ripe, 4 were 8-9 ripe, 10 ripe and over-ripe. The corresponding classification accuracy was 85%, and the recall rate of 6-7 ripe was 80%. The corresponding confusion matrix is ​​shown in the following table:

[0070] Table 1. Confusion matrix of maturity classification (field test of 20 durians)

[0071]

[0072] The final insect-infested fruit identification result is: 3 out of 5 insect-infested fruits are identified as insect-infested fruits, and 2 are identified as good fruits, with a classification accuracy of 60%.

[0073] During the operation of the pipeline, the operation of the durian machine can be roughly divided into three stages, including the X-ray shooting stage, the 3D reconstruction stage, and the model prediction stage. For a single durian, the three stages are carried out in series, taking 6 seconds (180 degrees, 180 photos), 5.21 seconds (RTX4090 graphics card), and 2 seconds (model prediction + image preprocessing). For the pipeline, each stage can be operated in parallel, that is, the three stages can process three different durians, so the average prediction time for a durian is 6 seconds per durian.

[0074] Furthermore, this embodiment also provides a storage medium on which a program is stored. When the program is executed, one or more steps of the aforementioned durian nondestructive testing method are implemented, and the specific implementation process will not be repeated.

[0075] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A durian nondestructive testing method, characterized in that, The following steps are involved: Obtaining a three-dimensional reconstruction of the durian to be tested; Based on the three-dimensional reconstruction, respectively, the maturity classification model and the insect-fruit recognition model that have been trained are used to perform maturity prediction and insect-fruit prediction, and corresponding maturity classification results and insect-fruit recognition results are obtained; Among them, the maturity classification model is constructed based on the multi-scale VGG network and trained using the pre-acquired durian maturity data set; the insect fruit identification model is constructed based on the ResNet3D network and trained using the pre-screened durian slice data set.

2. The durian nondestructive testing method according to claim 1, wherein The maturity classification model includes a multi-scale VGG network, a threshold segmentation processing module and a classification fully connected layer. The multi-scale VGG network includes a first convolutional layer, an intermediate layer and an embedding layer connected in sequence. The maturity classification model trained is used to perform maturity prediction. The specific process includes: Inputting the three-dimensional reconstructed tomogram obtained according to the three-dimensional reconstructed body into the first convolution layer to obtain a first intermediate feature, a second intermediate feature and a third intermediate feature respectively, and inputting the three-dimensional reconstructed tomogram into a threshold segmentation processing module to obtain a density feature; The first intermediate feature, the second intermediate feature and the third intermediate feature are respectively processed by corresponding intermediate layers and then spliced ​​in the embedding layer to obtain a shape feature; The shape feature and the density feature are concatenated and input into the classification fully connected layer, and the final maturity classification result is output.

3. The durian nondestructive testing method according to claim 2, wherein The convolution kernel sizes of the first convolution layer include 3×3, 15×15 and 96×96, which correspond to outputting the first intermediate feature, the second intermediate feature and the third intermediate feature, respectively.

4. The durian nondestructive testing method according to claim 2, wherein The training process of the maturity classification model specifically includes: Performing image processing and analysis on the three-dimensional reconstructed body to obtain a three-dimensional reconstructed tomogram; The three-dimensional reconstructed tomogram is input into the constructed maturity classification model, and is trained using the durian maturity data set and a cross entropy loss value, wherein the cross entropy loss value is calculated using shape features and density features extracted at each iteration.

5. The durian nondestructive testing method according to claim 1, wherein The screening process of the durian slice data set specifically includes: Using a sliding window of a preset size to traverse a preset three-dimensional reconstructed volume data set, and sequentially calculating the average values ​​of the voxel values ​​of the blocks within the sliding window; When the voxel value average value meets the first preset condition, standard deviation screening is performed; When the screened standard deviation meets the second preset condition, the current cut is determined to be a durian cut for training; Among them, the first preset condition is that the voxel value average is between the first hyperparameter and the second hyperparameter, and the first hyperparameter and the second hyperparameter are used to determine that the cut pieces are at the boundary of the durian shell; the second preset condition is that the standard deviation is greater than the third hyperparameter, and the third hyperparameter is used to determine that the cut pieces are not distributed in the gaps or empty rooms of the durian shell.

6. The durian nondestructive testing method according to claim 1, wherein The trained insect-fruit recognition model is used to predict insect-fruit, which specifically includes the following steps: S601, performing sliding window cutting on the three-dimensional reconstructed body to select the current cut block for insect-infested fruit prediction; S602, inputting the selected current cut block into the trained insect fruit recognition model to obtain a preliminary recognition result; S603, when the preliminary recognition result is good, return to S601; S604, when the preliminary recognition result is an insect fruit, determine whether the current cut is a vertex cut, if so, return to step S601, otherwise execute S605, the vertex cut is the last cut of the current three-dimensional reconstruction volume sliding window; S605, obtaining other cut pieces whose Manhattan distance to the current cut piece is 1, and inputting the trained insect-fruit recognition model to obtain a second recognition result, if the second recognition result contains insect-fruit, the final recognition result of the current cut piece is insect-fruit, otherwise the final recognition result of the current cut piece is good fruit; S606, after the sliding window ends, the number of insect-infested fruits in the final recognition results of all the cut pieces is obtained. If it is greater than a preset threshold, the final recognition result of the current durian to be detected is an insect-infested fruit.

7. The durian nondestructive testing method according to any one of claims 1 to 6, characterized in that The process of obtaining the three-dimensional reconstruction volume specifically includes: Get the projection image of the durian to be detected; The projection image is reconstructed into a three-dimensional reconstruction volume using the FDK algorithm.

8. A durian nondestructive testing device, characterized in that: The device is used to implement the method according to any one of claims 1 to 6, and the device at least comprises a three-dimensional reconstruction module and a model prediction module, wherein: The three-dimensional reconstruction module is used to obtain a three-dimensional reconstruction of the durian to be detected; The model prediction module is used to perform maturity prediction and insect-infested fruit prediction based on the three-dimensional reconstruction volume using the trained maturity classification model and insect-infested fruit recognition model to obtain corresponding maturity classification results and insect-infested fruit recognition results; Among them, the maturity classification model is constructed based on the multi-scale VGG network and trained using the pre-acquired durian maturity data set; the insect fruit identification model is constructed based on the ResNet3D network and trained using the pre-screened durian slice data set.

9. The durian nondestructive testing device according to claim 8, characterized in that, The device also includes an X-ray shooting module, which is used to obtain a projection image of the durian to be detected, and the projection image is input into the three-dimensional reconstruction module.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 6 is implemented.

Images