A Mildewed Melon Seeds Calibration Method and System Based on Machine Vision

Abstract

The present invention discloses a mildewed melon seed calibration method and system based on machine vision, including: S1: During the conveying process of melon seeds, double-camera positions are used to collect melon seed images and fuse the images; S2: Remove the stains on the melon seed images to obtain the complete images of each melon seed; S3: Construct a prototype network and perform pre-training. Use the pre-trained prototype network model to identify and remove qualified melon seed images from the complete images of melon seeds to obtain mildewed melon seed images; S4: Select the border of the fused melon seed image as the reference line for camera calibration to obtain camera parameters; S5: Determine the picking order of mildewed melon seeds and calibrate the actual mildewed melon seeds. The present invention automatically realizes the identification of mildewed melon seeds through image processing and deep learning methods; aiming at the large intra-class differences between qualified melon seeds and mildewed melon seeds, the prototype network method is used to improve the classification accuracy; output the position data of mildewed melon seeds in sequence to improve the calibration efficiency of mildewed melon seeds.

Description

Technical Field

[0001] The present invention relates to the technical field of mildewed melon seed recognition, and in particular to a calibration method and system for mildewed melon seeds based on machine vision. Background Art

[0002] Melon seeds are extremely prone to oxidation and mildew due to improper storage. Mildewed melon seeds have a poor taste, and eating such melon seeds is very harmful to physical health. At present, the detection of mildewed melon seeds is mainly completed manually. This method has low efficiency and cannot be applied to large-scale melon seed production lines. At the same time, the shapes, colors, and textures of mildewed melon seeds are complex and diverse, and the features are not unified, so they cannot be screened based on a single feature. The melon seed images obtained on the production line are unclear, incomplete, and contaminated. The existing screening technologies for mildewed melon seeds based on shape, color, and texture features have poor robustness and cannot meet the real-time requirements during the production process. Summary of the Invention

[0003] In view of this, the present invention provides a calibration method and system for mildewed melon seeds based on machine vision, aiming to provide a recognition method and system for mildewed melon seeds based on machine vision and image processing, and obtain a complete melon seed image by fusing multi-source images and processing image stains. Since the features of qualified melon seed images are relatively obvious, while the appearance of mildewed melon seeds is complex and the features are not unified. Therefore, further, a pre-trained prototype network model is used to identify and remove qualified melon seed images from the complete melon seed images, so as to obtain a mildewed melon seed recognition result with high accuracy. At the same time, the true position of the mildewed melon seeds is obtained by combining camera calibration, and based on considering the picking order of the mildewed melon seeds, the position data of the mildewed melon seeds is output in sequence to improve the calibration efficiency of the mildewed melon seeds.

[0004] To achieve the above object, a calibration method for mildewed melon seeds based on machine vision provided by the present invention includes the following steps:

[0005] S1: During the conveying process of the melon seeds, dual cameras are used to collect melon seed images, and the melon seed images collected by the dual cameras are fused to obtain a fused melon seed image;

[0006] S2: The fused melon seed image is preprocessed to remove stains in the image, and a preprocessed image is obtained; two-dimensional constant false alarm detection is performed on the preprocessed image, and combined with dilation processing, a complete image of each melon seed is obtained;

[0007] S3: A prototype network is constructed and pre-trained to obtain a pre-trained prototype network model; the pre-trained prototype network model is used to identify and remove qualified melon seed images from the complete melon seed images to obtain mildewed melon seed images;

[0008] S4: Select the border of the fused melon seed image obtained in step S1 as the reference line, perform camera calibration, and obtain camera parameters;

[0009] S5: Determine the sorting order of moldy melon seeds and calibrate the actual moldy melon seeds.

[0010] As a further improvement method of the present invention:

[0011] Optionally, the step S1 includes:

[0012] S101: Place two cameras on the front and side of the melon seed sorting channel, perform time calibration on the cameras to ensure that the cameras start synchronously and the image acquisition intervals are consistent, and respectively collect the front image Sxzm and the side image Sxcm of the melon seeds;

[0013] S102: Perform scale space transformation on the images Sxzm and Sxcm respectively:

[0014] C zm = Sxzm * Xszm

[0015] C cm = Sxcm * Xscm

[0016] Wherein, C zm represents the information of the image Sxzm in the new scale space domain, C cm represents the information of the image Sxcm in the new scale space domain, Xszm represents the scale coefficient of the image Sxzm, and Xscm represents the scale coefficient of the image Sxcm;

[0017] S103: Determine the occluded area Ω of the image Sxzm in the new scale space domain, and calculate the gradient field Ru of the images Sxzm and Sxcm in the area Ω:

[0018] Ru = Rzm + Rcm

[0019] Wherein, Rzm represents the gradient field of the image Sxzm in the area Ω, and Rcm represents the gradient field of the image Sxcm in the area Ω;

[0020] S104: Establish a Poisson equation, solve for the pixel values, and obtain the fused melon seed image:

[0021] Xs·x = Sd

[0022] Wherein, Xs represents the coefficient matrix constructed, x represents the pixel value of the fused image to be solved, and Sd represents the divergence value of the gradient field Ru.

[0023] Optionally, in the step S2, preprocess the fused melon seed image to remove stains in the image, including:

[0024] S201: Apply the histogram equalization method to the fused melon seed images to adjust the gray-scale distribution of the images and enhance the contrast of the fused melon seed images;

[0025] S202: Select any point Wd in the contaminated area of the fused melon seed images, select the reference standard point Bd, and calculate the first-order pixel derivative of point Wd:

[0026] ▽I(Wd) = (I(Bd) - I(Wd)) / (Wd - Bd)

[0027] where ▽I(Wd) represents the first-order pixel derivative of point Wd, I(Bd) represents the pixel value of the reference point Bd, the reference point Bd is located in the neighborhood of Wd, I(Wd) represents the pixel value of point Wd, and (Wd - Bd) represents the distance between point Wd and the reference point Bd;

[0028] S203: Calculate the new pixel value of point Wd:

[0029]

[0030] where I′(Wd) represents the new pixel value of point Wd, and ω(Wd,Bd) represents the weight function;

[0031] S204: Repeat steps S202 and S203 until all points in the contaminated area are repaired to obtain the preprocessed image.

[0032] Optionally, in step S2, perform two-dimensional constant false alarm detection on the preprocessed image and combine it with dilation processing to obtain the complete image of each melon seed, including:

[0033] S211: Perform two-dimensional constant false alarm detection on the preprocessed image for target localization to find the target point Mb;

[0034] S212: Select the dilation reference image Pz and perform dilation processing on the target point to obtain the complete image of each melon seed.

[0035] Optionally, in step S3, construct a prototype network and perform pre-training to obtain a pre-trained prototype network model, including:

[0036] S301: Use the qualified melon seed image set and the mildewed melon seed image set to construct a support set and a query set;

[0037] S302: Use the ResNet network to extract feature information from each image in the support set, use the attention mechanism to perform enhanced feature extraction on the feature information, and then use the prototype network to perform mean processing on the features to generate the prototype representations of qualified melon seeds and mildewed melon seeds;

[0038] S303: Extract the feature information of each image in the query set using the ResNet network, enhance the feature extraction of the feature information using the attention mechanism, and then perform a mean processing on the features using the prototype network to obtain the prototype representation of the query set images;

[0039] S304: Calculate the Manhattan distances between the prototype representation of the query set images and the prototype representations of qualified melon seeds and mildewed melon seeds to realize the judgment of the query set images;

[0040] S305: Repeat steps S302, S303, and S304 for training to obtain a pre-trained prototype network model.

[0041] Optionally, in step S3, using the pre-trained prototype network model to identify and remove the qualified melon seed images from the complete images of melon seeds to obtain the mildewed melon seed images, including:

[0042] S311: Construct a test data set using the complete images of melon seeds, and use the pre-trained prototype network model to identify the test data set to obtain the qualified melon seed images;

[0043] S312: Remove the qualified melon seed images from the complete images of melon seeds to obtain the mildewed melon seed images.

[0044] Optionally, step S4 includes:

[0045] S401: Select the fused melon seed image border obtained in step S1 as the reference line, and determine the corresponding relationship between the reference line and the camera border:

[0046] Sfd·Sbk = Snc·Sfx·Ck

[0047] where Sfd represents the scaling factor of each point on the camera border, Sbk represents the coordinates of each point on the camera border, Snc represents the camera parameter matrix to be solved, Sfx represents the rotation matrix, and Ck represents the coordinates of each point on the reference line;

[0048] S402: Establish the Kruppa equation using the quadratic surface and fine-tune the camera parameter matrix Snc through self-calibration;

[0049] S403: The elements in the matrix Snc are the camera parameters.

[0050] Optionally, in step S5, determining the picking order of the mildewed melon seeds includes:

[0051] S501: Mark the mildewed melon seeds in the fused melon seed images according to the mildewed melon seed images;

[0052] S502: In the fused melon seed image marked with mildewed melon seeds, demarcate the mildewed melon seeds in each row:

[0053] Mx1 - Mx2 ≤ Hj

[0054] Wherein, Mx1 and Mx2 represent the abscissas of two mildewed melon seeds, and Hj represents the row spacing;

[0055] In the fused image, demarcate the mildewed melon seeds in each column:

[0056] My1 - My2 ≤ Lj

[0057] Wherein, My1 and My2 represent the ordinates of two mildewed melon seeds, and Lj represents the column spacing;

[0058] S503: Starting from the first row and the first column, number the mildewed melon seeds in sequence to determine the picking order of the mildewed melon seeds.

[0059] Optionally, in step S5, calibrating the actual mildewed melon seeds includes:

[0060] S511: Determine the position of the mildewed melon seeds in the camera coordinate system:

[0061] Sxw · Snc = Txw

[0062] Wherein, Sxw represents the position of the mildewed melon seeds in the camera coordinate system, Snc represents the fine-tuned camera parameter matrix, and Txw represents the position of the mildewed melon seeds in the fused melon seed image;

[0063] Determine the position of the mildewed melon seeds in the earth coordinate system:

[0064] Sxw = Xz · Sjw + Py

[0065] Wherein, Xz represents the rotation relationship matrix between the camera coordinate system and the earth coordinate system, Sjw represents the position of the mildewed melon seeds in the earth coordinate system, and Py represents the translation relationship matrix between the camera coordinate system and the earth coordinate system;

[0066] S512: According to the picking order of the mildewed melon seeds, output the actual position information of the mildewed melon seeds in sequence.

[0067] The present invention also provides a mildewed melon seed calibration system based on machine vision, including:

[0068] Image fusion module: Collect melon seed images, perform scale transformation and image fusion on the images, and establish an image fusion quality evaluation system;

[0069] Image preprocessing module: Enhance the contrast of the fused image, repair the contaminated area, and perform two-dimensional constant false alarm detection to obtain the complete image of the melon seeds;

[0070] Image recognition module: Construct an image set, perform feature extraction and pre-training of the prototype network, recognize qualified melon seed images, and obtain moldy melon seed images after rejection.

[0071] Camera calibration module: Select a reference line, perform camera calibration, and obtain camera parameters.

[0072] Moldy melon seed calibration module: Determine the picking order of moldy melon seeds and perform calibration of moldy melon seeds.

[0073] Beneficial effects

[0074] Through the fusion of multi-source images, image stain processing, and a two-stage deep learning network, the present invention obtains a high-accuracy recognition result of moldy melon seeds. At the same time, by combining camera calibration, the accurate real position of moldy melon seeds is obtained, and based on considering the picking order of moldy melon seeds, the position data of moldy melon seeds is output in sequence, improving the calibration efficiency of moldy melon seeds.

[0075] The method of the present invention fuses and removes stains from the melon seed images of two cameras, ensuring that the images are unobstructed and clear; through the deep learning network, qualified melon seeds with relatively consistent features are recognized, and moldy melon seeds are obtained through rejection operations, improving the recognition accuracy of moldy melon seeds. It reduces the work of collecting diverse moldy melon seed image data. After performing scale transformation on the images, the method of the present invention uses the Poisson equation to solve for the pixel values of the fused image, effectively retaining the gradient information of the source images and achieving seamless fusion of the new images.

[0076] Through two-dimensional constant false alarm detection combined with dilation processing, the present invention is conducive to obtaining a complete image of melon seeds for further recognition by the subsequent network; through the attention mechanism, the deep learning network is strengthened to extract effective features of melon seeds, and training the prototype network is beneficial to improving the recognition performance. The present invention uses a self-calibration method to fine-tune the camera parameter matrix, which is beneficial to improving the accuracy of the camera parameter matrix; by spatially sorting the numbers of moldy melon seeds, it is beneficial to determine the optimal picking order of melon seeds. Brief description of the drawings

[0077] Figure 1 It is a schematic flowchart of a method for calibrating moldy melon seeds based on machine vision according to an embodiment of the present invention.

[0078] Figure 2 It is a schematic diagram of the output result after processing step S3 according to an embodiment of the present invention.

[0079] Figure 3 It is another schematic diagram of the output result after processing step S3 according to an embodiment of the present invention. Detailed implementation manners

[0080] The present invention will be further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any transformation or replacement made based on the teachings of the present invention falls within the protection scope of the present invention.

[0081] Example 1:

[0082] A method for calibrating mildewed melon seeds based on machine vision, as Figure 1 shown, includes the following steps:

[0083] S1: During the conveying process of the melon seeds, collect the melon seed images by using a dual-camera setup, and fuse the melon seed images collected by the dual-camera setup to obtain the fused melon seed images;

[0084] S101: Place two cameras on the front and side of the melon seed sorting channel, calibrate the cameras in terms of time to ensure that the cameras start synchronously and the image acquisition intervals are consistent, and respectively collect the front image Sxzm and the side image Sxcm of the melon seeds;

[0085] S102: Perform scale-space transformation on the images Sxzm and Sxcm respectively:

[0086] C zm = Sxzm * Xszm

[0087] C cm = Sxcm * Xscm

[0088] Among them, C zm represents the information of the image Sxzm in the new scale-space domain, C cm represents the information of the image Sxcm in the new scale-space domain, Xszm represents the scale coefficient of the image Sxzm, and Xscm represents the scale coefficient of the image Sxcm;

[0089] S103: Determine the occluded region Ω of the image Sxzm in the new scale-space domain, and calculate the gradient field Ru of the images Sxzm and Sxcm in the region Ω:

[0090] Ru = Rzm + Rcm

[0091] Among them, Rzm represents the gradient field of the image Sxzm in the region Ω, and Rcm represents the gradient field of the image Sxcm in the region Ω;

[0092] S104: Establish a Poisson equation, solve for the pixel values, and obtain the fused melon seed images:

[0093] Xs·x = Sd

[0094] Among them, Xs represents the coefficient matrix constructed, x represents the pixel values of the fused image to be solved, and Sd represents the divergence value of the gradient field Ru.

[0095] S2: Preprocess the fused melon seed images to remove stains in the images and obtain the preprocessed images; perform two-dimensional constant false alarm detection on the preprocessed images and combine it with dilation processing to obtain the complete images of each melon seed;

[0096] S201: Use the histogram equalization method for the fused melon seed images to adjust the gray level distribution of the images and enhance the contrast of the fused melon seed images;

[0097] S202: Select any point Wd in the contaminated area of the fused melon seed images, select the reference standard point Bd, and calculate the first-order pixel derivative of point Wd:

[0098] ▽I(Wd) = (I(Bd) - I(Wd)) / (Wd - Bd)

[0099] where, ▽I(Wd) represents the first-order pixel derivative of point Wd, I(Bd) represents the pixel value of the reference point Bd, the reference point Bd is located in the neighborhood of Wd, I(Wd) represents the pixel value of point Wd, and (Wd - Bd) represents the distance between point Wd and the reference point Bd;

[0100] S203: Calculate the new pixel value of point Wd:

[0101]

[0102] where, I′(Wd) represents the new pixel value of point Wd, and ω(Wd, Bd) represents the weight function;

[0103] S204: Repeat steps S202 and S203 until all points in the contaminated area are repaired to obtain the preprocessed images.

[0104] S205: Perform two-dimensional constant false alarm detection on the preprocessed images for target localization to find the target point Mb;

[0105] S206: Select the dilation reference image Pz and perform dilation processing on the target point to obtain the complete images of each melon seed.

[0106] S3: Construct a prototype network and perform pre-training to obtain a pre-trained prototype network model; use the pre-trained prototype network model to identify and remove qualified melon seed images from the complete images of melon seeds, refer to Figures 2-3 to obtain the mildewed melon seed images;

[0107] S301: Manually pre-sort the actual melon seeds in advance, determine the qualified melon seeds and the mildewed melon seeds, and take pictures of the two types of melon seeds respectively to obtain a qualified melon seed image set and a mildewed melon seed image set;

[0108] S302: Construct a support set and a query set using the qualified melon seed image set and the mildewed melon seed image set;

[0109] S303: Use the ResNet network to extract feature information from each image in the support set, use the attention mechanism to perform enhanced feature extraction on the feature information, and then use the prototype network to perform mean processing on the features to generate a qualified melon seed prototype representation and a mildewed melon seed prototype representation;

[0110] S304: Use the ResNet network to extract feature information from each image in the query set, use the attention mechanism to perform enhanced feature extraction on the feature information, and then use the prototype network to perform mean processing on the features to obtain the query set image prototype representation;

[0111] S305: Calculate the Manhattan distances between the query set image prototype representation and the qualified melon seed prototype representation and the mildewed melon seed prototype representation to realize the judgment of the query set images;

[0112] S306: Repeat steps S303, S304, and S305 for training to obtain a pre-trained prototype network model;

[0113] S307: Perform size normalization and orientation normalization on the complete images of the melon seeds obtained in step S206 to construct a test data set;

[0114] S308: Use the pre-trained prototype network model to identify the test data set to obtain qualified melon seed images;

[0115] S309: Remove the qualified melon seed images from the complete images of the melon seeds obtained in step S206 to obtain mildewed melon seed images.

[0116] S4: Select the fused melon seed image border obtained in step S1 as a reference line to perform camera calibration to obtain camera parameters;

[0117] S401: Select the fused melon seed image border obtained in step S1 as a reference line to determine the corresponding relationship between the reference line and the camera border:

[0118] Sfd·Sbk = Snc·Sfx·Ck

[0119] where, Sfd represents the scaling coefficient of each point on the camera border, Sbk represents the coordinates of each point on the camera border, Snc represents the camera parameter matrix to be solved, Sfx represents the rotation matrix, and Ck represents the coordinates of each point on the reference line;

[0120] S402: Use a quadratic surface to establish the Kruppa equation and fine-tune the camera parameter matrix Snc through self-calibration;

[0121] S403: The elements in the matrix Snc are the camera parameters.

[0122] S5: Determine the sorting order of the mildewed melon seeds and calibrate the actual mildewed melon seeds.

[0123] S501: Mark the mildewed melon seeds in the fused melon seed image according to the mildewed melon seed image;

[0124] S502: In the fused melon seed image with marked mildewed melon seeds, delimit the mildewed melon seeds in each row:

[0125] Mx1 - Mx2 ≤ Hj

[0126] where Mx1 and Mx2 represent the abscissas of two mildewed melon seeds, and Hj represents the row spacing;

[0127] In the fused image, delimit the mildewed melon seeds in each column:

[0128] My1 - My2 ≤ Lj

[0129] where My1 and My2 represent the ordinates of two mildewed melon seeds, and Lj represents the column spacing;

[0130] S503: Starting from the first row and the first column, number the mildewed melon seeds in sequence to determine the sorting order of the mildewed melon seeds.

[0131] S504: Determine the position of the mildewed melon seeds in the camera coordinate system:

[0132] Sxw · Snc = Txw

[0133] where Sxw represents the position of the mildewed melon seeds in the camera coordinate system, Snc represents the fine-tuned camera parameter matrix, and Txw represents the position of the mildewed melon seeds in the fused melon seed image;

[0134] Determine the position of the mildewed melon seeds in the earth coordinate system:

[0135] Sxw = Xz · Sjw + Py

[0136] where Xz represents the rotation relationship matrix between the camera coordinate system and the earth coordinate system, Sjw represents the position of the mildewed melon seeds in the earth coordinate system, and Py represents the translation relationship matrix between the camera coordinate system and the earth coordinate system;

[0137] S505: According to the sorting order of the mildewed melon seeds, output the actual position information of the mildewed melon seeds in sequence.

[0138] Embodiment 2:

[0139] A method and system for calibrating mildewed melon seeds based on machine vision, including the following five modules:

[0140] Image fusion module: Collect melon seed images, perform scale transformation and image fusion on the images, and establish an image fusion quality evaluation system;

[0141] Image preprocessing module: Enhance the contrast of the fused image, repair the contaminated area, perform two-dimensional constant false alarm detection, and obtain the complete image of the melon seeds;

[0142] Image recognition module: Construct an image set, perform feature extraction and prototype network pre-training, recognize qualified melon seed images, and obtain mildewed melon seed images after rejection;

[0143] Camera calibration module: Select a reference line, perform camera calibration, and obtain camera parameters;

[0144] Mildewed melon seed calibration module: Determine the picking order of mildewed melon seeds and perform mildewed melon seed calibration.

[0145] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments. And the terms "including", "comprising" or any other variant thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, device, article or method. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, device, article or method including the element.

[0146] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) as described above and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0147] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for calibrating moldy melon seeds based on machine vision, characterized in that: include: Step S1: During the melon seed transmission process, two cameras are used to collect melon seed images, and the melon seed images collected by the two cameras are fused to obtain a fused melon seed image; The step S1 comprises: S101: two cameras are placed on the front and side of the melon seed sorting channel to collect the front image Sxzm and side image Sxcm of the melon seeds respectively; S102: Perform scale space transformation on images Sxzm and Sxcm respectively: C zm =Sxzm*Xszm C cm =Sxcm*Xscm Among them, C zm Represents the information of the image Sxzm in the new scale space domain, C cm Represents the information of the image Sxcm in the new scale space domain, Xszm represents the scale coefficient of the image Sxzm, and Xscm represents the scale coefficient of the image Sxcm; S103: Determine the region Ω where the image Sxzm is blocked in the new scale space domain, and calculate the gradient field Ru of the images Sxzm and Sxcm in the region Ω: Ru=Rzm+Rcm Among them, Rzm represents the gradient field of the image Sxzm in the region Ω, and Rcm represents the gradient field of the image Sxcm in the region Ω; S104: Establish Poisson's equation, solve pixel values, and obtain the fused melon seed image: Xs·x=Sd Among them, Xs represents the constructed coefficient matrix, x represents the pixel value of the fused image to be solved, and Sd represents the divergence value of the gradient field Ru; Step S2: preprocessing the fused melon seed image to remove stains in the image and obtain a preprocessed image; using two-dimensional constant false alarm detection on the preprocessed image and combining it with expansion processing to obtain a complete image of each melon seed; Step S3: constructing a prototype network and performing pre-training to obtain a pre-trained prototype network model; using the pre-trained prototype network model to identify qualified melon seeds images from the complete images of melon seeds and remove them to obtain moldy melon seeds images; In step S3, a prototype network is constructed and pre-trained to obtain a pre-trained prototype network model, including: S301: constructing a support set and a query set using a qualified melon seed image set and a moldy melon seed image set; S302: using a ResNet network to extract feature information from each image in the support set, using an attention mechanism to enhance feature extraction of the feature information, and then using a prototype network to perform mean processing on the features to generate a prototype representation of qualified melon seeds and a prototype representation of moldy melon seeds; S303: using a ResNet network to extract feature information from each image in the query set, using an attention mechanism to enhance feature extraction of the feature information, and then using a prototype network to perform mean processing on the features to obtain a prototype representation of the query set image; S304: Calculate the Manhattan distance between the prototype representation of the query set image and the prototype representation of the qualified melon seeds and the prototype representation of the moldy melon seeds to determine the query set image; S305: Repeat steps S302, S303 and S304 to perform training and obtain a pre-trained prototype network model; In step S3, a pre-trained prototype network model is used to identify qualified melon seed images from the complete melon seed images and remove them to obtain moldy melon seed images, including: S311: construct a test data set using the complete image of melon seeds, and use the pre-trained prototype network model to identify the test data set to obtain qualified melon seed images; S312: Eliminate qualified melon seed images from the complete melon seed images to obtain moldy melon seed images; Step S4: Select the frame of the fused melon seed image obtained in step S1 as a reference line, perform camera calibration, and obtain camera parameters; Step S5: Determine the order of picking the moldy melon seeds, and calibrate the actual moldy melon seeds according to the camera parameters obtained in step S4.

2. The method for calibrating moldy melon seeds based on machine vision according to claim 1, characterized in that: In step S2, the fused melon seed image is preprocessed to remove stains in the image, including: S201: using a histogram equalization method on the fused melon seed image to adjust the image grayscale distribution and enhance the contrast of the fused melon seed image; S202: Select any point Wd in the contaminated area of ​​the fused melon seed image, select a reference standard point Bd, and calculate the first-order derivative of the pixel at point Wd: in, represents the first-order derivative of the pixel at point Wd, I(Bd) represents the pixel value of the reference point Bd, the reference point Bd is located in the neighborhood of Wd, I(Wd) represents the pixel value of point Wd, (Wd-Bd) represents the distance between point Wd and reference point Bd; S203: Calculate the new pixel value of point Wd: Where I′(Wd) represents the new pixel value of point Wd, and ω(Wd,Bd) represents the weight function; S204: Repeat steps S202 and S203 until all points in the polluted area are repaired to obtain a preprocessed image.

3. The method for calibrating moldy melon seeds based on machine vision according to claim 1, characterized in that: In step S2, two-dimensional constant false alarm detection is used on the pre-processed image, and expansion processing is combined to obtain a complete image of each melon seed, including: S211: Using two-dimensional constant false alarm detection on the preprocessed image to locate the target and find the target point Mb; S212: Select the dilation reference image Pz and perform dilation processing on the target point Get a complete image of each melon seed.

4. The method for calibrating moldy melon seeds based on machine vision according to claim 1, characterized in that: The step S4 comprises: S401: Select the frame of the fused melon seed image obtained in step S1 as a reference line, and determine the corresponding relationship between the reference line and the camera frame: Sfd·Sbk=Snc·Sfx·Ck Among them, Sfd represents the scaling factor of each point on the camera frame, Sbk represents the coordinates of each point on the camera frame, Snc represents the camera parameter matrix to be solved, Sfx represents the rotation matrix, and Ck represents the coordinates of each point on the reference line; S402: Use quadratic surfaces to establish the Kruppa equation and fine-tune the camera parameter matrix Snc through self-calibration; S403: The elements in the matrix Snc are the camera parameters.

5. The method for calibrating moldy melon seeds based on machine vision according to claim 1, characterized in that: In step S5, determining the order of picking moldy melon seeds includes: S501: marking the moldy melon seeds in the fused melon seeds image according to the moldy melon seeds image; S502: In the fused melon seed image in which the moldy melon seeds are marked, demarcate each row of moldy melon seeds: Mx1-Mx2≤Hj Among them, Mx1, Mx2 represent the horizontal coordinates of two moldy melon seeds, and Hj represents the row spacing; In the fused image, delineate the moldy melon seeds in each column: My1-My2≤Lj Among them, My1, My2 represent the vertical coordinates of two moldy melon seeds, and Lj represents the column spacing; S503: Starting from the first row and the first column, the moldy melon seeds are numbered in sequence to determine the order in which the moldy melon seeds are picked.

6. The method for calibrating moldy melon seeds based on machine vision according to claim 1, characterized in that: In step S5, the actual moldy melon seeds are calibrated including: S511: Determine the position of the moldy melon seeds in the camera coordinate system: Sxw=Snc·Txw Among them, Sxw represents the position of the moldy melon seeds in the camera coordinate system, Snc represents the fine-tuned camera parameter matrix, and Txw represents the position of the moldy melon seeds in the fused melon seed image; Determine the position of the moldy melon seeds in the geodetic coordinate system: Sxw=Xz·Sjw+Py Among them, Xz represents the rotation relationship matrix between the camera coordinate system and the earth coordinate system, Sjw represents the position of the moldy melon seeds in the earth coordinate system, and Py represents the translation relationship matrix between the camera coordinate system and the earth coordinate system; S512: Outputting the actual location information of the moldy melon seeds in sequence according to the order in which the moldy melon seeds are picked.

7. A machine vision-based system for identifying moldy melon seeds, characterized in that: include: Image fusion module: collect melon seed images, perform image scale transformation and image fusion on them, and establish an image fusion quality evaluation system; Image preprocessing module: enhances the contrast of the fused image, repairs the contaminated area, and performs two-dimensional constant false alarm detection to obtain a complete image of the melon seeds; Image recognition module: construct an image set, perform feature extraction and prototype network pre-training, identify qualified melon seed images, and remove moldy melon seed images; Camera calibration module: select reference lines, perform camera calibration, and obtain camera parameters; Moldy melon seeds calibration module: determines the picking order of moldy melon seeds and calibrates the moldy melon seeds according to the obtained camera parameters; To realize a method for calibrating moldy melon seeds based on machine vision as described in any one of claims 1-6.

Images