Multi-dimensional grain screening system and method based on deep learning

Abstract

The application discloses a multi-dimensional grain screening system and method based on deep learning, relates to the technical field of agricultural product processing, and solves the technical problem that the prior art lacks consideration of the influence of the screening environment on the visual detection parameters, which reduces the quality of the visual detection features and leads to low accuracy and efficiency of the grain screening system; device adjustment parameter values are generated according to grain data and environment data; adjusted detection images and point cloud data are obtained based on the device adjustment parameter values, and a detection result is generated in combination with voiceprint data; an alarm signal is generated according to the detection result, and screening is performed; data of external defect detection is analyzed in quality, the data quality can be improved by adaptively changing device parameters, external and internal defects of the grain are processed step by step, the screening speed is improved, and resource consumption is reduced; and the data combination weight is dynamically adjusted according to the data quality during external defect detection, and the screening accuracy and efficiency are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of agricultural product processing, and in particular to a multi-dimensional grain screening system and method based on deep learning. BACKGROUND

[0002] Grain screening, as a key link in grain processing and agricultural production, is self-evident in importance. Before grain harvesting, storage and processing, there are often mold and surface defects such as bumps on the surface of the grain, and even insects inside the grain. These problems not only affect the overall quality of the grain and reduce its market value, but also may cause damage to subsequent production products and affect product quality. Therefore, through scientific and effective screening methods such as air separation, screening, specific gravity separation and magnetic separation, the grain is accurately separated to ensure the purity and quality of the grain, which is of great significance to food safety, processing efficiency and sustainable agricultural development.

[0003] The prior art often lacks consideration of the influence of the screening environment on the visual detection parameters when screening the grain, resulting in reduced quality of visual detection features and low accuracy and efficiency of the grain screening system. Therefore, the grain screening system still needs to be further improved. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes a multi-dimensional grain screening system and method based on deep learning to solve the technical problem that the prior art lacks consideration of the influence of the screening environment on the visual detection parameters, resulting in reduced quality of visual detection features and low accuracy and efficiency of the grain screening system.

[0005] To achieve the above-mentioned purpose, the first aspect of the present application provides a multi-dimensional grain screening system based on deep learning, comprising: a data acquisition module, a data analysis module, a warning module and a database; the data acquisition module and the data analysis module are electrically and / or communicatively connected; the data analysis module and the warning module are electrically and / or communicatively connected; the database is electrically and / or communicatively connected with the data acquisition module, the data analysis module and the warning module, respectively;

[0006] The data acquisition module: acquires environment data, grain data and device data through a data acquisition device; the environment data includes dust concentration, temperature, humidity and light intensity; the device data includes device ID and device parameters; the grain data includes grain parameters, detection images, point cloud data and voiceprint data;

[0007] The data analysis module: generates device adjustment parameter values according to the grain data and the environment data; assigns the device adjustment parameter values to corresponding device parameters, obtains adjusted detection images and point cloud data based on the device adjustment parameter values; generates a detection result according to the voiceprint data, the adjusted detection images and the point cloud data; generates an alarm signal according to the detection result and performs screening;

[0008] The early warning module: makes a prompt according to the alarm signal and contacts a manager;

[0009] The database is used to store data of each module and store historical data required for training a model.

[0010] The application carries out in-depth quality analysis on the data obtained by external defect detection through the above steps, flexibly adjusts device parameters with the help of an environment adaptive mechanism, and thus maximizes the data quality. In addition, the application processes grain internal and external defect detection in steps, not only significantly improves the screening speed, but also effectively reduces resource consumption. In the external defect detection process, the application can dynamically adjust data combination weights in real time according to data quality, and thus comprehensively improves the accuracy and efficiency of screening.

[0011] Further, the grain parameters include grain vibration intensity, and the device adjustment parameter values are generated according to the grain data and the environment data, including:

[0012] Obtain grain data and environment data and device parameters;

[0013] Extract the image signal-to-noise ratio and image contrast of the detection image;

[0014] Extract the point cloud integrity and point cloud density of the point cloud data;

[0015] Determine whether the image signal-to-noise ratio and the image contrast are greater than the corresponding parameter thresholds;

[0016] Yes, do nothing;

[0017] No, generate image device adjustment parameter values according to the device parameters and the environment data;

[0018] Determine whether the point cloud integrity and the point cloud density corresponding to the point cloud data are greater than the corresponding parameter thresholds;

[0019] Yes, do nothing;

[0020] No, generate point cloud device adjustment parameter values according to the device parameters, the grain vibration intensity and the environment data;

[0021] The device adjustment parameter values include image device adjustment parameter values and point cloud device adjustment parameter values.

[0022] When conducting external defect detection, this application considers the significant impact of environmental data on the corresponding detection data. Therefore, when the quality of the corresponding detection data is poor, the equipment parameters can be dynamically adjusted according to environmental factors to improve the data quality, thus providing data support for improving the accuracy of grain screening.

[0023] Furthermore, the step of generating image device adjustment parameters based on device parameters and environmental data includes:

[0024] Acquire environmental data and equipment parameters; the environmental data includes dust concentration FN, humidity SD, and light intensity GQ; the equipment parameters include equipment wavelength SB, equipment light intensity SG, and equipment polarization angle SPJ;

[0025] Construct the device optimization function TSYF corresponding to the detected image; the formula of the device optimization function is:

[0026] Wherein, TSNRF(SB, SG, SPJ|FN, SD, GQ) is the image signal-to-noise ratio calculation function for the detected image; TDBF(SB, SG, SPJ|FN, SD, GQ) is the image contrast calculation function for the detected image;

[0027] The dust concentration FN, humidity SD, and light intensity GQ are substituted into the device optimization function, and the image device adjustment parameter values ​​are obtained by solving the device optimization function TSYF using the CPSO algorithm. The image device adjustment parameter values ​​refer to the device parameter values ​​that enable the image signal-to-noise ratio and image contrast of the detected image to reach the optimal values ​​under the current environmental data.

[0028] Furthermore, the step of generating point cloud device adjustment parameter values ​​based on equipment parameters, grain vibration intensity, and environmental data includes:

[0029] Acquire equipment parameters, grain vibration intensity GZQ, and environmental data; the environmental data includes dust concentration FN and light intensity GQ; the equipment parameters include laser wavelength GB, laser power JG, pulse frequency MP, scanning angle SJD, and scanning speed SV;

[0030] Construct the device optimization function DSYF corresponding to the point cloud data; the formula of the device optimization function is:

[0031]

[0032] Wherein, DWXF(GB, JG, MP, SJD, SV|FN, GZQ, GQ) is the point cloud integrity calculation function for point cloud data; DMF(GB, JG, MP, SJD, SV|FN, GZQ, GQ) is the point cloud density calculation function for point cloud data.

[0033] The grain vibration intensity GZQ, the dust concentration FN and the illumination intensity GQ are substituted into the equipment optimization function, and the point cloud equipment adjustment parameter value is obtained by solving the equipment optimization function DSYF through the NSGA-II algorithm; the point cloud equipment adjustment parameter value refers to the equipment parameter value that can make the point cloud integrity and the point cloud density of the point cloud data reach the optimal value under the current environment data.

[0034] The present application carries out deep analysis on the detection image and the point cloud data required for external defect detection, and accordingly constructs a special equipment optimization function. When the detection data quality does not reach the expectation, this function can rely on the current environment data to optimize various indicators of the detection data in all directions and maximize, thereby significantly improving the detection data quality. In the subsequent external defect identification link, the detection data will have more abundant and effective features, which lays a solid data foundation for the improvement of the accuracy of the grain screening system.

[0035] Further, the generating a detection result according to the voiceprint data, the adjusted detection image and the point cloud data comprises:

[0036] Obtaining a detection label, voiceprint data, an adjusted detection image and point cloud data; the detection label comprises an internal detection label and an external detection label;

[0037] Inputting the internal detection label and the voiceprint data into a multi-modal defect identification model to obtain an internal label corresponding to the internal detection label and a defect result; the internal label comprises an internal defect label and a non-internal defect label; the multi-modal defect identification model is constructed by an artificial intelligence model;

[0038] Judging whether the internal label is the internal defect label;

[0039] Yes, no operation is performed;

[0040] No, generating a detection result according to the point cloud data and the detection image.

[0041] Further, the generating a detection result according to the point cloud data and the detection image comprises:

[0042] Obtaining the point cloud data and the detection image and an external detection label;

[0043] Extracting point cloud integrity and point cloud density corresponding to the point cloud data;

[0044] Extracting image signal-to-noise ratio and image contrast corresponding to the detection image;

[0045] Generating a data quality score according to the point cloud integrity, the point cloud density, the image signal-to-noise ratio and the image contrast, and generating a decision weight according to the data quality score;

[0046] The decision weight is combined with the corresponding point cloud data and detection image to obtain decision data;

[0047] The detection outer label and the decision data are input into a multi-modal defect recognition model to obtain an outer label corresponding to the detection outer label and a defect result; the outer label includes an outer defect label and a non-outer defect label; the multi-modal defect recognition model is constructed by an artificial intelligence model.

[0048] Further, the data quality score is generated according to the point cloud integrity, the point cloud density, the image signal-to-noise ratio and the image contrast, and the decision weight is generated according to the data quality score, including:

[0049] The point cloud integrity DWX, the point cloud density DM, the image signal-to-noise ratio TSNR and the image contrast TDB are extracted;

[0050] The point cloud data quality score DZP is calculated by the formula ; wherein min() represents a minimum value operation, and DZF(DWX, DM) represents a point cloud data quality score function constructed according to the nonlinear relationship between the point cloud integrity DWX and the point cloud density DM and the point cloud data quality score;

[0051] The image data quality score TZP is calculated by the formula ; wherein min() represents a minimum value operation, and TZF(TSNR, TDB) represents an image data quality score function constructed according to the nonlinear relationship between the image signal-to-noise ratio TSNR and the image contrast TDB and the image data quality score;

[0052] The detection image weight function TJQF(DZP, TZP) is constructed according to the nonlinear relationship between the point cloud data quality score DZP and the image data quality score TZP and the decision weight corresponding to the detection image;

[0053] The point cloud data quality score DZP and the image data quality score TZP are substituted into the detection image weight function to obtain the decision weight TJQ corresponding to the detection image.

[0054] The decision weight DJQ corresponding to the point cloud data is calculated by the difference between the sum of the weights and the decision weight TJQ corresponding to the detection image.

[0055] The application collects the detection image and the point cloud data generated by the adjusted acquisition device, monitors the quality of the data in real time by means of the plurality of data quality scoring functions constructed in advance, quantifies and limits the data quality scores in a specific range, dynamically adjusts the decision weights corresponding to the scores based on the differences between the scores, ensures that the detection data with higher data quality scores can be allocated to higher decision weights, so that in the defect identification stage, the high-quality data can provide more effective features, thereby effectively improving the accuracy of grain screening.

[0056] Further, the multi-modal defect identification model is constructed by an artificial intelligence model, comprising:

[0057] Obtaining a plurality of historical decision data, historical external labels and historical defect results corresponding to a plurality of detection external labels, and a plurality of historical voiceprint data, historical internal labels and historical defect results corresponding to a plurality of detection internal labels;

[0058] The plurality of historical decision data, historical external labels and historical defect results corresponding to the plurality of detection external labels are divided into training data, verification data and test data corresponding to the detection external labels; and the training data, verification data and test data corresponding to the detection external labels are preprocessed to obtain a training set, a verification set and a test set corresponding to the detection external labels;

[0059] An artificial intelligence model is selected as a base model corresponding to the detection external labels;

[0060] The base model corresponding to the detection external labels is trained by the training set corresponding to the detection external labels, and the learning rate and other hyperparameters are adjusted on the verification set corresponding to the detection external labels to obtain a pre-trained model corresponding to the detection external labels;

[0061] The pre-trained model corresponding to the detection external labels is verified on the test set corresponding to the detection external labels, and finally a multi-modal defect identification model with the input of the detection external labels and the corresponding decision data and the output of the external labels and the defect results corresponding to the detection external labels is obtained;

[0062] The plurality of historical voiceprint data, historical internal labels and historical defect results corresponding to the plurality of detection internal labels are divided into training data, verification data and test data corresponding to the detection internal labels; and the training data, verification data and test data corresponding to the detection internal labels are preprocessed to obtain a training set, a verification set and a test set corresponding to the detection internal labels;

[0063] An artificial intelligence model is selected as a base model corresponding to the detection internal labels;

[0064] The base model corresponding to the detection inner label is trained by detecting the training set corresponding to the inner label, and the learning rate and other hyperparameters are adjusted on the validation set corresponding to the detection inner label to obtain the pre-training model corresponding to the detection inner label;

[0065] Through the verification of the pre-training model corresponding to the detection inner label on the test set corresponding to the detection inner label, the multi-modal defect recognition model with the input of the detection inner label and the corresponding voiceprint data and the output of the internal label corresponding to the detection inner label and the defect result is finally obtained.

[0066] Further, the generation of the alarm signal according to the detection result and the screening include:

[0067] Obtaining a detection result; the detection result includes an external label and an internal label and a corresponding defect result; the defect result includes a defect category and a defect degree level;

[0068] Screening the grains with the external defect label and the internal defect label in the external label and the internal label;

[0069] Real-time acquisition of the total screening quantity SZS and the sum QS of the defect quantities of the external defect label and the internal defect label;

[0070] The defect rate QL is calculated through the ratio between the sum QS of the defect quantities and the total screening quantity SZS;

[0071] When the defect rate is greater than the defect threshold, an alarm signal of the overall quality of the grains is generated;

[0072] Otherwise, no operation is performed.

[0073] Another aspect of the application provides a multi-dimensional grain screening method based on deep learning, comprising:

[0074] Obtaining environment data, grain data and device data; the environment data includes dust concentration, temperature, humidity and light intensity; the device data includes device ID and device parameters; the grain data includes grain parameters, detection images, point cloud data and voiceprint data;

[0075] Generating device adjustment parameter values according to the grain data and the environment data;

[0076] Assigning the device adjustment parameter values to the corresponding device parameters, and obtaining the adjusted detection images and point cloud data based on the device adjustment parameter values;

[0077] Generating a detection result according to the voiceprint data, the adjusted detection images and the point cloud data;

[0078] Generating an alarm signal according to the detection result and performing screening;

[0079] A prompt is made according to the alarm signal, and a manager is contacted.

[0080] Compared with the prior art, the application has the beneficial effects that:

[0081] 1、The application generates device adjustment parameter values according to grain data and environment data; assigns the device adjustment parameter values to corresponding device parameters, obtains adjusted detection images and point cloud data based on the device adjustment parameter values; generates a detection result according to voiceprint data, the adjusted detection images and the point cloud data; generates an alarm signal according to the detection result and performs screening, performs quality analysis on data of external defect detection, changes device parameters through environmental influence adaptation to maximize improve data quality, simultaneously processes external defect detection in steps, reduces resource consumption while improving screening speed; and dynamically adjusts data combination weights according to data quality during external defect detection, improves screening accuracy and screening efficiency.

[0082] 2、The application analyzes detection images and point cloud data required for external defect detection, constructs corresponding device optimization functions, maximizes improves data indicators of detection data under current environment data when the quality of the detection data is poor, thereby improving the quality of the detection data, and enables the detection data to have more effective features during subsequent external defect recognition, thereby providing solid data support for improving the accuracy of the grain screening system.

[0083] 3、The application considers that when grains are screened, whether internal defects or external defects, screening is performed as long as defects exist, and if internal and external screening is performed at the same time, especially for grains without defects, the calculation resources consumed by detection are huge, which greatly reduces the screening efficiency, therefore, the application processes screening in steps, uses specific data to detect specific defects, reduces the consumption of calculation resources during detection, and improves the screening efficiency of the grain screening system.

[0084] 4、The application obtains detection images and point cloud data adjusted by the device, monitors data quality in real time through a plurality of data quality scoring functions constructed by the application, and quantizes the quality scores within a certain range; dynamically adjusts the decision weights corresponding to the plurality of data quality scores according to the differences between the data quality scores, ensures that the decision weight corresponding to detection data with a high data quality score is higher, provides more effective features in defect recognition, and further increases the accuracy of screening. BRIEF DESCRIPTION OF DRAWINGS

[0085] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required by the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0086] Figure 1 The principle schematic diagram of the multi-dimensional grain screening system based on deep learning of the present application;

[0087] Figure 2 The generation flowchart of the device adjustment parameter value of the present application;

[0088] Figure 3 The generation flowchart of the detection result of the present application;

[0089] Figure 4 The multi-dimensional grain screening method flowchart based on deep learning of the present application. DETAILED DESCRIPTION

[0090] The technical solutions of the present application will be described in detail below in conjunction with the embodiments. Obviously, the described embodiments only constitute some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0091] Please refer to Figure 1 The first aspect embodiment of the present application provides a multi-dimensional grain screening system based on deep learning, which comprises a data acquisition module, a data analysis module, a warning module and a database; the data acquisition module and the data analysis module are electrically and / or communicatively connected; the data analysis module and the warning module are electrically and / or communicatively connected; the database is electrically and / or communicatively connected with the data acquisition module, the data analysis module and the warning module, respectively;

[0092] The data acquisition module: acquires environmental data, grain data and device data through a data acquisition device; the environmental data includes dust concentration, temperature, humidity and light intensity; the device data includes device ID and device parameters; the grain data includes grain parameters, detection images, point cloud data and voiceprint data; the detection image refers to the grain image acquired by the combination of a camera and a light source, the point cloud data refers to the point cloud data of the grain surface acquired by a laser radar, and the voiceprint data refers to the data when the grain is detected by voiceprint; the data acquisition device includes various sensors, etc.

[0093] The data analysis module generates a device adjustment parameter value according to the grain data and the environment data, the device adjustment parameter value being a value that optimally adjusts a device parameter in the current environment; the device adjustment parameter value is assigned to the corresponding device parameter, and adjusted detection image and point cloud data are obtained based on the device adjustment parameter value; a detection result is generated according to the voiceprint data, the adjusted detection image and the point cloud data, the detection result being a result under a grain screening standard, such as a defect type and a defect degree on a grain surface, a defect type and a defect degree inside the grain and the like; an alarm signal is generated according to the detection result, and screening is performed;

[0094] The early warning module makes a prompt according to the alarm signal, and contacts a management personnel; the alarm signal includes a grain overall quality poor alarm signal and the like.

[0095] The database is used to store data of each module and store historical data required for training a model.

[0096] For details, please refer to Figure 2 The grain parameters in the embodiment include a grain vibration intensity and a device adjustment parameter value generated according to the grain data and the environment data, including:

[0097] Grain data and environment data and device parameters are obtained; the grain vibration intensity is a value of intensity of vibration of the grain during detection;

[0098] An image signal-to-noise ratio and an image contrast of the detection image are extracted; the image signal-to-noise ratio is a ratio of useful information to noise in the image; the image contrast is a luminance difference between different regions in the image, and a higher contrast makes the image look clearer and the details more obvious;

[0099] Point cloud integrity and point cloud density of the point cloud data are extracted; the point cloud integrity reflects a degree of coverage of a point cloud on a surface of a target object, and in the embodiment, the point cloud integrity is calculated by a missing detection method based on point density statistics; the point cloud density is a distribution of a number of points in a unit volume, and in the embodiment, the point cloud density is calculated by a voxelization method;

[0100] It is determined whether the image signal-to-noise ratio and the image contrast are both greater than corresponding parameter thresholds; the parameter thresholds are set according to experience, and in the embodiment, the parameter threshold corresponding to the image signal-to-noise ratio is set to 35 dB, and the parameter threshold corresponding to the image contrast is set to 0.8;

[0101] No, no operation is performed;

[0102] Yes, an image device adjustment parameter value is generated according to the device parameters and the environment data;

[0103] whether the point cloud integrity and the point cloud density corresponding to the point cloud data are greater than the corresponding parameter threshold value; the parameter threshold value is set according to experience, and in the embodiment, the parameter threshold value corresponding to the point cloud integrity is set to 85%, and the parameter threshold value corresponding to the point cloud density is set to 100 points / cm2;

[0104] No, do nothing;

[0105] Yes, generate the point cloud device adjustment parameter value according to the device parameter, the grain vibration intensity and the environmental data;

[0106] The device adjustment parameter value includes the image device adjustment parameter value and the point cloud device adjustment parameter value.

[0107] In the embodiment, when carrying out external defect detection, the environmental data has a significant influence on the detection data, and when the quality of the detection data is poor, the system can dynamically adjust the device parameter according to the environmental factors in real time, so as to improve the data quality, and further provide solid and reliable data support for high-accuracy grain screening work.

[0108] In the embodiment, the image device adjustment parameter value is generated according to the device parameter and the environmental data, including:

[0109] Obtain the environmental data and the device parameter; the environmental data includes the dust concentration FN, the humidity SD and the light intensity GQ; the device parameter includes the device wavelength SB, the device light intensity SG and the device polarization angle SPJ;

[0110] Construct a device optimization function TSYF corresponding to the detection image; the formula of the device optimization function is:

[0111] TSNRF(SB, SG, SPJ|FN, SD, GQ) is an image signal-to-noise ratio calculation function of the detection image; and TDBF(SB, SG, SPJ|FN, SD, GQ) is an image contrast calculation function of the detection image.

[0112] The specific formula expression of the image signal-to-noise ratio calculation function of the detection image is:

[0113] ; ψ(SB) represents the wavelength-dependent quantum efficiency, which is set according to experience; N dark represents the sensor dark noise, which is a constant, and the specific value is set according to experience; k gq , k fn and k sd respectively represent the environmental light interference coefficient, the dust attenuation coefficient and the humidity attenuation coefficient, and k gq , k fn and k sd∈(0, 1), the specific value is set according to experience, and in the embodiment, k gq , k fn and k sd are respectively set as 0.003, 0.02 and 0.015;

[0114] The specific formula expression of the image contrast calculation function of the detection image is:

[0115] ; Wherein, FSL mb (SB, SPJ) and FSL bg (SB, SPJ) are respectively represented as the target reflectivity and the background reflectivity, and the specific values are set according to experience; SG sat is represented as the device light intensity saturation threshold, and the specific value is set according to experience, and in the embodiment, SG sat is set as 800W / m²; BFN is represented as the standard dust concentration, and the specific value is set according to experience, and in the embodiment, BFN is set as 120mg / m³;

[0116] The dust concentration FN, the humidity SD and the light intensity GQ are substituted into the device optimization function, and the device optimization function TSYF is solved through the CPSO algorithm to obtain the image device adjustment parameter value; the image device adjustment parameter value refers to the device parameter value that can make the image signal-to-noise ratio and the image contrast of the detection image reach the optimal value under the current environmental data; the image signal-to-noise ratio calculation function and the image contrast calculation function in the embodiment are both prediction functions, which are the predicted values obtained according to the values of several parameters.

[0117] In the embodiment, the point cloud device adjustment parameter value is generated according to the device parameters, the grain vibration intensity and the environmental data, which includes:

[0118] The device parameters, the grain vibration intensity GZQ and the environmental data are obtained; the environmental data includes the dust concentration FN and the light intensity GQ; the device parameters include the laser wavelength GB, the laser power JG, the pulse frequency MP, the scanning angle SJD and the scanning speed SV;

[0119] The device optimization function DSYF corresponding to the point cloud data is constructed; the formula of the device optimization function is:

[0120]

[0121] Wherein, DWXF(GB, JG, MP, SJD, SV|FN, GZQ, GQ) is the point cloud integrity calculation function of the point cloud data; DMF(GB, JG, MP, SJD, SV|FN, GZQ, GQ) is the point cloud density calculation function of the point cloud data;

[0122] The specific formula expression of the point cloud integrity calculation function of the point cloud data is:

[0123] ; h gq , h fn and h cl respectively represent the scattering attenuation coefficient of the light intensity on the laser, the scattering attenuation coefficient of the dust concentration on the laser and the material absorption coefficient, and h gq , h fn and h cl ∈(0, 1), the specific values are set according to experience, in the embodiment, h gq , h fn and h cl are respectively set to 0.005, 0.025 and 0.8; BSV represents the optimal scanning speed reference value, the specific value is set according to experience, in the embodiment, BSV is set to 2 m / s;

[0124] The specific formula expression of the point cloud density calculation function of the point cloud data is:

[0125] ; ψ(GB) represents the wavelength-dependent photoelectric conversion efficiency, the specific value is set according to experience; WZQ represents the stable vibration intensity, the specific value is set according to experience, in the embodiment, WZQ is set to 200 μm / s², which represents the sensor stable limit intensity value;

[0126] The grain vibration intensity GZQ, the dust concentration FN and the light intensity GQ are substituted into the device optimization function, and the device optimization function DSYF is solved by the NSGA-II algorithm to obtain the point cloud device adjustment parameter value; the point cloud device adjustment parameter value refers to the device parameter value that can make the point cloud integrity and the point cloud density of the point cloud data reach the optimal value under the current environmental data; the point cloud integrity calculation function and the point cloud density calculation function in the embodiment are both prediction functions, which are prediction values obtained according to the values of several parameters.

[0127] Referring to Figure 3 , the detection result is generated according to the voiceprint data, the adjusted detection image and the point cloud data in the embodiment, which includes:

[0128] The detection label, the voiceprint data, the adjusted detection image and the point cloud data are obtained; the detection label includes an internal detection label and an external detection label;

[0129] The internal detection label and the voiceprint data are input into the multi-modal defect recognition model to obtain the internal label corresponding to the internal detection label and the defect result; the internal label includes an internal defect label and a non-internal defect label; the multi-modal defect recognition model is constructed by an artificial intelligence model;

[0130] whether the internal label is a label with internal defects;

[0131] Yes, do nothing;

[0132] No, generate a detection result according to the point cloud data and the detection image.

[0133] In another embodiment, the detection result is generated according to the detection image, the point cloud data and the voiceprint data, comprising:

[0134] obtaining detection label, point cloud data and voiceprint data and detection image; the detection label comprises detection internal label and detection external label;

[0135] generating an external label corresponding to the detection external label and a defect result according to the detection external label, the point cloud data and the detection image;

[0136] whether the external label is a label with external defects;

[0137] Yes, do nothing;

[0138] No, input the detection internal label and the voiceprint data into a multi-modal defect recognition model to obtain an internal label corresponding to the detection internal label and a defect result; the multi-modal defect recognition model is constructed by an artificial intelligence model.

[0139] The embodiment fully considers that in the grain screening process, whether it is internal defects or external defects, as long as there are defects, screening is required. However, if internal and external defect detection is carried out at the same time, especially for grains without defects, a large amount of computing resources will be consumed, thereby significantly reducing the screening efficiency. To solve this problem, the application adopts a step-by-step screening process, that is, specific data is used to detect specific defects, which effectively reduces the consumption of computing resources in the detection process and significantly improves the overall screening efficiency of the grain screening system.

[0140] In the embodiment, the detection result is generated according to the point cloud data and the detection image, comprising:

[0141] obtaining point cloud data and detection image and detection external label;

[0142] extracting point cloud integrity and point cloud density corresponding to the point cloud data;

[0143] extracting image signal-to-noise ratio and image contrast corresponding to the detection image;

[0144] generating a data quality score according to the point cloud integrity, the point cloud density, the image signal-to-noise ratio and the image contrast, and generating a decision weight according to the data quality score;

[0145] combining the decision weight with the corresponding point cloud data and detection image to obtain decision data;

[0146] The detection outer label and the decision data are input into the multi-modal defect identification model to obtain an external label corresponding to the detection outer label and a defect result; the external label includes an external defect label and a non-external defect label; the multi-modal defect identification model is constructed through an artificial intelligence model.

[0147] In the embodiment, the data quality score is generated according to the point cloud integrity, the point cloud density, the image signal-to-noise ratio and the image contrast, and the decision weight is generated according to the data quality score, which includes:

[0148] The point cloud integrity DWX, the point cloud density DM, the image signal-to-noise ratio TSNR and the image contrast TDB are extracted.

[0149] The point cloud data quality score DZP is calculated through the formula ; wherein, min() represents a minimum value operation, and the min() is set to make DZP ∈ [0, 1], and DZF(DWX, DM) represents a point cloud data quality score function constructed according to the nonlinear relationship between the point cloud integrity DWX and the point cloud density DM and the point cloud data quality score; ; wherein, DM max represents an upper limit value of the point cloud density, and the specific value is set according to experience, and in the embodiment, DM max is set to 200 points / cm2; γ represents an exponential coefficient, γ ∈ (0, 1), and the specific value is set according to experience, and in the embodiment, γ is set to 0.7; the higher the point cloud density and the point cloud integrity corresponding to the point cloud data, the better the quality of the point cloud data, and more accurate effective features can be provided in defect identification;

[0150] The image data quality score TZP is calculated through the formula ; wherein, min() represents a minimum value operation, and the min() is set to make TZP ∈ [0, 1], and TZF(TSNR, TDB) represents an image data quality score function constructed according to the nonlinear relationship between the image signal-to-noise ratio TSNR and the image contrast TDB and the image data quality score;

[0151] The specific formula expression of the image data quality score function is:

[0152] ; ; wherein, clip(x, a, b) represents a limit function, which is used to limit x in the range of a and b, and a < b, such as a = 1 and b = 3, if x is 0, the output result is 1; similarly, if x is 4, the output result is 3; TSNR minThe minimum value of the image signal-to-noise ratio is represented as TSNR qd The minimum threshold value of the image signal-to-noise ratio reaching the quality requirement is represented as TDB qd The minimum threshold value of the image contrast reaching the quality requirement is represented as TDB, and the specific value is set according to experience. In this embodiment, TSNR min , TSNR qd , and TDB qd are respectively set as 25 dB, 35 dB, and 0.8.

[0153] A detection image weight function TJQF(DZP, TZP) is constructed according to a nonlinear relationship between the point cloud data quality score DZP and the image data quality score TZP and the decision weight corresponding to the detection image.

[0154] The specific formula expression of the detection image weight function is: ; wherein d is a competition factor, d>0, the setting of d is to amplify the effect of quality difference, and the specific value is set according to experience. In this embodiment, d is set as 3; the more the difference between the two data qualities, the more the high-quality modality weight tends to 1.

[0155] The point cloud data quality score DZP and the image data quality score TZP are substituted into the detection image weight function to obtain the decision weight TJQ corresponding to the detection image.

[0156] The decision weight DJQ corresponding to the point cloud data is obtained by difference calculation of the sum of weights and the decision weight TJQ corresponding to the detection image. The sum of weights is set according to experience. In this embodiment, the sum of weights is set as 1.

[0157] In another embodiment, the decision weight can be obtained through a weight generation model, including:

[0158] The point cloud integrity, the point cloud density, the image signal-to-noise ratio, and the image contrast are obtained.

[0159] The point cloud integrity, the point cloud density, the image signal-to-noise ratio, and the image contrast are integrated into weight analysis data.

[0160] The weight analysis data is input into the weight generation model to obtain the decision weight.

[0161] The weight generation model is constructed through a machine learning model, including:

[0162] A plurality of historical weight analysis data and historical decision weights are obtained.

[0163] The historical weight analysis data and historical decision weight are divided into training data, verification data and test data, and the training data, verification data and test data are preprocessed to obtain a training set, a verification set and a test set; the ratio between the training set, the test set and the verification set is 7:2:1;

[0164] The machine learning model is selected as the base model; the machine learning model includes a BP model and the like;

[0165] The base model is trained through each training set, and the learning rate and other hyperparameters are adjusted on the verification set to obtain a pre-trained model;

[0166] The pre-trained model is verified on the test set, and finally a weight generation model that inputs weight analysis data and outputs decision weight is obtained.

[0167] The multi-modal defect recognition model in the embodiment is constructed through an artificial intelligence model, including:

[0168] A plurality of historical decision data corresponding to a plurality of detection external labels, historical external labels and historical defect results, and a plurality of historical voiceprint data corresponding to a plurality of detection internal labels, historical internal labels and historical defect results are obtained;

[0169] The plurality of historical decision data corresponding to the plurality of detection external labels, the historical external labels and the historical defect results are divided into training data, verification data and test data corresponding to the detection external labels; and the training data, the verification data and the test data corresponding to the detection external labels are preprocessed to obtain a training set, a verification set and a test set corresponding to the detection external labels; the ratio between the training set, the test set and the verification set is 7:2:1;

[0170] An artificial intelligence model is selected as a base model corresponding to the detection external label; the artificial intelligence model selects a convolutional neural network model and the like;

[0171] The base model corresponding to the detection external label is trained through the training set corresponding to the detection external label, and the learning rate and other hyperparameters are adjusted on the verification set corresponding to the detection external label to obtain a pre-trained model corresponding to the detection external label;

[0172] The pre-trained model corresponding to the detection external label is verified on the test set corresponding to the detection external label, and finally a multi-modal defect recognition model that inputs the detection external label and its corresponding decision data and outputs the external label and the defect result corresponding to the detection external label is obtained;

[0173] The historical voiceprint data corresponding to the plurality of detection inner labels, the historical inner labels and the historical defect results are divided into training data, verification data and test data corresponding to the detection inner labels; and the training data, the verification data and the test data corresponding to the detection inner labels are preprocessed to obtain a training set, a verification set and a test set corresponding to the detection inner labels; the ratio between the training set, the test set and the verification set is 7:2:1;

[0174] An artificial intelligence model is selected as a base model corresponding to the detection inner label; the artificial intelligence model is a convolutional neural network model or the like;

[0175] The base model corresponding to the detection inner label is trained through the training set corresponding to the detection inner label, and the learning rate and other hyperparameters are adjusted on the verification set corresponding to the detection inner label to obtain a pre-trained model corresponding to the detection inner label;

[0176] The pre-trained model corresponding to the detection inner label is verified on the test set corresponding to the detection inner label, and finally a multi-modal defect recognition model is obtained, which inputs the detection inner label and the corresponding voiceprint data and outputs the inner label and the defect result corresponding to the detection inner label.

[0177] In the embodiment, the corresponding multi-dimensional defect recognition model is trained in the corresponding historical data, and the corresponding model branch is selected for discrimination through the detection label during detection, so that the speed and accuracy of grain screening are greatly improved, the labor cost during screening is reduced, and the efficiency of grain screening is improved.

[0178] In the embodiment, the alarm signal is generated according to the detection result and screening is performed, which includes:

[0179] Obtaining a detection result; the detection result includes an outer label and an inner label and a corresponding defect result; the defect result includes a defect category and a defect degree level;

[0180] Grains with outer defect labels and inner defect labels in the outer label and the inner label are screened;

[0181] The total screening quantity SZS and the sum QS of the defect quantities of the grains with the outer defect labels and the inner defect labels are obtained in real time;

[0182] The defect rate QL is calculated through the ratio between the sum QS of the defect quantities and the total screening quantity SZS;

[0183] When the defect rate is greater than a defect threshold, the defect threshold is set according to experience, and in the embodiment, the defect threshold is set to 0.1, and a grain overall quality poor alarm signal is generated;

[0184] Otherwise, no operation is performed.

[0185] The embodiment improves the efficiency of automatic screening by transferring the grains to the corresponding positions according to the detection results, and simultaneously monitors the grains in the screening batch in real time, timely issues an alarm when the defect rate is too high, and improves the relevant management personnel, which can provide data support for subsequent grain source selection strategy.

[0186] Please refer to Figure 4 Another aspect of the embodiment of the present application provides a multi-dimensional grain screening method based on deep learning, comprising:

[0187] Obtain environment data, grain data and equipment data; the environment data includes dust concentration, temperature, humidity and light intensity; the equipment data includes equipment ID and equipment parameters; the grain data includes grain parameters, detection images, point cloud data and voiceprint data;

[0188] Generate equipment adjustment parameter values according to the grain data and the environment data;

[0189] Assign the equipment adjustment parameter values to the corresponding equipment parameters, and obtain the adjusted detection images and point cloud data based on the equipment adjustment parameter values;

[0190] Generate detection results according to the voiceprint data, the adjusted detection images and the point cloud data;

[0191] Generate an alarm signal and perform screening according to the detection results;

[0192] Make a prompt according to the alarm signal, and contact the management personnel.

[0193] Some data in the above formula are calculated by removing the dimension and taking the numerical value, and the formula is obtained by software simulation of a large amount of collected data to obtain a formula closest to the real situation; the preset parameters and the preset threshold in the formula are set by the person skilled in the art according to the actual situation or obtained by a large amount of data simulation.

[0194] The working principle of the present application is: acquiring environment data, grain data and equipment data; generating equipment adjustment parameter values according to the grain data and the environment data; assigning the equipment adjustment parameter values to corresponding equipment parameters, obtaining adjusted detection images and point cloud data based on the equipment adjustment parameter values; generating a detection result according to the voiceprint data, the adjusted detection images and the point cloud data; generating an alarm signal according to the detection result and performing screening; making a prompt according to the alarm signal and contacting a management personnel, performing quality analysis on the data of external defect detection, changing the equipment parameters through environmental influence adaptation to maximize the improvement of data quality, and processing the internal and external defect detection of the grain in steps, which can improve the screening speed while reducing resource consumption; and dynamically adjusting the data combination weight according to the data quality during the external defect detection, which improves the screening accuracy and efficiency, avoids the problem that the prior art lacks consideration of the influence of the screening environment on the visual detection parameters, reduces the quality of the visual detection features, and causes the accuracy and efficiency of the grain screening system to be low.

[0195] The above embodiments are only used to illustrate the technical method of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A multi-dimensional grain screening system based on deep learning, characterized in that, The application relates to a grain screening system and a method thereof. The system comprises a data collection module and a data analysis module connected with each other. The data collection module acquires environment data, grain data and equipment data; the environment data comprises dust concentration, temperature, humidity and illumination intensity; the equipment data comprises equipment ID and equipment parameters; the grain data comprises grain parameters, detection images, point cloud data and voiceprint data. The data analysis module generates equipment adjustment parameter values according to the grain data and the environment data, wherein the equipment adjustment parameter values are optimal values of the equipment parameters under the current environment; the equipment adjustment parameter values are assigned to the corresponding equipment parameters, and adjusted detection images and point cloud data are acquired based on the equipment adjustment parameter values. Detection results are generated according to the voiceprint data, the adjusted detection images and the point cloud data, wherein the detection results refer to results under a grain screening standard, and comprise defect types and defect degrees on a grain surface and defect types and defect degrees inside the grain. Alarm signals are generated according to the detection results and screening is performed. The grain parameters comprise grain vibration intensity, and the generation of the equipment adjustment parameter values according to the grain data and the environment data comprises the following steps. Grain data, environment data and equipment parameters are acquired. Image signal-to-noise ratio and image contrast of the detection images are extracted. Point cloud integrity and point cloud density of the point cloud data are extracted. It is judged whether the image signal-to-noise ratio and the image contrast are greater than corresponding parameter thresholds. Yes, no operation is performed. No, image equipment adjustment parameter values are generated according to the equipment parameters and the environment data. It is judged whether the point cloud integrity and the point cloud density corresponding to the point cloud data are greater than corresponding parameter thresholds. Yes, no operation is performed. No, point cloud equipment adjustment parameter values are generated according to the equipment parameters, the grain vibration intensity and the environment data. The equipment adjustment parameter values comprise the image equipment adjustment parameter values and the point cloud equipment adjustment parameter values.

2. The deep learning based multi-dimension grain sorting system of claim 1, wherein, The generation of the detection results according to the voiceprint data, the adjusted detection images and the point cloud data comprises the following steps. Detection labels, voiceprint data, adjusted detection images and point cloud data are acquired; the detection labels comprise internal detection labels and external detection labels. The internal detection labels and the voiceprint data are input into a multi-modal defect recognition model to obtain internal labels and defect results corresponding to the internal detection labels; the internal labels comprise internal defect labels and non-internal defect labels; the multi-modal defect recognition model is constructed by an artificial intelligence model. It is judged whether the internal labels are internal defect labels. Yes, no operation is performed. No, detection results are generated according to the point cloud data and the detection images.

3. The deep learning based multi-dimension grain sorting system of claim 2, wherein, The generation of the detection results according to the point cloud data and the detection images comprises the following steps. Point cloud data, detection images and external detection labels are acquired. Point cloud integrity and point cloud density corresponding to the point cloud data are extracted. Image signal-to-noise ratio and image contrast corresponding to the detection images are extracted. Data quality scores are generated according to the point cloud integrity, the point cloud density, the image signal-to-noise ratio and the image contrast, and decision weights are generated according to the data quality scores. Decision data are obtained by combining the decision weights with corresponding point cloud data and detection images. The detection external label and the decision data are input into the multi-modal defect identification model to obtain an external label corresponding to the detection external label and a defect result; the external label includes an external defect label and a non-external defect label; and the multi-modal defect identification model is constructed by an artificial intelligence model.

4. The deep learning based multi-dimension grain screening system according to claim 2 or claim 3, wherein, The multi-modal defect identification model is constructed by an artificial intelligence model, including: acquiring a plurality of historical decision data, historical external labels and historical defect results corresponding to a plurality of detection external labels, and a plurality of historical voiceprint data, historical internal labels and historical defect results corresponding to a plurality of detection internal labels; the plurality of historical decision data, historical external labels and historical defect results corresponding to the plurality of detection external labels are divided into training data, verification data and test data corresponding to the detection external labels; and the training data, verification data and test data corresponding to the detection external labels are preprocessed to obtain a training set, a verification set and a test set corresponding to the detection external labels; an artificial intelligence model is selected as a base model corresponding to the detection external labels; the base model corresponding to the detection external labels is trained by the training set corresponding to the detection external labels, and the learning rate and other hyperparameters are adjusted on the verification set corresponding to the detection external labels to obtain a pre-trained model corresponding to the detection external labels; the pre-trained model corresponding to the detection external labels is verified on the test set corresponding to the detection external labels, and finally a multi-modal defect identification model is obtained, in which the input is the detection external label and the corresponding decision data, and the output is the external label corresponding to the detection external label and the defect result; the plurality of historical voiceprint data, historical internal labels and historical defect results corresponding to the plurality of detection internal labels are divided into training data, verification data and test data corresponding to the detection internal labels; and the training data, verification data and test data corresponding to the detection internal labels are preprocessed to obtain a training set, a verification set and a test set corresponding to the detection internal labels; an artificial intelligence model is selected as a base model corresponding to the detection internal labels; the base model corresponding to the detection internal labels is trained by the training set corresponding to the detection internal labels, and the learning rate and other hyperparameters are adjusted on the verification set corresponding to the detection internal labels to obtain a pre-trained model corresponding to the detection internal labels; the pre-trained model corresponding to the detection internal labels is verified on the test set corresponding to the detection internal labels, and finally a multi-modal defect identification model is obtained, in which the input is the detection internal label and the corresponding voiceprint data, and the output is the internal label corresponding to the detection internal label and the defect result.

5. The deep learning based multi-dimension grain screening system of claim 1, wherein, The detection result is obtained; the detection result includes the external label and the internal label and the corresponding defect result; the defect result includes the defect category and the defect degree level; the internal label includes the internal defect label and the non-internal defect label; and the external label includes the external defect label and the non-external defect label; the grains with the external defect label and the internal defect label in the external label and the internal label are screened; the total number of screening SZS and the sum QS of the defect quantities of the external defect label and the internal defect label are obtained in real time; ​ The defect rate QL is calculated by the ratio between the sum of the number of defects QS and the total number of screening SZS; When the defect rate is greater than the defect threshold, a poor overall quality of grain warning signal is generated; Otherwise, no operation is performed.

6. The multi-dimensional grain screening method based on deep learning, applied to the multi-dimensional grain screening system based on deep learning of any one of claims 1-5, characterized in that, The method comprises: Obtaining environment data, grain data and device data; the environment data comprises dust concentration, temperature, humidity and light intensity; the device data comprises device ID and device parameters; the grain data comprises grain parameters, detection images, point cloud data and voiceprint data; Generating device adjustment parameter values according to the grain data and the environment data; Assigning the device adjustment parameter values to the corresponding device parameters, and obtaining adjusted detection images and point cloud data based on the device adjustment parameter values; Generating a detection result according to the voiceprint data, the adjusted detection images and the point cloud data; Generating an alarm signal according to the detection result and performing screening.

Images