Agricultural product quality detection method and system based on big data

Abstract

The present application relates to the technical field of spectral detection, in particular to a method and system for agricultural product quality detection based on big data, which obtains a sequence of spectral image frames of a plurality of continuous detection objects on a conveyor belt, performs detection object contour segmentation on the spectral image frames and divides them into a plurality of detection area bands, calculates the standard deviation of the mean value of the spectral reflectance intensity of each area band and the maximum value ratio as an edge distortion value, determines a first frame distortion coordinate and a second framing coordinate through the edge distortion value of the distorted object obtained from the edge distortion value, calculates the corresponding time difference and obtains the optimal value through neural network training. The present application solves the problem of repeated detection and waste of processing resources caused by edge distortion of the hyperspectral camera in continuous detection, and improves the online detection efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of spectral detection technology, in particular to a method and system for detecting the quality of agricultural products based on big data. BACKGROUND

[0002] Non-contact detection of agricultural products does not cause damage to the agricultural products, and is therefore widely used. Machine vision detection is widely used, which trains and identifies a classifier according to the types of surface defects of agricultural products, so as to identify various defects. Since the image detection of machine vision is based on visible light, it is sufficient to detect only surface defects, but it is difficult to cope with some non-obvious surface damage of fruits, sugar concentration identification, and surface pesticide residues that cannot be detected by the naked eye. The hyperspectral image collected by a hyperspectral camera is more detailed in the spectral dimension, and there are multiple channels in the spectral dimension. The data cube obtained by the hyperspectral device contains not only image information, but also spectral data and image information of any spectral segment of each point on the image in the spectral dimension, which is sufficient to capture the subtle features of fruits.

[0003] An online recognition system for agricultural products, such as a fruit continuous sorting and recognition line, fruits are continuously conveyed by a conveyor belt at a uniform speed through a hyperspectral camera detection box. The hyperspectral camera needs to take a spectral photo of each detection object, i.e. fruit, and analyze it in real time. The traditional way is to take a spectral photo for each detection object. When a single spectral photo is continuously taken, multiple detection objects will be included in the field of view. Compared with the traditional RGB image which has only 3 channels, the hyperspectral image has hundreds of channels, and the tens of times increase in data processing volume will affect the response speed of online processing. Specifically, multiple detection objects in the field of view, the detection object directly below the hyperspectral camera can obtain the best spectral photo. For a push-broom spectral camera, the detection object at the edge will have different degrees of image distortion at the maximum scanning angle, resulting in inaccurate detection. The detection object that can directly obtain the detection result in the field of view does not need to be detected again, and only the object at the edge position that is not accurate needs to be detected again to ensure that it can be detected clearly when taking a photo next time. Therefore, as the conveyor belt runs, the detection object at the edge slowly approaches the hyperspectral camera and then moves away from the hyperspectral camera. Therefore, it is necessary to determine the timing of taking a photo of the detection object again when the detection object is out of the field of view of the hyperspectral camera. SUMMARY

[0004] (1) Technical problem to be solved

[0005] The purpose of the present application is to provide a method and system for detecting the quality of agricultural products based on big data, to solve the problem of large spectral data processing volume, and to identify more agricultural product detection objects under the same spectral processing volume.

[0006] (2) Technical solutions

[0007] To achieve the above object, in one aspect, the application provides a big data-based agricultural product quality detection method, which comprises the following steps:

[0008] Obtaining a sequence of spectral image frames of a plurality of detection objects on a conveying belt, the sequence of spectral image frames being continuously obtained by a hyperspectral camera at a fixed frame rate, and each spectral image frame being marked with a time stamp and object position coordinates on the conveying belt;

[0009] Segmenting and identifying the contour of the detection object, finding the edge distortion value of the current spectral image frame, and marking the detection object with an edge distortion value greater than a preset distortion threshold as a distortion object; calculating the longest time for two spectral frame acquisitions before and after the distortion object directly below the hyperspectral camera as a time difference;

[0010] In the case where the conveying belt speed remains unchanged, the time difference sequence is obtained by the time stamp of the first spectral image frame of the distortion object collected at different conveying belt positions and the time stamp of the corresponding second frame, and the first frame distortion coordinates of the distortion object form a detection coordinate sequence; the mapping relationship between the detection coordinate sequence and the time difference sequence is established to obtain a spectral frame acquisition network;

[0011] The time period when the detection object enters the acquisition field of view is divided into a plurality of acquisition time points by a preset time interval, a plurality of acquisition time points are randomly selected and used as a plurality of first frame acquisition time points, the first frame acquisition time point is used to obtain the second frame acquisition time point through the spectral frame acquisition network; the time difference between the first frame acquisition time point and the second frame acquisition time point is calculated, and the first frame acquisition time point and the second frame acquisition time point corresponding to the maximum time difference are used as the spectral acquisition scheme of the hyperspectral camera for the detection object.

[0012] Further, the method for segmenting and identifying the contour of the detection object, finding the edge distortion value of the current spectral image frame, and marking the detection object with an edge distortion value greater than a preset distortion threshold as a distortion object; calculating the longest time for two spectral frame acquisitions before and after the distortion object directly below the hyperspectral camera as a time difference comprises the following steps:

[0013] Segmenting and identifying the contour of the detection object in the spectral image frame, dividing the contour into a plurality of detection area bands perpendicular to the moving direction of the conveying belt, and sequentially calculating the spectral reflectance intensity average of the pixel points in each detection area band in a preset spectral band range, taking the ratio of the standard deviation of the spectral reflectance intensity average of the detection area band to the maximum value in the spectral reflectance intensity average sequence as the edge distortion value of the current spectral image frame;

[0014] The first frame distortion coordinate and the second frame distortion coordinate are obtained by the edge distortion values of the distortion object before and after the high-spectrum camera; and the time difference is obtained by the time stamps of the first spectral image frame and the second spectral image frame.

[0015] Further, the method of obtaining the first frame distortion coordinate and the second frame distortion coordinate by the edge distortion values of the distortion object before and after the high-spectrum camera, and obtaining the time difference by the time stamps of the first spectral image frame and the second spectral image frame comprises:

[0016] When the edge distortion value obtained from the detection area band of the first spectral image frame obtained before the detection object reaches the high-spectrum camera is greater than the preset distortion threshold, the detection object is marked as a distortion object, the coordinate of the detection area band is marked as the first frame distortion coordinate, and the areas before and after the first frame distortion coordinate of the first spectral image frame are marked as the first distortion area and the first identification area, respectively; when the edge distortion value is less than or equal to the preset distortion threshold, the first frame distortion coordinate is set as the coordinate corresponding to the boundary of the first spectral image frame.

[0017] When the second spectral image frame obtained after the distortion object reaches the high-spectrum camera, the edge distortion value of each frame of the second spectral image frame is calculated in reverse order according to the second spectral image frame, and the corresponding second frame distortion coordinate is marked, the areas before and after the second frame distortion coordinate of the second spectral image frame are marked as the second distortion area and the second identification area, respectively, until the first distortion area is calculated in the second identification area for the first time, and the corresponding second frame distortion coordinate is marked as the second view coordinate, and the second spectral image frame corresponding to the second view coordinate is marked as the second view frame; and the time difference is obtained by the time stamps of the first spectral image frame and the second view frame.

[0018] Further, the method of taking the ratio of the standard deviation of the spectral reflectance intensity mean value of the detection area band to the maximum value in the sequence of the spectral reflectance intensity mean value as the edge distortion value of the current spectral image frame comprises:

[0019] The kth detection area band in the current spectral image frame is obtained, and the spectral reflectance intensity value I kpq of the pth pixel point in the qth spectral band in the detection area band is extracted in the preset spectral band range, wherein k is the detection area band number, p is the pixel point number, and q is the spectral band number; the spectral reflectance intensity mean value R k of the kth detection area band in the preset spectral band range is calculated.

[0020]

[0021] Wherein P is the total number of pixel points contained in the kth detection area band, and Q is the total number of spectral bands in the preset spectral band range;

[0022] The average spectral reflectance intensity R of all detection area bands in the current spectral image frame is calculated k The average spectral reflectance intensity sequence R1, R2,..., R is arranged in ascending order according to the detection area band number k K , wherein K is the total number of detection area bands in the current spectral image frame; and the arithmetic mean of the average spectral reflectance intensity sequence is calculated

[0023]

[0024] The standard deviation S of the average spectral reflectance intensity sequence is calculated

[0025]

[0026] The maximum value of the average spectral reflectance intensity sequence is found, and the ratio of the standard deviation S to the maximum average spectral reflectance intensity M is taken as the edge distortion value of the current spectral image frame

[0027] Further, the method for establishing a mapping relationship between the detection coordinate sequence and the time difference sequence to obtain the spectral frame acquisition network comprises:

[0028] The first frame distortion coordinate of the ith distortion object in the detection coordinate sequence is taken as the neural network input vector X i , and the corresponding time difference is taken as the neural network target output vector Y i , wherein i is the distortion object number; a multi-layer feedforward neural network is constructed, the number of input layer nodes is set to A, the number of hidden layer nodes is set to B, the number of output layer nodes is set to C, and the total number of network layers is set to L;

[0029] The weight parameters W and the bias parameters V between the layers of the neural network are initialized j j , wherein j is the parameter index; the weighted input Z of the hth neuron in the hidden layer is calculated h :

[0030]

[0031] , wherein N g is the output value of the gth neuron in the input layer, W g is the corresponding connection weight, and V h is the bias of the hth neuron in the hidden layer;

[0032] The weighted input Z h ​The output N of the hth neuron of the hidden layer is obtained by the hyperbolic tangent activation function h :

[0033]

[0034] The output value of the output layer neuron is calculated using the same method, and the final prediction time difference of the network is obtained by forward propagation; the error between the predicted time difference and the actual time difference Y i is calculated using the mean square error loss function, and the weight parameters W j and the bias parameters V j are updated by the back propagation algorithm; the training process is repeated until the network converges, and the trained neural network is used as the mapping relationship of the coordinate sequence of the detection object to the time difference sequence, which is denoted as the spectral frame acquisition network.

[0035] Based on the same inventive concept, in another aspect, the present application also provides a big data-based agricultural product quality detection system, which comprises:

[0036] a spectral acquisition module for acquiring a spectral image frame sequence of a plurality of detection objects on a conveyor belt, the spectral image frame sequence being continuously obtained by a hyperspectral camera at a fixed frame rate, and each spectral image frame being marked with a timestamp and an object position coordinate on the conveyor belt;

[0037] a time difference acquisition module for segmenting and identifying the contour of the detection object, finding the edge distortion value of the current spectral image frame, and marking the detection object with an edge distortion value greater than a preset distortion threshold as a distorted object; the longest time difference between the acquisition of two spectral frames is calculated as the time difference by the edge distortion values of the distorted object before and after the hyperspectral camera directly below;

[0038] a spectral frame acquisition training module for, under the condition that the speed of the conveyor belt remains unchanged, obtaining a time difference sequence by the timestamp of the first spectral image frame and the timestamp of the corresponding second frame of the distorted object collected at different positions of the conveyor belt, and obtaining a detection coordinate sequence by the first frame of the distorted coordinate of the distorted object; a mapping relationship between the detection coordinate sequence and the time difference sequence is established to obtain a spectral frame acquisition network;

[0039] an acquisition optimization module for dividing the time period when the detection object enters the acquisition field of view into a plurality of acquisition time points by a preset time interval, randomly selecting a plurality of acquisition time points as a plurality of first frame acquisition time points, and obtaining the second frame acquisition time point by the first frame acquisition time point through the spectral frame acquisition network; calculating the time difference between the first frame acquisition time point and the second frame acquisition time point and taking the maximum value of the time difference as the first frame acquisition time point and the second frame acquisition time point corresponding to the spectral acquisition scheme of the hyperspectral camera for the detection object.

[0040] Further, the system further comprises:

[0041] a distortion recognition module, configured to segment and recognize a contour of a detection object in a spectral image frame, divide the contour into a plurality of detection area bands in a direction perpendicular to a moving direction of the conveying belt, and calculate a spectral reflectance intensity mean value of a pixel in each detection area band in a preset spectral band range in a forward order according to the detection area bands and a distance of the hyperspectral camera, and take a ratio of a standard deviation of the spectral reflectance intensity mean value of the detection area band to a maximum value in a sequence of the spectral reflectance intensity mean value as an edge distortion value of a current spectral image frame;

[0042] a first-frame distortion coordinate and a second-frame distortion coordinate of the first-frame distortion coordinate are obtained through the edge distortion values of the distortion object before and after the distortion object is directly below the hyperspectral camera; and a time difference is obtained through time stamps corresponding to the first spectral image frame and the second view frame.

[0043] Further, the system further comprises:

[0044] a time difference calculation module, configured to, when the edge distortion value obtained from the detection area band of the first spectral image frame obtained before the detection object reaches directly below the hyperspectral camera is greater than a preset distortion threshold, mark the detection object as a distortion object, mark a coordinate of the detection area band as a first-frame distortion coordinate, and mark a region before and after the first-frame distortion coordinate of the first spectral image frame as a first distortion region and a first recognition region, respectively; and when the edge distortion value is less than or equal to the preset distortion threshold, set the first-frame distortion coordinate as a coordinate corresponding to a boundary of the first spectral image frame;

[0045] when a second spectral image frame is obtained after the distortion object reaches directly below the hyperspectral camera, calculate an edge distortion value of each second spectral image frame in a reverse order of the second spectral image frames and mark a corresponding second-frame distortion coordinate, mark a region before and after the second-frame distortion coordinate of the second spectral image frame as a second distortion region and a second recognition region, respectively, until the first distortion region is in the second recognition region for the first time, mark the corresponding second-frame distortion coordinate as a second view coordinate, and mark the second spectral image frame corresponding to the second view coordinate as a second view frame; and obtain a time difference through time stamps corresponding to the first spectral image frame and the second view frame.

[0046] Further, the system further comprises:

[0047] an edge distortion value calculation module, configured to obtain a kth detection area band in a current spectral image frame, extract a spectral reflectance intensity value I kpq of a pth pixel in the detection area band in a qth spectral band in a preset spectral band range, wherein k is a detection area band number, p is a pixel number, and q is a spectral band number; and calculate a spectral reflectance intensity mean value R k of the kth detection area band in the preset spectral band range.

[0048]

[0049] wherein P is the total number of pixels contained in the kth detection zone band, and Q is the total number of spectral bands in the preset spectral band range;

[0050] The average spectral reflectance intensity R of all detection zone bands in the current spectral image frame is calculated as follows: k The average spectral reflectance intensity sequence R1, R2,..., R is arranged in ascending order of the detection zone band number k, and the arithmetic mean value of the average spectral reflectance intensity sequence is calculated as follows: K wherein K is the total number of detection zone bands in the current spectral image frame; and the arithmetic mean value of the average spectral reflectance intensity sequence is calculated as follows:

[0051]

[0052] The standard deviation S of the average spectral reflectance intensity sequence is calculated as follows:

[0053]

[0054] The maximum value M of the average spectral reflectance intensity sequence is found, and the ratio of the standard deviation S to the maximum value M of the average spectral reflectance intensity is taken as the edge distortion value of the current spectral image frame.

[0055] Further, the system further comprises:

[0056] The neural network training module is configured to take the first frame of the distortion coordinates of the ith distortion object in the distortion coordinate sequence as a neural network input vector X i , and take the corresponding time difference as a neural network target output vector Y i , wherein i is the distortion object number; a multi-layer feedforward neural network is constructed, the number of input layer nodes is set as A, the number of hidden layer nodes is set as B, the number of output layer nodes is set as C, and the total number of network layers is set as L;

[0057] The weight parameters W j and the bias parameters V j between the layers of the neural network are initialized, wherein j is the parameter index; the weighted input Z h of the hth neuron in the hidden layer is calculated as follows:

[0058]

[0059] wherein N g is the output value of the gth neuron in the input layer, W g is the corresponding connection weight, and V h is the bias of the hth neuron in the hidden layer;

[0060] The weighted input Z h The output N of the hth neuron in the hidden layer is obtained by the hyperbolic tangent activation function h :

[0061]

[0062] The output value of the output layer neuron is calculated using the same method, and the final prediction time difference of the network is obtained by forward propagation; the error between the predicted time difference and the actual time difference Y i is calculated using the mean square error loss function, and the weight parameter W j and the bias parameter V j are updated by the back propagation algorithm; the training process is repeated until the network converges, and the trained neural network is used as the mapping relationship of the detection coordinate sequence to the time difference sequence, denoted as the spectral frame acquisition network.

[0063] (3) Advantages

[0064] Compared with the prior art, the beneficial effects of the present application are that in the process of detecting agricultural product objects by online conveyor belts, more agricultural product detection objects are identified and detected under the same amount of spectral data processing, and the overall spectral data processing cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 The flowchart of the method for detecting the quality of agricultural products based on big data in embodiment 1 of the present application is shown.

[0066] Figure 2 The module block diagram of the system for detecting the quality of agricultural products based on big data in embodiment 2 of the present application is shown.

[0067] Figure 3 The schematic diagram of the inventive concept of the present application is shown. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0069] Before the example, the application scenario of the present application concept needs to be described, in a certain fruit (apple, citrus and winter jujube) spectrum recognition sorting line, a hyperspectral camera is adopted, push-scan imaging, the minimum resolution is 2.5 nm, there are more than 500 spectral bands, and 168 spectral bands are actually adopted as channels. The near-infrared band (400-1000 nm) is adopted for analyzing the sugar content and moisture content of apples, citrus and winter jujube, and the short-wave infrared band (900-1700 nm) is adopted for recognizing hidden defects such as internal rot and water heart disease. The camera exposure time is 14 ms, the conveying belt moving speed is 0.6 m / s, and the distance between the sample platform and the lens is 40 cm. The hyperspectral imaging system is preheated before use, and the preheating time is set to 30 min in the experiment. The detection object just enters the field of view of the hyperspectral camera as the initial moment, and the detection object is not directly below the hyperspectral camera, but at the boundary of the shooting field of view. At this time, the spectral image boundary of the hyperspectral camera is distorted, and the closer to the boundary of the field of view, the greater the distortion. When the distortion reaches a certain degree, the spectral image at this time is distorted. Therefore, the distorted region is marked, and when the distorted region of the detection object is about to enter the distorted region behind the hyperspectral camera, timely shooting is performed to ensure that every two frames of spectral images are continuous. In the case that the position of the hyperspectral camera is fixed, different detection objects (different detection objects have different reflection rates and other parameters, which will cause different distortion parameters) and the distance between the lens of the hyperspectral camera have a difference, which will affect the position of the distorted region of the distortion. It is very important to judge the position of the distorted region of the distortion to judge the timing of shooting the second frame of spectral images before the detection object leaves the hyperspectral camera.

[0070] Embodiment 1: as shown in Figure 1 and Figure 3 The present embodiment provides a method for detecting the quality of agricultural products based on big data, which comprises the following steps:

[0071] Obtaining a sequence of spectral image frames of a plurality of detection objects on a conveying belt, the sequence of spectral image frames being continuously obtained by a hyperspectral camera at a fixed frame rate, and each spectral image frame being marked with a time stamp and object position coordinates on the conveying belt;

[0072] Segmenting and identifying the contour of the detection object, finding the edge distortion value of the current spectral image frame, and marking the detection object with an edge distortion value greater than a preset distortion threshold as a distorted object; calculating the longest time difference between the acquisition of two spectral frames before and after the distorted object is directly below the hyperspectral camera as the time difference;

[0073] The time difference sequence is obtained by the time difference between the time stamp of the first spectral image frame collected by the distortion object at different positions of the conveying belt and the time stamp of the corresponding second view frame, and the detection coordinate sequence is obtained by the first frame distortion coordinates of the distortion object.

[0074] The time period when the detection object enters the collection field of view is divided into multiple collection time points by a preset time interval, multiple collection time points are randomly selected and used as the collection time points of the first frames, and the collection time points of the first frames are used to obtain the collection time points of the second frames through the spectral frame collection network; the time difference between the collection time points of the first frames and the collection time points of the second frames is calculated, and the collection time points of the first frames and the collection time points of the second frames corresponding to the maximum time difference are used as the spectral collection scheme of the hyperspectral camera for the detection object.

[0075] Exemplarily, the conveying belt running speed is set to 0.6 meters per second, the hyperspectral camera is fixed 40 centimeters above the conveying belt, and the camera exposure time is set to 14 milliseconds. 100 red fuji apples with a diameter of 80 millimeters are placed on the conveying belt in turn, and the apple spacing is kept at 150 millimeters. The hyperspectral camera is started to continuously shoot, and a spectral image containing 168 spectral bands is obtained every 14 milliseconds. The first apple enters the shooting field of view from a position 300 millimeters in front of the camera center point, and the shooting lasts for 250 seconds, obtaining a sequence of 17857 spectral image frames. Each spectral image is labeled with a time stamp, the first frame is recorded as 0.000 seconds, the 100th frame is recorded as 1.386 seconds, and the 17857th frame is recorded as 249.798 seconds. The real-time position coordinates of each apple are recorded, the initial coordinates of the first apple are 300 millimeters in front, and after 1.386 seconds, it moves to the position directly below the camera 0 millimeters, and after another 1.386 seconds, it moves to the position 300 millimeters behind and leaves the shooting field of view. Since the spacing between each apple is 150 millimeters, the second apple enters the shooting field of view 0.25 seconds later than the first apple, and the 100th apple enters the shooting field of view 24.75 seconds later than the first apple.

[0076] The obtained 17857 frame spectral images are processed frame by frame for apple contour segmentation and edge distortion value calculation. Due to the optical distortion of the push-broom hyperspectral camera at the edge of the field of view, the farther away from the center of the field of view, the more serious the image distortion, especially at the maximum scanning angle position, which will cause serious distortion of spectral data, affecting the accuracy of sugar content and internal defect detection. The 5th frame spectral image of the first apple entering the shooting field of view is analyzed, and the apple contour is divided into 12 detection area bands perpendicular to the moving direction of the conveyor belt. The average spectral reflectance intensity of each area band in the 400-1000 nm wavelength range is calculated. The standard deviation of the 12 spectral reflectance intensity averages is 38.6, the maximum spectral reflectance intensity average is 285.7, and the ratio of the two is 0.135, which exceeds the preset distortion threshold of 0.120. This apple is marked as a distortion object. The apple position coordinate at this time is recorded as 280 mm in front, marked as the first frame distortion coordinate. Continue to track the apple, when it runs to the 100th frame position, the edge distortion value decreases to 0.042, and when it runs to the 195th frame position, the edge distortion value increases to 0.128.

[0077] The time difference between the 5th frame timestamp 0.056 seconds and the 195th frame timestamp 2.716 seconds is calculated to be 2.660 seconds. The first frame distortion coordinate 280 mm and the time difference 2.660 seconds form a set of data. The remaining 99 apples are processed in the same way, and the distortion coordinate range is between 275 mm and 285 mm, corresponding to a time difference range of 2.645 seconds to 2.675 seconds. Due to the differences in size, shape, surface roughness and reflectivity of different apples, the distortion parameters are slightly different, and a large amount of sample data is needed to establish an accurate mapping relationship. The 100 first frame distortion coordinate data form a detection coordinate sequence, and the 100 time difference data form a time difference sequence, and a three-layer feedforward neural network is constructed to establish a nonlinear mapping relationship between the two sequences. The number of input layer nodes is set to 1, the number of hidden layer nodes is set to 15, and the number of output layer nodes is set to 1. 80 samples are used for training and 20 samples are used for verification. After 1500 iterations of training, a converged spectral frame acquisition network is obtained.

[0078] The 4.6-second time period in which the new detection object enters the shooting field of view is evenly divided into 46 acquisition time points at 0.1-second intervals. The 5th, 15th, 25th, 35th, and 42nd acquisition time points are randomly selected as the first frame candidate shooting time points, corresponding to 0.5 seconds, 1.5 seconds, 2.5 seconds, 3.5 seconds, and 4.2 seconds, respectively. The corresponding apple position coordinates 270 mm, 210 mm, 150 mm, 90 mm, and 48 mm are input into the trained spectral frame acquisition network, and the second frame acquisition time points are calculated as 3.165 seconds, 4.165 seconds, 5.165 seconds, 6.165 seconds, and 6.865 seconds, respectively. The time differences of each group are 2.665 seconds, 2.665 seconds, 2.665 seconds, 2.665 seconds, and 2.665 seconds, respectively. Since the time differences are equal, the first frame acquisition time point of 0.5 seconds and the second frame acquisition time point of 3.165 seconds corresponding to the first group are taken as the optimal spectral acquisition scheme for the detection object. Through this network prediction method based on big data training, the best double-frame shooting time can be quickly determined according to the distorted position of the apple when it initially enters the field of view, ensuring detection accuracy and avoiding repeated shooting and wasting of computing resources.

[0079] Further, the segmentation and identification of the detection object's contour and the finding of the edge distortion value of the current spectral image frame and the marking of the detection object with an edge distortion value greater than the preset distortion threshold as a distorted object; the method for calculating the longest time difference between two spectral frame acquisitions through the edge distortion values of the distorted object before and after the hyperspectral camera directly below includes:

[0080] Segmenting and identifying the contour of the detection object in the spectral image frame, dividing the contour into multiple detection area bands perpendicular to the moving direction of the conveyor belt and sorting the detection area bands in a positive direction according to the distance from the hyperspectral camera, and sequentially calculating the average spectral reflectance intensity of the pixel points in each detection area band within the preset spectral band range. The ratio of the standard deviation of the spectral reflectance intensity of the detection area band to the maximum value in the sequence of the average spectral reflectance intensity is taken as the edge distortion value of the current spectral image frame.

[0081] The first frame distortion coordinate and the second frame distortion coordinate of the first frame distortion coordinate are obtained through the edge distortion values of the distorted object before and after the hyperspectral camera directly below. The time difference is obtained through the time stamps corresponding to the first spectral image frame and the second frame.

[0082] Exemplarily, for the above-mentioned obtained spectral image frame sequence, the 5th frame of spectral image when the first apple just enters the shooting field of view is selected for detailed analysis. The edge detection algorithm is used to identify the complete contour boundary of the apple, and the contour is elliptical with a long axis of 82 mm and a short axis of 78 mm. The identified apple contour is divided into 12 equal-width detection area bands perpendicular to the moving direction of the conveying belt, and each detection area band has a width of 6.8 mm. According to the straight-line distance between each detection area band and the hyperspectral camera, the first detection area band is closest to the camera at 395 mm, and the 12th detection area band is farthest from the camera at 477 mm. The spectral reflectance intensity values of all pixel points in each detection area band within the preset spectral wavelength range of 400 nm to 1000 nm are extracted.

[0083] The average spectral reflectance intensity of 428 pixel points contained in the first detection area band within 168 spectral wavelength bands is calculated. The spectral reflectance intensity values of the 428 pixel points are added one by one, the first pixel point has an accumulated value of 48572 in 168 wavelength bands, the second pixel point has an accumulated value of 47896, and so on to the 428th pixel point with an accumulated value of 49203. The sum of the accumulated values of all pixel points is 20863584, divided by the total number of pixel points 428 and the total number of spectral wavelength bands 168, and the average spectral reflectance intensity of the first detection area band is calculated as 290.3. The average spectral reflectance intensity of the remaining 11 detection area bands is calculated in the same way, which are 285.7, 282.1, 278.6, 275.2, 271.8, 268.4, 265.1, 261.7, 258.3, 254.9, and 251.5. The 12 average spectral reflectance intensity values are arranged in ascending order of detection area band number to form an average spectral reflectance intensity sequence: 290.3, 285.7, 282.1, 278.6, 275.2, 271.8, 268.4, 265.1, 261.7, 258.3, 254.9, 251.5.

[0084] The arithmetic mean of the sequence of spectral reflectance intensity averages is calculated by adding the 12 values to obtain a sum of 3243.6 and dividing by the total number of bands 12 to obtain an arithmetic mean of 270.3. The standard deviation of the sequence of spectral reflectance intensity averages is calculated by first calculating the square of the difference of each value from the mean: the square of the difference of the first value is 400.0, the square of the difference of the second value is 238.1, and so on. The sum of the squares of the differences is 1688.4, which is divided by 11 to obtain the square root, which is the standard deviation 12.4. The largest spectral reflectance intensity average in the sequence of spectral reflectance intensity averages is found to be 290.3. The ratio of the standard deviation 12.4 to the largest spectral reflectance intensity average 290.3 is calculated to obtain an edge distortion value of the current spectral image frame of 0.043. Since the push-broom hyperspectral camera produces non-uniform spectral response at the edge of the field of view, the ratio of the standard deviation to the maximum value can effectively quantify the degree of edge distortion, and the larger the ratio, the more severe the spectral data distortion.

[0085] The process of the apple movement is continuously tracked, and when the apple runs to the 85th frame position before the hyperspectral camera is directly below, the edge distortion value is calculated according to the same method as 0.135, which exceeds the preset distortion threshold value 0.120. The apple is marked as a distortion object, and the detection zone band coordinates corresponding to the 85th frame are recorded as 280 mm in front, which is marked as the first frame distortion coordinate. The area before the 280 mm position in front of the first frame distortion coordinate is marked as the first distortion area, and the area after the first frame distortion coordinate is marked as the first recognition area. After the apple runs to the hyperspectral camera directly below, the edge distortion value of each frame is calculated in reverse order from the last frame. The edge distortion value of the 195th frame is 0.128, the edge distortion value of the 194th frame is 0.125, and the edge distortion value of the 193rd frame is 0.118. The edge distortion value 0.118 calculated for the first time at the 193rd frame is less than the preset distortion threshold value 0.120, and the detection zone band coordinates corresponding to this frame are recorded as 275 mm in the rear, which is marked as the second frame distortion coordinate. The time difference is calculated to be 1.512 seconds by the timestamp 1.176 seconds of the 85th frame and the timestamp 2.688 seconds of the 193rd frame.

[0086] Further, the method of obtaining the first frame distortion coordinate of the first frame distortion coordinate and the second frame distortion coordinate by the edge distortion values of the distortion object before and after the hyperspectral camera is directly below includes:

[0087] When the edge distortion value of the detection area band of the first spectral image frame obtained before the detection object reaches directly below the hyperspectral camera is greater than the preset distortion threshold, the detection object is marked as a distortion object, the coordinates of the detection area band are marked as the first frame distortion coordinates, and the areas before and after the first frame distortion coordinates of the first spectral image frame are marked as the first distortion area and the first identification area, respectively; when the edge distortion value is less than or equal to the preset distortion threshold, the first frame distortion coordinates are set as the coordinates corresponding to the boundary of the first spectral image frame;

[0088] When the second spectral image frame is obtained after the distortion object reaches directly below the hyperspectral camera, the edge distortion value of each frame of the second spectral image frame is calculated in reverse order of the second spectral image frame and the corresponding second frame distortion coordinates are marked, the areas before and after the second frame distortion coordinates of the second spectral image frame are marked as the second distortion area and the second identification area, respectively, until the first distortion area is in the second identification area for the first time and the corresponding second frame distortion coordinates are marked as the second view coordinates, and the second spectral image frame corresponding to the second view coordinates is marked as the second view frame; and the time difference is obtained through the time stamps corresponding to the first spectral image frame and the second view frame.

[0089] Exemplarily, according to the above-mentioned calculation result of the edge distortion value, the specific positions of the first frame distortion coordinates and the second frame distortion coordinates are further determined. When the first apple reaches directly below the hyperspectral camera, the edge distortion value is calculated frame by frame from the 5th frame, the edge distortion value of the 5th frame is 0.147, the edge distortion value of the 15th frame is 0.142, the edge distortion value of the 25th frame is 0.138, the edge distortion value of the 35th frame is 0.133, the edge distortion value of the 45th frame is 0.129, the edge distortion value of the 55th frame is 0.124, and the edge distortion value of the 65th frame is 0.119. For the first time, the edge distortion value 0.119 of the 65th frame is less than the preset distortion threshold 0.120, but in order to ensure the reliability of the data, the tracing is continued to the 85th frame, and the edge distortion value of the 85th frame is 0.135, which exceeds the preset distortion threshold 0.120. The apple is marked as a distortion object, the center position coordinates of the apple corresponding to the 85th frame are recorded as 220 mm in front of the center point of the camera, which are marked as the first frame distortion coordinates. All areas before the position 220 mm in front of the first frame distortion coordinates are marked as the first distortion area, and all areas after the first frame distortion coordinates are marked as the first identification area. This division method ensures that the first distortion area contains all image areas with unreliable detection quality.

[0090] When the apple runs to the position directly below the hyperspectral camera, corresponding to the 140th frame, the edge distortion value reaches the minimum value of 0.038, and the best imaging quality is obtained. When the apple continues to move forward beyond the position directly below the camera, the edge distortion value is calculated in reverse order from the last frame of the shooting sequence. Starting from the reverse analysis of the 210th frame, the apple has completely left the shooting field of view, the edge distortion value of the 200th frame is 0.156, the 195th frame is 0.148, the 190th frame is 0.141, the 185th frame is 0.134, the 180th frame is 0.126, and the 175th frame is 0.118. The edge distortion value 0.118 calculated for the first time at the 175th frame is less than the preset distortion threshold 0.120, and the coordinate of the center of the apple corresponding to this frame is recorded as 195 mm behind the center of the camera, which is marked as the second frame distortion coordinate. All areas before the second frame distortion coordinate, i.e. 195 mm behind the center of the camera, are marked as the second distortion area, and all areas after the second frame distortion coordinate are marked as the second identification area.

[0091] Verify whether the first distortion area is completely contained in the second identification area. The range of the first distortion area is all areas before the position 220 mm in front of the center of the camera, and the range of the second identification area is all areas after the position 195 mm behind the center of the camera. Due to the continuous movement of the conveyor belt, the position 220 mm in front of the first distortion area will move to the position 195 mm behind after a period of time, so the first distortion area is indeed completely contained in the second identification area, satisfying condition 1. It is also verified that the 175th frame is indeed the frame in which the edge distortion value is last less than the threshold before the apple leaves the shooting field of view. By checking the edge distortion values of the 176th to 200th frames, which are all greater than 0.120, it is confirmed that the 175th frame is the last effective spectral image frame, satisfying condition 2. The coordinate 195 mm behind the 175th frame is recorded as the second framing coordinate, and the 175th frame is recorded as the second framing frame. This determination method ensures that the apple region in the distortion area during the first shooting can obtain clear and accurate spectral data during the second shooting, while avoiding invalid shooting after the apple completely leaves the field of view.

[0092] The time difference of 1.260 seconds is calculated by the 85th frame timestamp 1.176 seconds and the 175th frame timestamp 2.436 seconds, which represents the time window from the first time the apple enters the effective shooting area to the last time it leaves the effective shooting area, and is also the optimal shooting interval to ensure obtaining complete and high-quality spectral data of the apple. Since the hyperspectral image contains rich information of 168 bands, the data volume increases by tens of times compared to traditional RGB image, and accurate control of the shooting time is crucial to reduce the data processing burden and improve online detection efficiency. According to the same method, the remaining 99 apples are processed to determine their respective first frame distortion coordinates, second framing coordinates and corresponding time difference data, providing sufficient training samples for the subsequent mapping network of distortion coordinates and time difference.

[0093] Further, the method of taking the ratio of the standard deviation of the spectral reflectance intensity mean of the detection area band to the maximum value in the spectral reflectance intensity mean sequence as the edge distortion value of the current spectral image frame comprises:

[0094] Obtain the kth detection area band in the current spectral image frame, and extract the spectral reflectance intensity value I kpq of the pth pixel point in the qth spectral band in the detection area band within the preset spectral band range, where k is the detection area band number, p is the pixel point number, and q is the spectral band number; calculate the spectral reflectance intensity mean R k of the kth detection area band within the preset spectral band range:

[0095]

[0096] where P is the total number of pixel points included in the kth detection area band, and Q is the total number of spectral bands within the preset spectral band range;

[0097] Calculate the spectral reflectance intensity mean R k of all detection area bands in the current spectral image frame: K where K is the total number of detection area bands in the current spectral image frame; calculate the arithmetic mean of the spectral reflectance intensity mean sequence R

[0098]

[0099] Calculate the standard deviation S of the spectral reflectance intensity mean sequence:

[0100]

[0101] The maximum spectral reflectance intensity mean value in the spectral reflectance intensity mean value sequence is recorded as M; the ratio of the standard deviation S to the maximum spectral reflectance intensity mean value M is taken as the edge distortion value of the current spectral image frame

[0102] Exemplarily, according to the detection area band division result obtained above, the edge distortion value of the current spectral image frame is calculated in detail, specifically, the spectral image of the first apple at the 85th frame position is selected as the calculation object, and the apple contour is divided into 12 detection area bands in this frame. The spectral data of all pixel points in the first detection area band is extracted, the first detection area band contains 428 pixel points, and each pixel point contains the reflectance intensity values of 168 spectral bands in the 400 nanometer to 1000 nanometer waveband range. The spectral reflectance intensity value of the first pixel point in the first spectral band in the first detection area band is recorded as 289, the second spectral band value is 291, and so on to the 168th spectral band value of 295. The values of the 168 spectral bands of this pixel point are added to obtain 48572. The spectral band cumulative values of the remaining 427 pixel points in the detection area band are calculated in the same way, which are 47896, 49203, 48745, 47652, and so on.

[0103] The spectral reflectance intensity mean value of the first detection area band in the preset spectral band range is calculated, specifically, the cumulative values of the 428 pixel points are added to obtain a total of 20863584, and the total is divided by the product of the total number of pixel points 428 and the total number of spectral bands 168, that is, 20863584 divided by 71904, to calculate the spectral reflectance intensity mean value of the first detection area band as 290.3. The remaining 11 detection area bands are processed in the same way, the second detection area band contains 435 pixel points, and the spectral reflectance intensity mean value is 285.7, the third detection area band contains 441 pixel points, and the spectral reflectance intensity mean value is 282.1, and so on to the 12th detection area band containing 398 pixel points, and the spectral reflectance intensity mean value is 251.5. The spectral reflectance intensity mean values of the 12 detection area bands are arranged in ascending order of detection area band number to form a spectral reflectance intensity mean value sequence: 290.3, 285.7, 282.1, 278.6, 275.2, 271.8, 268.4, 265.1, 261.7, 258.3, 254.9, 251.5.

[0104] The arithmetic mean of the sequence of mean spectral reflectance intensity values is calculated by adding the 12 values in the sequence, 290.3 + 285.7 + 282.1 + 278.6 + 275.2 + 271.8 + 268.4 + 265.1 + 261.7 + 258.3 + 254.9 + 251.5, to obtain a sum of 3243.6. The sum of 3243.6 is divided by the total number of bands in the detection region, 12, to obtain an arithmetic mean of 270.3. The standard deviation of the sequence of mean spectral reflectance intensity values is calculated by first calculating the difference between each value and the mean value of 270.3, 290.3 - 270.3 = 20.0 for the first value, 285.7 - 270.3 = 15.4 for the second value, and so on to obtain 12 differences: 20.0, 15.4, 11.8, 8.3, 4.9, 1.5, -1.9, -5.2, -8.6, -12.0, -15.4, -18.8. Each difference is squared to obtain the squared differences: 400.0, 237.2, 139.2, 68.9, 24.0, 2.3, 3.6, 27.0, 74.0, 144.0, 237.2, 353.4.

[0105] The 11 squared differences are added together, 400.0 + 237.2 + 139.2 + 68.9 + 24.0 + 2.3 + 3.6 + 27.0 + 74.0 + 144.0 + 237.2 + 353.4, to obtain a sum of 1710.8. The sum of 1710.8 is divided by the total number of bands in the detection region minus 1, i.e., 11, to obtain 155.5, which is then square rooted to obtain a standard deviation of 12.5. The largest mean spectral reflectance intensity value in the sequence of mean spectral reflectance intensity values is found by comparing the 12 values, and is found to be 290.3 for the first detection region band, which is recorded as the maximum mean spectral reflectance intensity value of 290.3. The standard deviation of 12.5 is divided by the maximum mean spectral reflectance intensity value of 290.3 to obtain 0.043, which is used as the edge distortion value for the current spectral image frame. This calculation method can effectively quantify the imaging quality difference of the push-broom hyperspectral camera at different field of view positions. The standard deviation reflects the degree of dispersion of the spectral data, and the maximum value reflects the spectral intensity of the best imaging region. The smaller the ratio of the two, the better the image quality.

[0106] By comparing the calculated edge distortion value of 0.043 in the spectral image of frame 85 with the preset distortion threshold of 0.120, it was found that the value was less than the threshold, indicating that although the apple had not yet reached the optimal position directly under the camera, it was within the acceptable imaging quality range. Continuing to calculate the edge distortion values ​​of the apple in other frames, the value was 0.135 when the apple was in frame 65, exceeding the preset threshold. The value dropped to its lowest point of 0.028 when the apple was directly under the camera in frame 140, and reached a value of 0.118 when the apple was about to leave the effective shooting range in frame 175. This quantitative calculation method accurately determines the optimal shooting time for each detected object during conveyor belt movement, ensuring high-quality spectral data for subsequent sugar content detection and internal defect identification.

[0107] Furthermore, the method for establishing the mapping relationship between the detection coordinate sequence and the time difference sequence to obtain the spectral frame acquisition network includes:

[0108] The distortion coordinates of the first frame of the i-th distorted object in the detected coordinate sequence are used as the input vector X of the neural network. i The corresponding time difference is used as the target output vector Y of the neural network. i , where i is the index of the distorted object; construct a multi-layer feedforward neural network, set the number of input layer nodes to A, the number of hidden layer nodes to B, the number of output layer nodes to C, and the total number of network layers to L;

[0109] Initialize the weight parameters W between the layers of the neural network j and bias parameter V j , where j is the parameter index; calculate the weighted input Z of the h-th neuron in the hidden layer. h :

[0110]

[0111] Where N g W is the output value of the g-th neuron in the input layer. g V represents the corresponding connection weight. h This represents the bias of the h-th neuron in the hidden layer;

[0112] Weighted input Z h The output N of the h-th neuron in the hidden layer is obtained using the hyperbolic tangent activation function. h :

[0113]

[0114] The output values ​​of the output layer neurons are calculated using the same method, and the final predicted time difference of the network is obtained through forward propagation. The mean squared error loss function is used to calculate the predicted time difference and the actual time difference Y. ierror between them, the weight parameters W and the bias parameters V are updated by back propagation algorithm j and bias parameters V j ; repeat the training process until the network converges, and the trained neural network is taken as the mapping relationship of the detection coordinate sequence to the time difference sequence, denoted as the spectrum frame acquisition network.

[0115] Exemplarily, according to the detection coordinate sequence and the time difference sequence established above, a neural network mapping relationship is constructed, specifically, 100 first frame distortion coordinate data of apples are grouped to form a detection coordinate sequence, and the coordinate range is between 275 mm and 285 mm in front, and 100 corresponding time difference data are grouped to form a time difference sequence, and the time difference range is between 1.245 s and 1.275 s. Take the first frame distortion coordinate 280 mm of the first apple as the neural network input vector, and the corresponding time difference 1.260 s as the neural network target output vector. Take the first frame distortion coordinate 278 mm of the second apple as the second input vector, and the corresponding time difference 1.255 s as the second target output vector, and so on to establish 100 groups of input and output data pairs. A three-layer feedforward neural network is constructed, the number of nodes in the input layer is set to 1 for receiving a single distortion coordinate value, the number of nodes in the hidden layer is set to 15, the number of nodes in the output layer is set to 1 for outputting the predicted time difference value, and the total number of network layers is set to 3.

[0116] The weight parameters and bias parameters between the layers of the neural network are initialized. The initial values of the 15 weight parameters from the input layer to the hidden layer are set to 0.23, -0.18, 0.31, -0.27, 0.19, -0.34, 0.26, -0.21, 0.29, -0.16, 0.33, -0.25, 0.17, -0.32, 0.28, the initial values of the bias parameters of the 15 neurons in the hidden layer are set to 0.12, -0.09, 0.15, -0.11, 0.08, -0.14, 0.13, -0.07, 0.16, -0.10, 0.14, -0.12, 0.09, -0.13, 0.11. The initial values of the 15 weight parameters from the hidden layer to the output layer are set to 0.42, -0.38, 0.45, -0.41, 0.39, -0.46, 0.43, -0.37, 0.44, -0.40, 0.47, -0.35, 0.36, -0.48, 0.34, and the initial value of the bias parameter of the neuron in the output layer is set to 0.05.

[0117] The weighted input of the first neuron in the hidden layer is calculated by multiplying the output value of the input layer neuron 280 mm by the corresponding connection weight 0.23 to obtain 64.4, and adding the bias of the first neuron in the hidden layer 0.12 to obtain the weighted input 64.52. The weighted inputs of the remaining 14 neurons in the hidden layer are calculated in the same way, which are -50.28, 86.95, -75.59, 53.32, -95.16, 72.93, -58.61, 81.36, -44.70, 92.54, -69.88, 47.63, -89.47, and 78.51, respectively. The weighted input 64.52 of the first neuron in the hidden layer is calculated by the hyperbolic tangent activation function. First, the 64.52 power of the natural constant e is calculated to obtain a very large value, and the negative 64.52 power of the natural constant e is calculated to obtain a very small value close to 0. After subtraction and division by the sum of the two, the output value of the first neuron in the hidden layer is close to 1.0. The output values of the remaining 14 neurons in the hidden layer are calculated in the same way. Since the absolute values of the weighted inputs are large, the output values after the hyperbolic tangent activation function are close to 1.0 or -1.0.

[0118] The output layer neurons are processed using the same weighted summation and activation function calculation method. Specifically, the output values of the 15 neurons in the hidden layer are multiplied by the corresponding weights from the hidden layer to the output layer, and then summed to obtain the weighted input of the output layer neuron, and then the final predicted time difference of the network is obtained by the activation function. The mean square error loss function is used to calculate the error between the predicted time difference and the actual time difference 1.260 seconds. The calculation formula is the square of the difference between the predicted value and the actual value. The first forward propagation obtains a predicted time difference of 1.178 seconds, and the error between the predicted time difference and the actual time difference 1.260 seconds is 0.082 seconds, and the square error is 0.00672. The gradient of the error with respect to each weight parameter and bias parameter is calculated by the backpropagation algorithm, and the parameter values are updated according to the gradient descent method, and the learning rate is set to 0.01.

[0119] The training process is repeated for 100 groups of training data, and each time the entire data is completed, it is called a repeated learning number. After the first repeated learning number, the average error of all samples is 0.0156, and after the 100th repeated learning number, the average error is reduced to 0.0023, and after the 500th repeated learning number, the average error is reduced to 0.0008, and after the 1000th repeated learning number, the average error is reduced to 0.0003, and after the 1500th repeated learning number, the average error is stable at about 0.0002 and no longer decreases significantly, and it is judged that the network has converged. The network performance is tested using 20 validation samples that did not participate in the training, and the input distortion coordinates are 282 mm, and the predicted time difference is 1.264 seconds, and the actual time difference is 1.268 seconds, with an error of only 0.004 seconds, and the prediction accuracy reaches 99.7%. The trained neural network is used as a mapping relationship between the coordinate sequence of the detection object and the time difference sequence, which can quickly predict the optimal double-frame shooting time interval according to the first frame of distortion coordinates of the new detection object, and is recorded as a spectral frame acquisition network. This neural network mapping method based on big data training is more suitable for individual differences of different varieties and sizes of apples than traditional empirical formulas, and significantly improves the accuracy and applicability of the spectral acquisition scheme.

[0120] Example 2: based on the same inventive concept, as Figure 2 The embodiment also provides a big data-based agricultural product quality detection system, as shown in the figure, which comprises:

[0121] A spectral acquisition module is used to obtain a sequence of spectral image frames of a plurality of detection objects on a conveyor belt, and the spectral image frame sequence is obtained continuously by a hyperspectral camera at a fixed frame rate. Each spectral image frame is labeled with a timestamp and an object position coordinate on the conveyor belt;

[0122] A time difference acquisition module is used to segment and identify the contour of the detection object, find the edge distortion value of the current spectral image frame, and mark the detection object with an edge distortion value greater than a preset distortion threshold as a distorted object. The longest time for two spectral frame acquisitions is calculated by the edge distortion values of the distorted object before and after the hyperspectral camera, and is recorded as the time difference;

[0123] A spectral frame acquisition training module is used to obtain a time difference sequence by using the timestamps of the first spectral image frame and the corresponding second view frame of the distorted object at different conveyor belt positions when the conveyor belt speed remains unchanged, and to obtain a detection coordinate sequence by using the first frame distortion coordinates of the distorted object. The mapping relationship between the detection coordinate sequence and the time difference sequence is established to obtain a spectral frame acquisition network;

[0124] The acquisition optimization module is configured to divide a time period when the detection object enters the acquisition field of view into a plurality of acquisition time points by a preset time interval, randomly select a plurality of acquisition time points as a plurality of first frame acquisition time points, and obtain second frame acquisition time points by the light spectrum frame acquisition network according to the first frame acquisition time points.

[0125] Further, the system further comprises:

[0126] The distortion identification module is configured to segment and identify the contour of the detection object in the light spectrum image frame, divide the contour into a plurality of detection area bands in a direction perpendicular to the moving direction of the conveying belt, and sort the detection area bands in a positive direction according to the distance from the hyperspectral camera, sequentially calculate the average light spectrum reflection intensity of the pixel points in each detection area band in a preset light spectrum waveband range, and take the ratio of the standard deviation of the light spectrum reflection intensity average of the detection area band to the maximum value in the light spectrum reflection intensity average sequence as the edge distortion value of the current light spectrum image frame.

[0127] The first frame distortion coordinates and the second frame distortion coordinates are obtained by the edge distortion values of the distortion object before and after the distortion object is directly below the hyperspectral camera. The time difference is obtained by the time stamps corresponding to the first light spectrum image frame and the second view frame.

[0128] Further, the system further comprises:

[0129] The time difference calculation module is configured to, when the edge distortion value obtained from the detection area band of the first light spectrum image frame acquired before the detection object reaches directly below the hyperspectral camera is greater than a preset distortion threshold, mark the detection object as a distortion object, mark the coordinates of the detection area band as first frame distortion coordinates, and mark the areas before and after the first frame distortion coordinates of the first light spectrum image frame as a first distortion area and a first identification area, respectively; and when the edge distortion value is less than or equal to the preset distortion threshold, set the first frame distortion coordinates as the coordinates corresponding to the boundary of the first light spectrum image frame.

[0130] When the second light spectrum image frame is acquired after the distortion object reaches directly below the hyperspectral camera, the edge distortion values of each second light spectrum image frame are sequentially calculated in a reverse order of the second light spectrum image frame, and the corresponding second frame distortion coordinates are marked, the areas before and after the second frame distortion coordinates of the second light spectrum image frame are marked as a second distortion area and a second identification area, respectively, until the first distortion area is in the second identification area for the first time, and the corresponding second frame distortion coordinates are marked as a second view coordinate, and the second light spectrum image frame corresponding to the second view coordinate is marked as a second view frame. The time difference is obtained by the time stamps corresponding to the first light spectrum image frame and the second view frame.

[0131] Further, the system further comprises:

[0132] an edge distortion value calculation module, configured to obtain a kth detection area band in a current spectral image frame, extract a spectral reflectance intensity value I kpq of a pth pixel point in the detection area band in a preset spectral band range, wherein k is a detection area band number, p is a pixel point number, and q is a spectral band number; calculate a spectral reflectance intensity mean value R k of the kth detection area band in the preset spectral band range:

[0133]

[0134] wherein P is a total number of pixel points contained in the kth detection area band, and Q is a total number of spectral bands in the preset spectral band range;

[0135] arrange the spectral reflectance intensity mean values R k of all the detection area bands in the current spectral image frame in ascending order of the detection area band number k to form a spectral reflectance intensity mean value sequence R1, R2, …, R K , wherein K is a total number of detection area bands in the current spectral image frame; calculate an arithmetic mean value of the spectral reflectance intensity mean value sequence

[0136]

[0137] calculate a standard deviation S of the spectral reflectance intensity mean value sequence:

[0138]

[0139] find a spectral reflectance intensity mean value with the maximum value in the spectral reflectance intensity mean value sequence, and denote the spectral reflectance intensity mean value as M; take a ratio of the standard deviation S to the maximum spectral reflectance intensity mean value M as an edge distortion value of the current spectral image frame

[0140] Further, the system further comprises:

[0141] a neural network training module, configured to take a first frame distortion coordinate of an ith distortion object in the detection coordinate sequence as a neural network input vector X i , take a corresponding time difference as a neural network target output vector Y i , wherein i is a distortion object number; construct a multi-layer feedforward neural network, set an input layer node number as A, a hidden layer node number as B, an output layer node number as C, and a total network layer number as L;

[0142] initialize weight parameters W j and bias parameters V between layers of the neural networkj where j is the parameter index; the weighted input Z h

[0143]

[0144] where N g is the output value of the gth neuron in the input layer, W g is the corresponding connection weight, V h is the bias of the hth neuron in the hidden layer;

[0145] The weighted input Z h is obtained by the hyperbolic tangent activation function, and the output N h of the hth neuron in the hidden layer is obtained.

[0146]

[0147] The output value of the neuron in the output layer is calculated using the same method, and the final prediction time difference of the network is obtained by forward propagation. The error between the predicted time difference and the actual time difference Y i is calculated using the mean square error loss function, and the weight parameters W j and the bias parameters V j are updated by the back propagation algorithm; the training process is repeated until the network converges, and the trained neural network is used as the mapping relationship between the detection coordinate sequence and the time difference sequence, denoted as the spectrum frame acquisition network.

[0148] It should be noted that, as for the system in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0149] Finally, it should be noted that although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to part of the technical features, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.​

Claims

1. A method for detecting agricultural product quality based on big data, characterized in that, The method includes: A sequence of spectral image frames of multiple consecutive objects to be detected on a conveyor belt is acquired. The sequence of spectral image frames is continuously acquired by a hyperspectral camera at a fixed frame rate. Each spectral image frame is marked with a timestamp and the coordinates of the position of the detected object on the conveyor belt. The contours of the detected objects are segmented and identified, and the edge distortion values ​​of the current spectral image frame are found. Detected objects with edge distortion values ​​greater than a preset distortion threshold are marked as distorted objects. The longest time between two spectral frames is calculated by using the edge distortion values ​​of the distorted object before and after it is directly below the hyperspectral camera, and recorded as the time difference. With the conveyor belt speed remaining constant, a time difference sequence is obtained by combining the timestamps of the first spectral image frames acquired at different positions on the conveyor belt and the timestamps of the corresponding second view frames. The distortion coordinates of the first frame of the distorted object are then combined to form a detection coordinate sequence. A mapping relationship between the detection coordinate sequence and the time difference sequence is established to obtain the spectral frame acquisition network. The time period during which the detected object enters the field of view is divided into multiple acquisition moments by a preset time interval. Multiple acquisition moments are randomly selected and used as multiple first frame acquisition moments. The acquisition moments of the first frames are used to obtain the acquisition moments of the second frames through a spectral frame acquisition network. The time difference between the acquisition moments of the first frames and the second frames is calculated, and the acquisition moments of the first and second frames corresponding to the maximum time difference are taken as the spectral acquisition scheme of the hyperspectral camera for the detected object. The method of segmenting and identifying the contour of the detected object, finding the edge distortion value of the current spectral image frame, and marking the detected object with the edge distortion value greater than a preset distortion threshold as a distorted object; and calculating the longest time difference between two spectral frames by using the edge distortion values ​​of the distorted object before and after it is directly below the hyperspectral camera, includes: The contour of the detected object within the spectral image frame is segmented and identified. The contour is divided into multiple detection area bands perpendicular to the direction of conveyor belt movement and sorted in the forward direction according to the distance between the detection area bands and the hyperspectral camera. The mean spectral reflectance intensity of the pixels in each detection area band within the preset spectral band range is calculated in turn. The ratio of the standard deviation of the mean spectral reflectance intensity of the detection area band to the maximum value in the sequence of mean spectral reflectance intensity is used as the edge distortion value of the current spectral image frame. The first frame distortion coordinates and the second frame distortion coordinates are obtained by using the edge distortion values ​​of the distorted object before and after it is directly below the hyperspectral camera; the time difference is obtained by using the timestamps corresponding to the first spectral image frame and the second viewfinder frame.

2. The method for agricultural product quality testing based on big data according to claim 1, characterized in that, The first frame distortion coordinates and the second frame distortion coordinates are obtained by using the edge distortion values ​​of the distorted object before and after it is directly below the hyperspectral camera. Methods for obtaining the time difference using the timestamps corresponding to the first spectral image frame and the second viewfinder frame include: When the edge distortion value of the detection region of the first spectral image frame acquired before the detected object reaches directly below the hyperspectral camera is greater than a preset distortion threshold, the detected object is marked as a distorted object, and the coordinates of the detection region are designated as the first frame distortion coordinates. The regions before and after the first frame distortion coordinates of the first spectral image frame are designated as the first distortion region and the first recognition region, respectively. When the edge distortion value is less than or equal to the preset distortion threshold, the first frame distortion coordinates are set to the coordinates corresponding to the boundary of the first spectral image frame. After the distorted object reaches directly below the hyperspectral camera, the second spectral image frame is acquired. The edge distortion value of each second spectral image frame is calculated sequentially in reverse order, and the corresponding second frame distortion coordinates are marked. The regions before and after the second frame distortion coordinates of the second spectral image frame are respectively recorded as the second distortion region and the second recognition region. This process continues until the first distortion region is found to be within the second recognition region, and the corresponding second frame distortion coordinates are recorded as the second framing coordinates. The second spectral image frame corresponding to the second framing coordinates is recorded as the second framing frame. The time difference is obtained through the timestamps corresponding to the first spectral image frame and the second framing frame.

3. The method for agricultural product quality testing based on big data according to claim 2, characterized in that, The method of using the ratio of the standard deviation of the mean spectral reflectance intensity of the detection region to the maximum value in the mean spectral reflectance intensity sequence as the edge distortion value of the current spectral image frame includes: Get the current spectral image frame number The detection region band is extracted within a preset spectral band. The pixel at the th point Spectral reflectance intensity values ​​for each spectral band ,in The detection area is numbered. The pixel number For the spectral band number; calculate the first... The average spectral reflectance intensity of the detection area within the preset spectral band. : ； in For the first The total number of pixels contained in the detection region band. This represents the total number of spectral bands within the preset spectral band range. The average spectral reflectance intensity of all detected regions within the current spectral image frame. According to the serial number of the testing area Arranged from smallest to largest to form a sequence of average spectral reflectance intensity ,in The total number of detected regions within the current spectral image frame; calculate the arithmetic mean of the spectral reflectance intensity mean sequence. : ； Calculate the standard deviation of the mean sequence of spectral reflectance intensity. : ； Find the mean spectral reflectance value with the largest value in the sequence of mean spectral reflectance values, and denote it as . Standard deviation With the mean of maximum spectral reflectance The ratio is used as the edge distortion value of the current spectral image frame. .

4. The method for agricultural product quality testing based on big data according to claim 1, characterized in that, The method for establishing the mapping relationship between the detection coordinate sequence and the time difference sequence to obtain the spectral frame acquisition network includes: The first in the detection coordinate sequence The first frame distortion coordinates of the distorted object are used as the input vector of the neural network. The corresponding time difference is used as the target output vector of the neural network. ,in The index of the distorted object; construct a multi-layer feedforward neural network, setting the number of input layer nodes to [value missing]. The number of hidden layer nodes is The number of output layer nodes is The total number of network layers is ; Initialize the weight parameters between the layers of the neural network and bias parameters ,in For parameter index; calculate the hidden layer's index. Weighted input of each neuron : ； in For the input layer The output value of each neuron For the corresponding connection weights, For the hidden layer Bias of each neuron; Weighted input The hidden layer is obtained by using the hyperbolic tangent activation function. The output of each neuron : ； The output values ​​of the output layer neurons are calculated using the same method, and the final predicted time difference is obtained through forward propagation. The mean squared error loss function is used to calculate the predicted time difference and the actual time difference. The error between them is used to update the weight parameters through the backpropagation algorithm. and bias parameters Repeat the training process until the network converges. The trained neural network is used as a mapping relationship from the detection coordinate sequence to the time difference sequence, and is denoted as the spectral frame acquisition network.

5. A big data-based agricultural product quality testing system, characterized in that: The system includes: The spectral acquisition module is used to acquire a sequence of spectral image frames of multiple consecutive detection objects on the conveyor belt. The sequence of spectral image frames is continuously acquired by a hyperspectral camera at a fixed frame rate. Each spectral image frame is marked with a timestamp and the coordinates of the detected object's position on the conveyor belt. The time difference acquisition module is used to segment and identify the contour of the detected object and find the edge distortion value of the current spectral image frame, and mark the detected object with the edge distortion value greater than the preset distortion threshold as a distorted object; the longest time to acquire two spectral frames is calculated by the edge distortion value of the distorted object before and after it is directly below the hyperspectral camera, and recorded as the time difference; The spectral frame acquisition training module is used to obtain a time difference sequence by combining the timestamps of the first spectral image frames acquired at different positions of the distorted object on the conveyor belt with the timestamps of the corresponding second view frames, while keeping the conveyor belt speed constant. The module then uses the distortion coordinates of the first frame of the distorted object to form a detection coordinate sequence. Finally, it establishes a mapping relationship between the detection coordinate sequence and the time difference sequence to obtain the spectral frame acquisition network. The acquisition optimization module is used to divide the time period during which the detected object enters the acquisition field of view into multiple acquisition moments by a preset time interval, randomly select multiple acquisition moments as multiple first frame acquisition moments, and use the acquisition moments of the first frame to obtain the acquisition moments of the second frame through the spectral frame acquisition network; calculate the time difference between the acquisition moments of the first frame and the second frame, and take the acquisition moment of the first frame and the acquisition moment of the second frame corresponding to the maximum time difference as the spectral acquisition scheme of the hyperspectral camera for the detected object. The system also includes: The distortion recognition module is used to segment and recognize the contour of the detected object within the spectral image frame. The contour is divided into multiple detection area bands perpendicular to the direction of the conveyor belt movement and sorted in the forward direction according to the distance between the detection area bands and the hyperspectral camera. The mean spectral reflectance intensity of the pixels in each detection area band within the preset spectral band range is calculated in turn. The ratio of the standard deviation of the mean spectral reflectance intensity of the detection area band to the maximum value in the sequence of mean spectral reflectance intensity is used as the edge distortion value of the current spectral image frame. The first frame distortion coordinates and the second frame distortion coordinates are obtained by using the edge distortion values ​​of the distorted object before and after it is directly below the hyperspectral camera; the time difference is obtained by using the timestamps corresponding to the first spectral image frame and the second viewfinder frame.

6. The agricultural product quality testing system based on big data according to claim 5, characterized in that, The system also includes: The time difference calculation module is used to mark the detected object as a distorted object when the edge distortion value of the detection area band of the first spectral image frame acquired before the detected object reaches directly below the hyperspectral camera is greater than a preset distortion threshold, and to mark the coordinates of the detection area band as the first frame distortion coordinates. The regions before and after the first frame distortion coordinates of the first spectral image frame are respectively marked as the first distortion region and the first recognition region. When the edge distortion value is less than or equal to the preset distortion threshold, the first frame distortion coordinates are set as the coordinates corresponding to the boundary of the first spectral image frame. After the distorted object reaches directly below the hyperspectral camera, the second spectral image frame is acquired. The edge distortion value of each second spectral image frame is calculated sequentially in reverse order, and the corresponding second frame distortion coordinates are marked. The regions before and after the second frame distortion coordinates of the second spectral image frame are respectively recorded as the second distortion region and the second recognition region. This process continues until the first distortion region is found to be within the second recognition region, and the corresponding second frame distortion coordinates are recorded as the second framing coordinates. The second spectral image frame corresponding to the second framing coordinates is recorded as the second framing frame. The time difference is obtained through the timestamps corresponding to the first spectral image frame and the second framing frame.

7. The agricultural product quality testing system based on big data according to claim 6, characterized in that, The system also includes: The edge distortion value calculation module is used to obtain the value of the first edge distortion value within the current spectral image frame. The detection region band is extracted within a preset spectral band. The pixel at the th point Spectral reflectance intensity values ​​for each spectral band ,in The detection area is numbered. The pixel number For the spectral band number; calculate the first... The average spectral reflectance intensity of the detection area within the preset spectral band. : ； in For the first The total number of pixels contained in the detection region band. This represents the total number of spectral bands within the preset spectral band range. The average spectral reflectance intensity of all detected regions within the current spectral image frame. According to the serial number of the testing area Arranged from smallest to largest to form a sequence of average spectral reflectance intensity ,in The total number of detected regions within the current spectral image frame; calculate the arithmetic mean of the spectral reflectance intensity mean sequence. : ； Calculate the standard deviation of the mean sequence of spectral reflectance intensity. : ； Find the mean spectral reflectance value with the largest value in the sequence of mean spectral reflectance values, and denote it as . Standard deviation With the mean of maximum spectral reflectance The ratio is used as the edge distortion value of the current spectral image frame. .

8. The agricultural product quality testing system based on big data according to claim 5, characterized in that, The system also includes: The neural network training module is used to train the first element in the detection coordinate sequence. The first frame distortion coordinates of the distorted object are used as the input vector of the neural network. The corresponding time difference is used as the target output vector of the neural network. ,in The index of the distorted object; construct a multi-layer feedforward neural network, setting the number of input layer nodes to [value missing]. The number of hidden layer nodes is The number of output layer nodes is The total number of network layers is ; Initialize the weight parameters between the layers of the neural network and bias parameters ,in For parameter index; calculate the hidden layer's index. Weighted input of each neuron : ； in For the input layer The output value of each neuron For the corresponding connection weights, For the hidden layer Bias of each neuron; Weighted input The hidden layer is obtained by using the hyperbolic tangent activation function. The output of each neuron : ； The output values ​​of the output layer neurons are calculated using the same method, and the final predicted time difference is obtained through forward propagation. The mean squared error loss function is used to calculate the predicted time difference and the actual time difference. The error between them is used to update the weight parameters through the backpropagation algorithm. and bias parameters Repeat the training process until the network converges. The trained neural network is used as a mapping relationship from the detection coordinate sequence to the time difference sequence, and is denoted as the spectral frame acquisition network.

Images