Multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection

Abstract

The present invention belongs to the field of fruit disease detection and analysis, and specifically provides a multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection, including the following modules: an integrated device module uses an integrated device to collect image data of target fruits; a data processing module performs intelligent spectral information enhancement processing on the hyperspectral images and videos respectively; a model deployment module distills the model for the hyperspectral image data to obtain a model that can be deployed on the edge end; a disease detection module splits the teacher model by coordinating the edge ends of different data types through the master control, obtains a sub-model obtained by splitting from multiple edge ends, and uses the sub-model to perform disease quality detection tasks on fruits on a conveyor belt. The present invention collects hyperspectral images and videos of fruits through integrated equipment, and sends them to each edge device for preprocessing, and then coordinates the edge ends of different data types through the master control to split the distillation model, and quantifies and deploys it on the edge device for detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of fruit disease detection and analysis, and in particular relates to a multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection. Background Art

[0002] Fruit disease detection is a science that uses artificial intelligence technologies such as computer vision and machine learning to analyze agricultural fruit images to achieve automatic identification, diagnosis and classification of plant diseases.

[0003] Traditional disease detection methods mainly rely on manual experience, and have problems such as low efficiency, low accuracy, and susceptibility to subjective factors. With the development of spectral analysis and deep learning, it is possible to efficiently and accurately detect whether the fruit is diseased or not. Hyperspectral cameras can obtain chemical information inside the fruit, such as key indicators such as nutrients, moisture, and protein. Deep learning in computer vision can obtain important information such as the physical condition of the fruit surface, such as growth status, leaf color, and plant morphology. However, how to use the image information obtained by computer vision to detect fruit diseases and pests is also an important challenge in this field. The phenotypic analysis method based on machine learning usually requires a large amount of data to support model training. In the case of difficulty in information collection and scarcity of real-time data, the model analysis results often deviate. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to provide a multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection. The system transmits hyperspectral image videos reflecting fruit disease information to the platform, and then uses a multi-edge device coordinated linkage method to accurately analyze the characteristics of the fruit, thereby constructing an entropy reduction control video detection platform for hyperspectral image multi-teacher distillation with multi-edge device linkage under overall control;

[0005] The present invention realizes accurate and efficient detection of fruit diseases by integrating hyperspectral data reflecting fruit diseases with the interaction between edge devices, thereby reducing the problems of low resource allocation and high consumption during training and detection, and detecting the performance traits of fruits based on the fusion and distillation of multi-faceted information.

[0006] To achieve the above object, the present invention provides the following technical solutions.

[0007] An embodiment of the present invention provides a multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection, including the following modules:

[0008] An integrated device module is used to collect image data of target fruits using an integrated device, wherein the image data includes hyperspectral images and videos; the integrated device includes an acquisition system, a light source, and a conveyor belt system, wherein the acquisition system includes an image acquisition system and a hyperspectral data acquisition system, wherein a frame rate buffer is introduced into the conveyor belt system to perform real-time control on the conveyor belt speed based on the video frame rate;

[0009] A data processing module, used for intelligently enhancing the spectral information of the hyperspectral image and video;

[0010] The model deployment module is used to distill the model for hyperspectral image data into a model that can be deployed on the edge;

[0011] The disease detection module is used to coordinate the edge splitting teacher model of different data types through the master control to obtain a sub-model obtained by splitting multiple edge ends, and use the sub-model to perform disease quality detection tasks on the fruits on the conveyor belt; wherein, in the process of performing the disease quality detection task, the particle swarm algorithm PSO is used as an optimization tool of the master control mechanism to allocate and coordinate the tasks of the edge end and perform disease detection on the target fruit.

[0012] Further, in the integrated device, the hyperspectral data acquisition system includes a hyperspectral imager, which collects reflectance spectral images of target fruits in multiple continuous narrow spectral bands; the hyperspectral imager includes a spectral calibration module, a data acquisition card and a spectral preprocessing module, the spectral calibration module ensures data accuracy, the data acquisition card transmits data, and the spectral preprocessing module improves spectral data quality;

[0013] The light source is an adjustable light source, which adjusts the intensity and spectral distribution according to the characteristics of the detection object and the ambient light conditions, and the control system of the light source is used to manage the switch, intensity and spectral distribution;

[0014] The conveyor belt system includes a conveyor belt, which transports the target fruit to be inspected, and the speed of the conveyor belt is adjusted according to the size and speed of the target fruit;

[0015] The integrated device also has a built-in synchronous control system, which uses a Raspberry Pi to synchronously control the image acquisition system, the hyperspectral data acquisition system, and the conveyor belt system, so that the collected data corresponds one-to-one with the position of the target fruit.

[0016] Furthermore, in the data processing module, the collected multispectral data is subjected to ISIS-SR, or intelligent spectral information enhancement; this technology aims to not only learn diverse contextual features through a spatial-spectral dual-branch feature extraction architecture, but also enhance the model's spatial-spectral joint representation capability to enhance the spectral information in the hyperspectral image, further optimize the feature extraction process, and improve the accuracy of classification and recognition;

[0017] The joint extraction of spatial-spectral features is:

[0018] ;

[0019] in, and are the extracted multi-scale spectral features and multi-scale spatial features respectively; m is a weight that can be optimized by a neural network; is the Sigmoid activation function, which maps the input weighted features to the [0,1] interval to indicate the importance of each feature; b is the bias term, which is used to adjust the bias of the fused features; Representing multi-scale spectral features The weight parameter of Representing multi-scale spatial features The weight parameter of

[0020] By analyzing the existing hyperspectral image data, intelligent spectral information enhancement technology can predict and supplement the missing spectral information and enhance the spectral resolution of the image; after processing, it is sent to different GPU ends for preprocessing.

[0021] Furthermore, for hyperspectral data, a multi-dimensional convolutional network is used, and the spatial and spectral features are extracted using deeply separable 3D convolutions. Multiple 1D convolutions are used in the network layer to extract spectral features of different scales, so as to capture the high-dimensional features of the hyperspectral with as little resource consumption as possible. On the basis of selective distillation of spectral data, spectral difference loss is introduced, while considering the importance of bands and inter-class differences. Among them:

[0022] The selective distillation loss is expressed as:

[0023] ;

[0024] in, is the weight of the i-th band, calculated by the gradient method; KL represents the Kullback-Leibler divergence; softmax represents the activation function; T represents the coefficient for adjusting the probability distribution of the output of the softmax function; represents the output value of the student model for the i-th band; represents the actual value of the ith band;

[0025] The spectral difference loss is expressed as: ;

[0026] in, is the feature center of the student model in the i-th category, is the feature center of the teacher model in the i-th category; represents the square of L2 norm;

[0027] The classification loss is expressed as: ;

[0028] Where f(x) represents the model's predicted value for input x; y represents the actual value of input x; n represents the number of input samples;

[0029] The total loss is expressed as: ;

[0030] Among them, α, β, and γ represent the weights for balancing different losses.

[0031] Furthermore, in the task of using the sub-model to detect the disease quality of fruits on the conveyor belt, for the hyperspectral data, the wavelet transform WT and IHS transform are used to decompose the high-resolution image through wavelet, and the obtained wavelet surfaces are superimposed on the image with intensity I after the multispectral image is transformed by IHS. The WTIHSConv model is constructed in combination with the convolutional network CNN, and the detection model based on the hyperspectral data is divided into multiple edge ends, and the sub-models at different angles are applied to each edge end; wherein:

[0032] In the first edge NPU, B-spline wavelet is applied to perform data enhancement processing on hyperspectral data based on WTIHSConv. B-spline wavelet is a wavelet basis function based on B-spline function, which is expressed as: , where i is the basis function index, which is used to indicate the position in the B-spline basis function sequence; p represents the order of the B-spline, which is used to define the smoothness and local control ability of the B-spline basis function; , Represents the B-spline basis function, which has advantages in computational efficiency and local control smoothness, and is conducive to fast and accurate feature extraction;

[0033] In the feature extraction process of the second edge NPU, the hyperspectral image is decomposed at multiple scales based on the WTIHSConv model, and multi-band feature fusion is used at each scale to extract the features at the hyperspectral spatial level.

[0034] In the feature extraction process of the third edge NPU, the WTIHSConv lightweight model plus deep separable 3D convolution is used to extract spectral features;

[0035] The fused features are input into the fourth edge GPU, and the pre-trained classification model is used to perform feature classification. The classification head is used to perform classification tasks on the extracted features and output the final detection results.

[0036] Furthermore, the particle swarm algorithm PSO is used as an optimization tool of the master control mechanism to allocate and coordinate tasks on the edge end, and the particle swarm algorithm is used to allocate tasks according to the performance and current status of each edge device and the importance of the task;

[0037] In the particle swarm algorithm, each particle represents a potential solution, and the dimension of each particle is taken as the number of tasks; for the task allocation problem, each solution is a possible task allocation scheme, in which each task is assigned to a specific edge device;

[0038] Initialization phase: Randomly initialize the position and velocity of each particle and assign an initial edge device to each task;

[0039] Fitness evaluation: Evaluate the position of each particle and calculate its fitness; the fitness function is defined based on the performance, current state, and importance of the task of the edge device;

[0040] Update speed and position: According to the update formula of the particle swarm algorithm, the speed and position of each particle are updated according to the current position and speed as well as the historical best position and the global best position, where:

[0041] The update formula of the particle swarm algorithm is as follows:

[0042] ;

[0043] ;

[0044] in, is the velocity of the ith particle in the tth generation, is the position of the ith particle in the tth generation, is the best historical position of the particle, is the global optimal position, w is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers in the interval [0,1] respectively;

[0045] Update the historical best position and the global best position: compare the fitness of the current particle with its historical best fitness, update the historical best position, then compare the historical best position of each particle with the global best fitness, and update the global best position;

[0046] Iterative optimization: Repeat the process of updating speed and position until the algorithm converges or reaches the preset stop condition. The preset stop condition is that the maximum number of iterations is reached or the fitness meets the requirements;

[0047] Final task allocation: Finally, the task allocation scheme is determined based on the global optimal position, and tasks are allocated to each edge device to achieve the best system performance and task completion efficiency;

[0048] Therefore, the master control monitors the dynamic situation of each edge device and combines the optimization results of the particle swarm algorithm to schedule and allocate tasks and coordinate the use of each edge device. This intelligent scheduling and allocation method can effectively improve the efficiency of the system and ensure that tasks can be processed on the most appropriate device, thereby optimizing the overall performance.

[0049] Furthermore, a frame rate buffer is introduced into the conveyor belt system. In the process of real-time control of the conveyor belt speed based on the video frame rate, the frame rate buffer is used to store redundant frame rate frames when the rate of the hyperspectral camera is faster than the rate of the conveyor belt, so as to balance the speed difference between the conveyor belt and the camera;

[0050] The filling and releasing strategies of the frame rate buffer include: filling new frames when the buffer is not full, and reducing the conveyor speed when the conveyor speed is faster than the processing speed of the hyperspectral camera to ensure clear and accurate data capture; setting the thresholds for buffer filling and releasing, adjusting and balancing the frame rate change rate, and ensuring system stability and reliability of data transmission.

[0051] Compared with the prior art, the multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection of the present invention has the following beneficial effects:

[0052] The present invention adopts intelligent spectral information enhancement ISIS-SR, which reduces the dimension of data by extracting sparse features in hyperspectral images while retaining key information, further optimizes the feature extraction process, improves the accuracy of classification and recognition, and realizes more rich feature information than in the original RGB image through more optimized features, thus solving the data sparsity problem.

[0053] The present invention uses a multi-source teacher distillation strategy, combines selective distillation and spectral difference distillation according to the characteristics of spectral data, to provide the student model with a more comprehensive and accurate learning goal, so that the model can flexibly adapt to the analysis tasks between multiple fruits or multiple types of the same fruit, avoiding the problem of the teacher model being too large in memory and difficult to deploy on edge devices, simplifying the model training process, and greatly improving the generalization and analysis capabilities of the model;

[0054] Third, the present invention innovatively utilizes wavelet transform WT and IHS transform to decompose high-resolution images through wavelets, and then superimposes the obtained wavelet surfaces onto the intensity I image after the multispectral image is transformed by IHS, so as to retain the spectral information of the multispectral image to the maximum extent, while improving the clarity and spatial resolution. The combination of the two solves the over-parameterization problem encountered by the convolutional neural network CNN when realizing a large receptive field;

[0055] Fourth, the present invention uses multi-edge device collaborative regulation and overall control to monitor the dynamic situation of each edge device, and combines the optimization results of the particle swarm algorithm to schedule and allocate tasks and coordinate the use of each edge device. This intelligent scheduling and allocation method can effectively improve the efficiency of the system and ensure that tasks can be processed on the most appropriate device, thereby optimizing the overall performance.

[0056] In summary, the present invention collects hyperspectral images and videos of fruits through integrated equipment, and sends them to various edge devices for preprocessing. The master control then coordinates the edge ends of different data types to split the distillation model, and quantifies and deploys it on the edge devices for detection. In this process, the master control allocates and adjusts different tasks according to the performance and usage of the edge devices. The present invention can accurately and efficiently analyze the disease quality status of fruits, and facilitates timely management and control of the health status of fruits. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0058] Figure 1 It is a structural block diagram of the multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection of the present invention;

[0059] Figure 2 It is a structural schematic diagram of the integrated equipment module in the source teacher distillation and entropy reduction distribution platform system of the present invention;

[0060] Figure 3 It is a logic block diagram of overall task scheduling in the multi-source teacher distillation and entropy reduction distribution platform system of the present invention;

[0061] Figure 4 It is a logical block diagram of the multi-source teacher distillation and entropy reduction distribution platform system of the present invention.

[0062] The reference numerals are as follows: 101, integrated equipment module; 1011, acquisition system; 1012, light source; 1013, conveyor belt system; 102, data processing module; 103, model deployment module; 104, disease detection module. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0064] At present, traditional disease detection methods mainly rely on manual experience, and have problems such as low efficiency, low accuracy, and susceptibility to subjective factors.

[0065] With the development of spectral analysis and deep learning, it has become possible to efficiently and accurately detect whether fruits are diseased or not; hyperspectral cameras can obtain chemical information inside the fruit, such as key indicators such as nutrients, moisture, and protein; deep learning in computer vision can obtain the physical conditions of the fruit surface, such as growth conditions, leaf color, plant morphology and other important information.

[0066] However, how to use the image information obtained by computer vision to detect fruit diseases and pests is also an important challenge in this field. The phenotyping method based on machine learning usually requires a large amount of data to support model training. When information collection is difficult and real-time data is scarce, the model analysis results often deviate.

[0067] To solve the above problems, the present invention provides a multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection. The distillation technology can combine data from different data sources, so that it can be more accurately integrated and reduce the memory occupied by the model, and it is more convenient to deploy to the edge device, and use the operating status and task allocation of the edge device to implement task coordination through reinforcement learning to obtain the maximum reasonable allocation of resources. The present invention collects hyperspectral images and videos of fruits through integrated equipment, and sends them to each edge device for preprocessing, and then coordinates the edge end of different data types through the master control to split the distillation model, and quantifies and deploys it on the edge device for detection. In this process, the master control allocates and adjusts different tasks according to the performance and usage of the edge device. The present invention can accurately and efficiently analyze the disease quality status of the fruit, and is convenient for timely control of the health status of the fruit.

[0068] The specific implementation of the multi-source teacher distillation and entropy reduction distribution platform system of the present invention is described in detail below in conjunction with specific embodiments.

[0069] like Figure 1 and 2 As shown, in one embodiment provided by the present invention, a multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection is provided, including the following modules:

[0070] The integrated device module 101 is used to collect image data of the target fruit using an integrated device, and the image data includes a hyperspectral image and a video; the integrated device includes a collection system 1011, a light source 1012 and a conveyor belt system 1013, and the collection system 1011 includes an image collection system 1011 and a hyperspectral data collection system 1012;

[0071] The conveyor belt system 1013 of the present invention introduces a frame rate buffer, and the conveyor belt speed is regulated in real time according to the requirements of the video frame rate;

[0072] In the embodiment of the present invention, an image acquisition system is used to acquire a video of a target fruit, and a hyperspectral data acquisition system is used to acquire a hyperspectral image. The hyperspectral data acquisition system includes a hyperspectral imager, and the hyperspectral imager acquires reflectance spectral images of the target fruit in a plurality of continuous narrow spectral bands; the hyperspectral imager includes a spectral calibration module, a data acquisition card, and a spectral preprocessing module, the spectral calibration module ensures data accuracy, the data acquisition card transmits data, and the spectral preprocessing module improves spectral data quality;

[0073] The wavelength range of the hyperspectral instrument selected in this example is 700nm to 9000nm;

[0074] Light source 1013 is used to adjust the light source intensity, direction and type during data collection;

[0075] The conveyor system 1014 includes a platform for placing fruits and a motor, and the platform has scale lines for accurately positioning the fruits;

[0076] Preferably, the light source is an adjustable light source, which adjusts the intensity and spectral distribution according to the characteristics of the detection object and the ambient light conditions, and the control system of the light source is used to manage the switch, intensity and spectral distribution;

[0077] Preferably, the conveyor belt system comprises a conveyor belt, which transports the target fruit to be detected, and the speed of the conveyor belt is adjusted according to the size and speed of the target fruit;

[0078] Preferably, the integrated device also has a built-in synchronous control system, which uses a Raspberry Pi to synchronously control the image acquisition system, the hyperspectral data acquisition system and the conveyor belt system, so that the collected data corresponds one-to-one with the position of the target fruit.

[0079] As a specific implementation method, when using a hyperspectral imager to collect data, pears, soybeans, and fruit leaves can be selected as target fruits to be detected and analyzed. The target fruit is placed on the stage, and the stage is first moved directly under the hyperspectral imager by calibrating the scale on the stage to ensure that the hyperspectral data collection area is the same as the image video shooting area.

[0080] For further information, please refer to Figure 1 , the present invention further comprises:

[0081] A data processing module 102 is used to perform intelligent spectral information enhancement on the hyperspectral image and video respectively;

[0082] In the data processing module 102, the collected multispectral data is subjected to ISIS-SR, i.e. intelligent spectral information enhancement. This technology aims to not only learn diverse context features through a spatial-spectral dual-branch feature extraction architecture, but also improve the spatial-spectral joint representation capability of the model and enhance the spectral information in the hyperspectral image to make up for the difficulties in data collection and the scarcity of real-time data.

[0083] The joint extraction of spatial-spectral features is expressed as:

[0084] ;

[0085] in, and are the extracted multi-scale spectral features and multi-scale spatial features respectively; m is a weight that can be optimized by a neural network; is the Sigmoid activation function, which maps the input weighted features to the [0,1] interval to indicate the importance of each feature; b is the bias term, which is used to adjust the bias of the fused features; Representing multi-scale spectral features The weight parameter of Representing multi-scale spatial features The weight parameter of

[0086] By analyzing the existing hyperspectral image data, intelligent spectral information enhancement technology can predict and supplement the missing spectral information and enhance the spectral resolution of the image. After processing, it is sent to different GPU terminals for preprocessing; as an example, the hyperspectral data of the target fruit is subjected to intelligent spectral information enhancement processing operations to predict and supplement the missing spectral information of the spectral data and enhance the spectral resolution of the image. The purpose is to make up for the difficulties in data collection and the scarcity of real-time data, and to improve the accuracy and consistency of spectral data;

[0087] Please continue to refer to Figure 1 , the present invention further comprises:

[0088] A model deployment module 103, configured to distill the model for the hyperspectral image data to obtain a model that can be deployed on the edge;

[0089] The disease detection module 104 is used to coordinate the edge splitting teacher model of different data types through the master control, obtain a sub-model obtained by splitting multiple edge ends, and use the sub-model to perform disease quality detection tasks on the fruits on the conveyor belt; wherein, in the process of performing the disease quality detection task, the particle swarm algorithm PSO is used as an optimization tool of the master control mechanism to allocate and coordinate the tasks of the edge end and perform disease detection on the target fruit.

[0090] Furthermore, for hyperspectral data, innovative wavelet transform WT and IHS transform are used to decompose high-resolution images through wavelets, and then the obtained wavelet surfaces are superimposed on the intensity I image after the multispectral image is transformed by IHS, so as to retain the spectral information of the multispectral image to the maximum extent, while improving the clarity and spatial resolution. The two are combined to solve the over-parameterization problem encountered by the convolutional neural network CNN when achieving a large receptive field. A multi-dimensional convolution network is used to extract spatial and spectral features using deeply separable 3D convolutions; multiple 1D convolutions are used in the network layer to extract spectral features of different scales, so as to capture the high-dimensional features of the hyperspectrum with as little resource consumption as possible; it should be noted that the model selected in this example is based on the characteristics of spectral data and lightweight requirements, and other models can be replaced according to personal needs;

[0091] Based on the selective distillation of spectral data, the present invention introduces spectral difference loss, taking into account the importance of bands and the difference between classes; wherein:

[0092] The selective distillation loss is expressed as:

[0093] ;

[0094] in, is the weight of the i-th band, calculated by the gradient method; KL represents the Kullback-Leibler divergence; softmax represents the activation function; T represents the coefficient for adjusting the probability distribution of the output of the softmax function; represents the output value of the student model for the i-th band; represents the actual value of the ith band;

[0095] The spectral difference loss is expressed as: ;

[0096] in, is the feature center of the student model in the i-th category, is the feature center of the teacher model in the i-th category; represents the square of L2 norm;

[0097] The classification loss is expressed as: ;

[0098] Where f(x) represents the model's predicted value for input x; y represents the actual value of input x; n represents the number of input samples;

[0099] The total loss is expressed as: ;

[0100] Among them, α, β, and γ represent the weights for balancing different losses.

[0101] Further, such as Figure 4 As shown, in the task of using the sub-model to detect the disease quality of fruits on the conveyor belt, the classification detection model based on the hyperspectral data is divided into three edge ends;

[0102] In the first edge NPU, the hyperspectral image is preprocessed, including denoising, correction, standardization and data enhancement; data enhancement includes rotation, flipping and cropping;

[0103] During the preprocessing stage, images will go through denoising, correction, and standardization to eliminate potential noise and ensure data consistency. Data enhancement will increase data diversity through various techniques such as rotation, flipping, and cropping, thereby improving the generalization ability of the model.

[0104] For hyperspectral data, wavelet transform WT and IHS transform are used to decompose high-resolution images through wavelet, and the obtained wavelet surfaces are superimposed on the image with intensity I after IHS transformation of multispectral images. The WTIHSConv model is constructed in combination with the convolutional network CNN, and the detection model based on hyperspectral data is divided into multiple edge ends.

[0105] Sub-models from different angles are applied to each edge to achieve collaboration of diverse sub-models, thereby solving the problem of data sparsity.

[0106] In the first edge NPU, B-spline wavelet is applied on the basis of WTIHSConv. This wavelet is a wavelet basis function based on the B-spline function: , where i is the basis function index, which is used to indicate the position in the B-spline basis function sequence; p represents the order of the B-spline, which is used to define the smoothness and local control ability of the B-spline basis function; , Represents the B-spline basis function, which has advantages in computational efficiency and local control smoothness, and is conducive to fast and accurate feature extraction;

[0107] In the process of feature extraction by the second edge NPU, the hyperspectral image is decomposed at multiple scales in addition to the WTIHSConv model, and multi-band feature fusion is used at each scale to extract features at the hyperspectral spatial level.

[0108] In the feature extraction process of the third edge NPU, the WTIHSConv lightweight model plus deep separable 3D convolution is used to extract spectral features;

[0109] The fused features are input into the pre-trained classification model in the fourth edge GPU for classification, and the classification head performs the classification task on the extracted features to output the final detection results.

[0110] This complete process plays a key role in hyperspectral data classification and detection. It integrates resources between different edge terminals and achieves accurate classification and recognition of complex hyperspectral image data through multiple stages of processing and feature extraction.

[0111] Further, such as Figure 3 As shown, the master control will reasonably allocate and coordinate tasks for each edge device according to their respective performance and current status and adaptability to the task according to the communication protocol. In this system, a particle swarm algorithm (PSO) is used as an optimization tool for the master control mechanism to achieve task allocation and coordination of edge devices; specifically, the particle swarm algorithm PSO provided by the present invention is used as an optimization tool for the master control mechanism to allocate and coordinate tasks on the edge end, and the particle swarm algorithm is used to allocate tasks according to the performance and current status of each edge device and the importance of the task;

[0112] In the particle swarm algorithm, each particle represents a potential solution, and the dimension of each particle is taken as the number of tasks; for the task allocation problem, each solution is a possible task allocation scheme, in which each task is assigned to a specific edge device;

[0113] Initialization phase: Randomly initialize the position and velocity of each particle and assign an initial edge device to each task;

[0114] Fitness evaluation: Evaluate the position of each particle and calculate its fitness. The fitness function is defined based on the performance, current state, and importance of the task of the edge device. A high fitness indicates a better task allocation solution.

[0115] Update speed and position: According to the update formula of the particle swarm algorithm, the speed and position of each particle are updated according to the current position and speed as well as the historical best position and the global best position, where:

[0116] The update formula of the particle swarm algorithm is as follows:

[0117]

[0118] ;

[0119] in, is the velocity of the ith particle in the tth generation, is the position of the ith particle in the tth generation, is the best historical position of the particle, is the global optimal position, w is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers in the interval [0,1] respectively;

[0120] Update the historical best position and the global best position: compare the fitness of the current particle with its historical best fitness, update the historical best position, then compare the historical best position of each particle with the global best fitness, and update the global best position;

[0121] Iterative optimization: Repeat the process of updating speed and position until the algorithm converges or reaches the preset stop condition. The preset stop condition is that the maximum number of iterations is reached or the fitness meets the requirements;

[0122] Final task allocation: Finally, the task allocation scheme is determined based on the global optimal position, and tasks are allocated to each edge device to achieve the best system performance and task completion efficiency;

[0123] Therefore, the master control of the present invention monitors the dynamic situation of each edge device and combines the optimization results of the particle swarm algorithm to schedule and allocate tasks and coordinate the use of each edge device. This intelligent scheduling and allocation method can effectively improve the efficiency of the system and ensure that tasks can be processed on the most appropriate device, thereby achieving overall performance optimization.

[0124] Furthermore, a frame rate buffer is introduced into the conveyor belt system. In the process of real-time control of the conveyor belt speed based on the video frame rate, the frame rate buffer is used to store redundant frame rate frames when the rate of the hyperspectral camera is faster than the rate of the conveyor belt, so as to balance the speed difference between the conveyor belt and the camera;

[0125] The filling and releasing strategies of the frame rate buffer include: filling new frames when the buffer is not full, and reducing the conveyor speed when the conveyor speed is faster than the processing speed of the hyperspectral camera to ensure clear and accurate data capture; setting the thresholds for buffer filling and releasing, adjusting and balancing the frame rate change rate, and ensuring system stability and reliability of data transmission.

[0126] The dynamic adjustment of the video frame rate of the present invention can balance the image quality and performance according to the real-time demand and the system load; in the system, the dynamic adjustment of the video frame rate can balance the image quality and performance according to the real-time demand and the system load; therefore, the present invention introduces a moderately sized frame rate buffer, the role of which is to store excess frame rate frames when the rate of the hyperspectral camera is faster than the rate of the conveyor belt, so as to balance the speed difference between the conveyor belt and the camera. The filling and release strategy is also crucial, that is, when the buffer is not full, new frames are filled, and when the conveyor belt speed is faster than the processing speed of the hyperspectral camera, the conveyor belt speed is reduced to ensure that the data is captured clearly and accurately. At the same time, the control mechanism plays a key role, and the thresholds for buffer filling and release need to be set to avoid buffer overflow or data loss, thereby achieving the adjustment and balance of the frame rate change rate, and ensuring the system stability and reliability of data transmission. Such a buffer mechanism design can effectively cope with the differences between different frame rate generation and processing speeds, improve the overall performance of the system, user experience and system stability.

[0127] The present invention adopts intelligent spectral information enhancement (ISIS-SR) to reduce the dimension of data by extracting sparse features in hyperspectral images while retaining key information, further optimizing the feature extraction process, improving the accuracy of classification and recognition, and achieving more rich feature information than in the original RGB image through more optimized features to solve data sparsity;

[0128] The present invention uses a multi-source teacher distillation strategy, combines selective distillation and spectral difference distillation according to the characteristics of spectral data, to provide the student model with a more comprehensive and accurate learning goal, so that the model can flexibly adapt to the analysis tasks between multiple fruits or multiple types of the same fruit, avoiding the problem of the teacher model being too large in memory and difficult to deploy on edge devices, simplifying the model training process, and greatly improving the generalization and analysis capabilities of the model;

[0129] The present invention adopts multi-edge device collaborative regulation and overall control to monitor the dynamic situation of each edge device, and combines the optimization results of the particle swarm algorithm to schedule and allocate tasks and coordinate the use of each edge device. This intelligent scheduling and allocation method can effectively improve the efficiency of the system and ensure that tasks can be processed on the most appropriate device, thereby achieving overall performance optimization.

[0130] In summary, the present invention collects hyperspectral images and videos of fruits through integrated equipment, and sends them to various edge devices for preprocessing. The master control then coordinates the edge ends of different data types to split the distillation model, and quantifies and deploys it on the edge devices for detection. In this process, the master control allocates and adjusts different tasks according to the performance and usage of the edge devices. This method can accurately and efficiently analyze the disease quality status of the fruit, and facilitates timely management and control of the health status of the fruit.

[0131] The above solutions are only an illustration of a preferred example, but are not limited thereto. When implementing the present invention, appropriate replacement and / or modification can be performed according to user needs.

[0132] The number of devices and processing scales described here are used to simplify the description of the present invention. Applications, modifications and variations of the present invention will be obvious to those skilled in the art.

[0133] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection, characterized in that: Includes the following modules: An integrated device module is used to collect image data of target fruits using an integrated device, wherein the image data includes hyperspectral images and videos; the integrated device includes an acquisition system, a light source, and a conveyor belt system, wherein the acquisition system includes an image acquisition system and a hyperspectral data acquisition system; wherein a frame rate buffer is introduced into the conveyor belt system to adjust the conveyor belt speed in real time based on the video frame rate; A data processing module, used for performing intelligent spectral information enhancement processing on the hyperspectral image and video respectively; The model deployment module is used to perform multi-source collaborative model distillation on hyperspectral image data to obtain a model that can be deployed on the edge; The disease detection module is used to coordinate the edge-end splitting teacher model of different data types through the master control to obtain multiple sub-models for the edge end, and use the sub-models to perform disease quality detection on the fruits on the conveyor belt; wherein, in the process of disease quality detection, the particle swarm algorithm PSO is used as an optimization tool of the master control mechanism to allocate and coordinate the tasks of the edge end and perform disease detection on the target fruit.

2. The multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection according to claim 1 is characterized in that: In the integrated device, the hyperspectral data acquisition system includes a hyperspectral imager, which collects reflectance spectrum images of target fruits in multiple continuous narrow spectral bands; the hyperspectral imager includes a spectral calibration module, a data acquisition card and a spectral preprocessing module; The light source is an adjustable light source, which can adjust the intensity and spectral distribution according to the characteristics of the detection object and the ambient light conditions; The conveyor belt system comprises a conveyor belt, and the conveyor belt is used to transport the target fruit to be detected; The integrated device has a built-in synchronous control system, which uses a Raspberry Pi to synchronously control the image acquisition system, the hyperspectral data acquisition system, and the conveyor belt system, so that the collected data corresponds one-to-one with the position of the target fruit.

3. The multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection according to claim 2 is characterized in that: In the data processing module, the collected multispectral data is subjected to intelligent spectral information enhancement processing through the spatial-spectral dual-branch feature extraction architecture, where the joint extraction of spatial-spectral features is expressed as: ； in, and are the extracted multi-scale spectral features and multi-scale spatial features respectively; m is a weight that can be optimized by a neural network; is the Sigmoid activation function, which is used to map the input weighted features to the [0,1] interval; b represents the bias term, which is used to adjust the bias of the fusion feature; Representing multi-scale spectral features The weight parameter of Representing multi-scale spatial features The weight parameter of After processing, the multispectral data is sent to different GPU terminals for preprocessing.

4. The multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection according to claim 3 is characterized in that: For hyperspectral data, a multi-dimensional convolutional network is used to extract spatial and spectral features using deeply separable 3D convolutions, and multiple 1D convolutions are used in the network layer to extract spectral features of different scales. Based on the selective distillation of spectral data, spectral difference loss is introduced; The selective distillation loss is expressed as: ； in, is the weight of the i-th band, calculated by the gradient method; KL represents the Kullback-Leibler divergence; softmax represents the activation function; T represents the coefficient for adjusting the probability distribution of the output of the softmax function; represents the output value of the student model for the i-th band; represents the actual value of the ith band; The spectral difference loss is expressed as: ; in, is the feature center of the student model in the i-th category, is the feature center of the teacher model in the i-th category; represents the square of L2 norm; The classification loss is expressed as: ; Where f(x) represents the model's predicted value for input x; y represents the actual value of input x; n represents the number of input samples; The total loss is expressed as: ; Among them, α, β, and γ represent the weights for balancing different losses.

5. The multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection according to claim 4 is characterized in that: In the disease quality detection of fruits on the conveyor belt using the sub-model, for hyperspectral data, wavelet transform WT and IHS transform are used to decompose the high-resolution image through wavelet, and each obtained wavelet surface is superimposed on the image with intensity I after the multispectral image is transformed by IHS. The WTIHSConv model is constructed in combination with the convolutional network CNN, and the detection model based on hyperspectral data is split into sub-models and applied to multiple edge ends, and sub-models at different angles are applied to each edge end; wherein: In the first edge NPU, B-spline wavelet is applied to perform data enhancement processing on hyperspectral data based on WTIHSConv. B-spline wavelet is a wavelet basis function based on B-spline function, which is expressed as: , where i is the basis function index, which is used to indicate the position in the B-spline basis function sequence; p represents the order of the B-spline, which is used to define the smoothness and local control ability of the B-spline basis function; , represents the B-spline basis function; In the feature extraction process of the second edge NPU, the hyperspectral image is decomposed at multiple scales based on the WTIHSConv model, and multi-band feature fusion is used at each scale to extract the features at the hyperspectral spatial level. In the feature extraction process of the third edge NPU, the WTIHSConv lightweight model plus deep separable 3D convolution is used to extract spectral features; The fused features are input into the fourth edge GPU, and the pre-trained classification model is used to perform feature classification. The classification head is used to perform classification tasks on the extracted features and output the final detection results.

6. The multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection according to claim 5 is characterized in that: Using the particle swarm algorithm (PSO) as an optimization tool for the master control mechanism, in the process of allocating and coordinating tasks on the edge, the particle swarm algorithm is used to allocate tasks based on the performance and current status of each edge device and the importance of the task; In the particle swarm algorithm, each particle represents a potential solution, and the dimension of each particle is taken as the number of tasks; for the task allocation problem, each solution is a possible task allocation scheme, in which each task is assigned to a specific edge device; Initialization phase: Randomly initialize the position and velocity of each particle and assign an initial edge device to each task; Fitness evaluation: Evaluate the position of each particle and calculate its fitness; the fitness function is defined based on the performance, current state, and importance of the task of the edge device; Update speed and position: According to the update formula of the particle swarm algorithm, the speed and position of each particle are updated according to the current position and speed as well as the historical best position and the global best position, where: The update formula of the particle swarm algorithm is as follows: ； ； in, is the velocity of the ith particle in the tth generation, is the position of the ith particle in the tth generation, is the best historical position of the particle, is the global optimal position, w is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers in the interval [0,1] respectively; Update the historical best position and the global best position: compare the fitness of the current particle with its historical best fitness, update the historical best position, then compare the historical best position of each particle with the global best fitness, and update the global best position; Iterative optimization: Repeat the process of updating speed and position until the algorithm converges or reaches the preset stop condition. The preset stop condition is that the maximum number of iterations is reached or the fitness meets the requirements; Final task allocation: Finally, the task allocation scheme is determined based on the global optimal position, and the tasks are allocated to each edge device.

7. The multi-source teacher distillation and entropy reduction distribution platform system for fruit disease detection according to claim 6 is characterized in that: A frame rate buffer is introduced into the conveyor belt system. In the process of real-time control of the conveyor belt speed based on the video frame rate, the frame rate buffer is used to store redundant frame rate frames when the rate of the hyperspectral camera is faster than the rate of the conveyor belt, so as to balance the speed difference between the conveyor belt and the camera; The filling and releasing strategies of the frame rate buffer include: filling new frames when the buffer is not full, reducing the conveyor belt rate when the conveyor belt speed is faster than the processing speed of the hyperspectral camera; setting the thresholds for buffer filling and releasing, and regulating and balancing the frame rate change rate.

Images