Intelligent tea sorting equipment control system and method

Abstract

The invention relates to the technical field of intelligent tea leaf sorting, and particularly discloses an intelligent tea leaf sorting equipment control system and method.The intelligent tea leaf sorting equipment control system comprises the steps that firstly, the speed value of a color sorter and a monitoring video of to-be-detected tea leaves within a preset time period are obtained, and data processing and feature extraction are conducted; the method comprises the following steps: obtaining a tea color feature vector and a speed feature vector, then constructing a color-speed associated feature vector by fusing the feature vectors, and determining whether the speed of a color sorter should be increased or decreased by using a classification module, so that the quality and the processing efficiency of tea are improved in an automatic sorting process; the influence of human cost and tea residue mixing on the tea quality is reduced, the continuously improved tea quality requirement can be met, and the development of the tea industry is promoted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent tea sorting, and more particularly, to a tea intelligent sorting device control system and method. BACKGROUND

[0002] Tea originated in China, contains catechin, cholestenone, caffeine, inositol, folic acid, pantothenic acid and other components, can improve human health, and is known as "one of the world's three major beverages". Tea has the characteristics of naturalness, healthiness, convenience and the like, and is deeply loved by the general public. With the continuous improvement of people's living standards, people at different life levels have different requirements for the quality of tea, and more and more people have higher and higher requirements for tea.

[0003] In the processing of tea, sorting is particularly important. Through sorting, impurities in tea can be removed, so that the same processing batch of tea is basically the same, which can significantly improve the quality of tea, and is also conducive to the control of the frying time and temperature in the tea processing process, and is conducive to improving the quality of tea. The existing tea sorting relies on manual work, which has low work efficiency and requires a large amount of manpower. On the other hand, the sorting process is time-consuming and the tea residue is mixed, which affects the quality of tea.

[0004] Therefore, a tea intelligent sorting device control system and method are expected. SUMMARY

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a tea intelligent sorting device control system and method, which first obtains the color sorter speed value in a predetermined time period and the monitoring video of the tea to be detected in the predetermined time period, and performs data processing and feature extraction to obtain the color feature vector and the speed feature vector of the tea, then constructs a color-speed correlation feature vector by fusing these feature vectors, and uses a classification module to determine whether the speed of the color sorter should be increased or decreased, so that in the automatic sorting process, the quality and processing efficiency of the tea are improved, the influence of the human cost and the tea residue mixing on the quality of the tea is reduced, which helps to meet the increasing requirements for the quality of tea and promote the development of the tea industry.

[0006] According to one aspect of the present application, a tea intelligent sorting device control system is provided, which comprises:

[0007] a data acquisition module, configured to acquire color sorter speed values at a plurality of predetermined time points in a predetermined time period and monitoring video of tea to be detected in the predetermined time period;

[0008] The data processing module is used to extract features from the speed values ​​of the color sorter at the multiple predetermined time points and the monitoring video of the tea to be tested during the predetermined time period to obtain the tea color feature vector and speed feature vector.

[0009] The fusion module is used to fuse the tea color feature vector and the speed feature vector to obtain a color-speed associated feature vector;

[0010] The classification result module is used to determine whether the speed value of the color sorter should be increased or decreased based on the color-speed correlation feature vector.

[0011] According to another aspect of this application, a control method for an intelligent tea sorting device is also provided, comprising:

[0012] Obtain the speed values ​​of the color sorter at multiple predetermined time points within a predetermined time period, as well as the monitoring video of the tea leaves to be tested during the predetermined time period;

[0013] Feature extraction is performed on the speed values ​​of the color sorter at the multiple predetermined time points and the monitoring video of the tea to be tested during the predetermined time period to obtain the tea color feature vector and speed feature vector;

[0014] The tea color feature vector and the velocity feature vector are fused to obtain a color-velocity related feature vector;

[0015] Based on the color-speed correlation feature vector, the speed value of the color sorter should be increased or decreased.

[0016] Compared with existing technologies, the intelligent tea sorting equipment control system and method provided in this application first acquires the speed value of the color sorter and the monitoring video of the tea to be tested within a predetermined time period, and performs data processing and feature extraction to obtain the tea color feature vector and speed feature vector. Then, by fusing these feature vectors, a color-speed correlation feature vector is constructed, and a classification module is used to determine whether the speed of the color sorter should be increased or decreased. In this automated sorting process, the quality and processing efficiency of tea are improved, the labor costs and the impact of tea residue mixing on tea quality are reduced, and the increasing requirements for tea quality are met, thus promoting the development of the tea industry. Attached Figure Description

[0017] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a block diagram of the control system for an intelligent tea sorting device according to an embodiment of this application.

[0019] Figure 2 This is a block diagram of the data processing module in the intelligent tea sorting equipment control system according to an embodiment of this application.

[0020] Figure 3 This is a block diagram of the monitoring video processing unit in the intelligent tea sorting equipment control system according to an embodiment of this application.

[0021] Figure 4 This is a schematic diagram of the architecture of a tea intelligent sorting equipment control system according to an embodiment of this application.

[0022] Figure 5 This is a flowchart of a control method for a smart tea sorting device according to an embodiment of this application. Detailed Implementation

[0023] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0024] Exemplary System

[0025] Figure 1 The diagram illustrates a block diagram of a tea intelligent sorting equipment control system according to an embodiment of this application. Figure 1 As shown, the intelligent tea sorting equipment control system 100 according to an embodiment of this application includes: a data acquisition module 110, used to acquire the speed values ​​of a color sorter at multiple predetermined time points within a predetermined time period and the monitoring video of the tea to be tested during the predetermined time period; a data processing module 120, used to extract features from the speed values ​​of the color sorter at the multiple predetermined time points and the monitoring video of the tea to be tested during the predetermined time period to obtain a tea color feature vector and a speed feature vector; a fusion module 130, used to fuse the tea color feature vector and the speed feature vector to obtain a color-speed correlation feature vector; and a classification result module 140, used to determine whether the speed value of the color sorter should be increased or decreased based on the color-speed correlation feature vector.

[0026] In this embodiment, the data acquisition module 110 is used to acquire the speed values ​​of the color sorter at multiple predetermined time points within a predetermined time period, as well as monitoring videos of the tea leaves to be inspected during the predetermined time period. It should be understood that by acquiring the speed values ​​of the color sorter, the operating speed of the color sorter at different time points can be understood, which is an important parameter in the tea sorting process. Simultaneously, the monitoring videos of the tea leaves provide visual information about the tea leaves, including their color, shape, and other features. These data can be processed using feature extraction algorithms to extract tea color feature vectors and speed feature vectors. The tea color feature vector reflects the color characteristics of the tea leaves and can be used to determine the quality and grade of the tea. The speed feature vector reflects the operating speed of the color sorter and can be used to control the working state of the color sorter. By fusing these feature vectors, a color-speed correlation feature vector can be constructed to determine the direction of speed adjustment for the color sorter, i.e., increasing or decreasing the speed, to achieve better tea sorting results. Therefore, acquiring the speed values ​​of the color sorter at multiple predetermined time points within a predetermined time period and the monitoring videos of the tea leaves to be inspected is for feature extraction and intelligent sorting to improve tea quality and processing efficiency. Specifically, multiple time points are selected within a predetermined time period to obtain the color sorter speed values ​​and monitoring videos. At each predetermined time point, the color sorter speed value is recorded. This can be obtained through the color sorter's sensors or monitoring system. At each predetermined time point, monitoring videos of the tea leaves are captured using a monitoring camera or system, ensuring the tea leaves are clearly visible in the video. The obtained color sorter speed values ​​and corresponding monitoring videos are stored and organized for subsequent processing and analysis.

[0027] In this embodiment, the data processing module 120 is used to extract features from the speed values ​​of the color sorter at multiple predetermined time points and the monitoring video of the tea to be tested during the predetermined time period to obtain a tea color feature vector and a speed feature vector. It should be understood that tea color is one of the important indicators of tea quality and grade. By analyzing the tea color information in the monitoring video, color features such as brightness, hue, and saturation can be extracted. Feature extraction algorithms can process the tea images in the monitoring video, for example, using image processing methods to extract color histograms, color space transformation, etc., to convert the tea color information into numerical features, forming a tea color feature vector. The speed of the color sorter is an important parameter controlling the tea sorting process. By recording the speed values ​​of the color sorter at multiple predetermined time points and performing feature extraction, a speed feature vector of the color sorter can be obtained. The speed feature vector can include indicators such as the average speed, maximum speed, minimum speed, and speed change trend of the color sorter. These features can be used to analyze the operating status of the color sorter and the effect of tea sorting. Therefore, by combining the tea color feature vector and the speed feature vector, a color-speed correlated feature vector can be constructed. By analyzing and comparing these feature vectors, the direction of speed adjustment for the color sorter during tea sorting can be determined to optimize the sorting effect. Feature extraction is a crucial step in transforming raw data into information representations that can be used for analysis and control, providing a foundation for subsequent intelligent sorting algorithms and control strategies. Specifically, extracting the color features of tea from monitoring videos requires image processing and analysis. Computer vision techniques, such as color space transformation, color histograms, and color distribution methods, can be used to transform tea color information into numerical features. These features can represent the brightness, hue, saturation, and other characteristics of tea color, forming a tea color feature vector. Simultaneously, extracting speed features from the color sorter speed values ​​at multiple predetermined time points requires numerical analysis. The average, maximum, and minimum speed values ​​can be calculated, or the speed change trend can be calculated using difference methods. These features can be used to describe the operating speed characteristics of the color sorter, forming a speed feature vector. Therefore, through feature extraction, we can extract useful information from the raw data and transform it into feature vectors that can be used for analysis and control. These feature vectors can be used in subsequent intelligent tea sorting algorithms and control strategies to improve the sorting effect and quality of tea.

[0028] In one embodiment of this application, Figure 2 The diagram illustrates a block diagram of the data processing module in the control system of an intelligent tea sorting device according to an embodiment of this application. Figure 2As shown, in the above-mentioned intelligent tea sorting equipment control system 100, the data processing module 120 includes: a monitoring video processing unit 121, used to perform convolutional encoding on the monitoring video of the tea to be tested during the predetermined time period to obtain the tea color feature vector; and a speed processing unit 122, used to obtain the speed feature vector by passing the speed values ​​of the color sorter at the multiple predetermined time points through a speed extractor.

[0029] further, Figure 3 The diagram illustrates a block diagram of the monitoring video processing unit in the control system of an intelligent tea sorting device according to an embodiment of this application. Figure 3 As shown, in the data processing module 120 of the above-mentioned intelligent tea sorting equipment control system 100, the monitoring video processing unit 121 includes: a key frame extraction subunit 1211, used to extract key frames of multiple tea areas from the monitoring video of the tea to be detected in the predetermined time period; and a color feature extraction subunit 1212, used to extract colors from the key frames of the multiple tea areas to obtain tea color feature vectors.

[0030] Accordingly, in a specific example of this application, the keyframe extraction subunit 1211 includes: extracting keyframes of multiple tea-growing regions from a monitoring video of tea leaves to be detected within a predetermined time period at a predetermined sampling frequency. It should be understood that the predetermined sampling frequency controls the number and frequency of keyframes extracted from the monitoring video. This balances the number of extracted frames and the consumption of computational resources, ensuring reasonable sampling of keyframes for the tea-growing regions. The tea-growing regions in the monitoring video may change due to the movement of the tea leaves or other factors. By using a predetermined sampling frequency, the changes in the tea-growing regions at different points in time can be captured for analysis and comparison. Keyframes are representative frame images in a video sequence that effectively summarize the video content. By extracting keyframes of multiple tea-growing regions at a predetermined sampling frequency, representative images of the tea-growing regions at different points in time can be obtained for subsequent feature extraction and analysis. Simultaneously, video sequences typically contain a large amount of redundant information, and consecutive frame images may be visually very similar. By extracting keyframes at a predetermined sampling frequency, redundant information can be reduced, and the most informative frame images in the video sequence can be extracted, thereby reducing the computational load of subsequent processing. Therefore, extracting keyframes of multiple tea-growing regions from a monitoring video of tea leaves within a predetermined time period at a predetermined sampling frequency can effectively control the number and frequency of extraction, capture changes in tea-growing regions, obtain representative images, and reduce redundant information. This will facilitate subsequent tea color feature extraction and analysis. Specifically, first, determine the sampling frequency for extracting keyframes from the monitoring video. This can be the number of frames extracted per second or per minute, set according to specific needs. Then, determine the starting time point in the monitoring video for keyframe extraction. This can be the start time of the predetermined time period, or other time points set according to specific needs. Further, use image processing and computer vision techniques to process each frame of the monitoring video to locate the tea-growing regions. Methods such as object detection, image segmentation, or feature extraction can be used to identify and locate tea-growing regions. Then, according to the set sampling frequency, select a keyframe from the located tea-growing region within each time interval. Several keyframe extraction methods can be used, such as selecting the most representative frame or frame selection based on image quality assessment. Next, save the extracted keyframes and record the time points where the keyframes are located. This allows associating keyframes with corresponding time points, facilitating subsequent analysis and comparison. The time points are updated according to the set sampling frequency to extract keyframes for the next time interval. This can be achieved by increasing a fixed time interval or updating based on the video's frame rate. Finally, the keyframe extraction process is complete when the predetermined time period ends or the required number of keyframes are extracted.

[0031] Specifically, in another specific example of this application, the speed values ​​of the color sorter at multiple predetermined time points are processed by a speed extractor to obtain a speed feature vector. It should be understood that by extracting keyframes from multiple tea-growing regions, color information of tea leaves under different positions, angles, and lighting conditions can be obtained. This provides more samples to represent the color features of tea leaves, increasing feature diversity. However, the color of tea leaves may vary in different regions. By extracting keyframes from multiple tea-growing regions, the overall color features of the tea leaves can be comprehensively considered. This helps to obtain the overall color distribution and features of the tea leaves. Keyframes are usually representative and stable frames in a video. Selecting keyframes for color extraction can reduce interference from motion blur or changes that may exist in the video. This helps to extract more accurate and stable tea leaf color features. At the same time, extracting keyframes for color feature extraction can reduce computational costs and time consumption. Compared to performing color extraction on every frame of the video, extracting only keyframes can speed up processing while preserving important color information.

[0032] Specifically, the color feature extraction subunit 1212 includes: a feature map extraction secondary subunit, used to extract color feature maps from keyframes of the multiple tea leaf regions; a first convolutional coding secondary subunit, used to pass the color feature maps through a color extractor based on a first convolutional neural network model to obtain tea leaf color feature maps; and a dimensionality reduction secondary subunit, used to perform global mean pooling on each feature matrix along the channel dimension of the tea leaf color feature maps to obtain tea leaf color feature vectors.

[0033] Accordingly, the feature map extraction sub-unit is used to extract color feature maps from keyframes of the multiple tea leaf regions. It should be understood that different tea leaf regions may have different color characteristics. Extracting color feature maps from multiple tea leaf regions can capture the color changes of tea leaves at different locations and regions. This helps to more accurately describe the color distribution and characteristics of tea leaves. By extracting multiple color feature maps, multiple color representations of tea leaf regions can be obtained. Different color feature maps can represent different color channels, color spaces, or color statistics of tea leaves. This provides a richer description of color features and helps to improve the expressive power of tea leaf color features. The color of tea leaves may vary at different regions and scales. By extracting color feature maps from multiple tea leaf regions, the overall color characteristics of tea leaves can be comprehensively considered, and the spatial distribution information of tea leaves can be captured. This helps to describe the color features of tea leaves more comprehensively. Keyframes of tea leaf regions are usually representative and stable frames in a video. By extracting color feature maps from keyframes of multiple tea leaf regions, tea leaf color features at different time points and in different regions can be obtained. This helps to increase the diversity of tea leaf color features and improve the robustness and reliability of the features.

[0034] Accordingly, the first convolutional coding secondary subunit is used to pass the color feature map through a color extractor based on a first convolutional neural network model to obtain a tea color feature map. It should be understood that a convolutional neural network is a powerful feature extractor that can learn and extract high-level feature representations from input images. By using the color extractor of the first convolutional neural network model, its learned color feature extraction capabilities can be utilized to extract more informative tea color features from multiple color feature maps. The activation function and nonlinear layers in the convolutional neural network model can perform nonlinear mapping on the input data, thereby capturing more complex color features. Through the color extractor of the first convolutional neural network model, multiple color feature maps can be nonlinearly mapped, improving the expressive power of tea color features. By using the color extractor of the first convolutional neural network model, multiple color feature maps can be fused. During training, the convolutional neural network model can learn the weights and correlations of features. By inputting multiple color feature maps into the network, the network can automatically learn the relationships between color features and generate a more comprehensive and consistent tea color feature map. Therefore, by using a color extractor based on a first convolutional neural network model, the automatic learning capabilities of deep learning can be leveraged to learn representations of tea color features from large amounts of data. This automatic learning capability can improve the accuracy and robustness of tea color features, making them more suitable for tea analysis and recognition tasks.

[0035] Specifically, firstly, multiple color feature maps C1, C2, ..., C are defined. n Initialize the color extractor of the convolutional neural network model, including convolutional layers, pooling layers, activation functions, etc. Then, for each color feature map C... i Processing: First, C i The input to the color extractor of the convolutional neural network model includes: C i The input is passed to the first convolutional layer. The convolutional layer performs a convolution operation on the input, generating a set of convolutional feature maps. These convolutional feature maps are then passed as input to the next layer. After forward propagation, the output feature map F is calculated. i The process includes: after passing through convolutional layers, the feature maps are passed to pooling layers; the pooling layers downsample the feature maps, reducing their size; pooling extracts the main features of the image while reducing computation; after pooling, the downsampled feature maps are obtained; the downsampled feature maps are then passed to activation functions for nonlinear transformation; activation functions introduce nonlinear relationships, increasing the model's expressive power; and after the activation functions, the final output feature map F is obtained. iTherefore, multiple color feature maps can be processed by a color extractor based on a first convolutional neural network model to obtain multiple tea color feature maps. These tea color feature maps can be used for further tea analysis, classification, and recognition tasks, providing more useful information to understand and characterize the color features of tea.

[0036] Accordingly, the second-level dimensionality reduction subunit is used to perform global mean pooling on each feature matrix along the channel dimension of the tea color feature map to obtain a tea color feature vector. It should be understood that through global mean pooling, each feature matrix can be averaged along the channel dimension to obtain a single numerical value representing the average value of that feature matrix. This average value can be seen as the intensity or weight of the color feature represented by that feature matrix. By performing global mean pooling on all feature matrices, a tea color feature vector can be obtained, where each element represents the average value of a feature matrix. This feature vector can capture the overall distribution and characteristics of tea color. Specifically, multiple tea color feature maps F1, F2, ..., F... are defined. n For each feature map F i Perform global mean pooling, including: pooling the feature map F along the channel dimension. i Perform an averaging operation to obtain a single numerical value representing the average value of the feature map. Repeat this operation for each feature map F. i Perform global mean pooling to obtain the corresponding feature values ​​v i Finally, the output is the tea color feature vector V = [v1, v2, ..., v]. n ], where each element v i Represents the corresponding feature map F i The average value of each feature map is calculated using global mean pooling. Therefore, global mean pooling transforms the color features of each feature map into a single numerical value, representing the average value of that feature map. These average values ​​form the tea color feature vector, where each element represents the average value of a feature map. The tea color feature vector can be used to represent the overall characteristics and distribution of tea color. Global mean pooling is calculated by averaging each feature matrix along the channel dimension to obtain a single numerical value. This can be achieved by calculating the average value of the feature matrices.

[0037] Specifically, the speed processing unit 122 is used to: arrange the speed values ​​of the color sorter at multiple predetermined time points into a speed input vector according to the time dimension, and then obtain a speed feature vector through a speed extractor based on a multi-scale neighborhood feature extraction module. It should be understood that the speed values ​​of the color sorter change over time; therefore, arranging these speed values ​​into a speed input vector according to the time dimension preserves the sequential information of the time series. This allows for better capture of the dynamic changes and trends in speed. The speed extractor based on the multi-scale neighborhood feature extraction module can extract features with different time scales from the speed input vector. This method can capture local changes in speed, overall trends, and possible periodic changes. By extracting multi-scale features, more comprehensive and richer speed information can be obtained. The speed feature vector can be used to represent and describe the operating status and performance of the color sorter. These features may include average speed, maximum speed, speed change rate, etc., from which the stability, operating efficiency, and possible anomalies of the color sorter can be inferred. Therefore, by arranging the speed values ​​of the color sorter into a speed input vector according to the time dimension, and using a speed extractor based on multi-scale neighborhood feature extraction, meaningful features can be extracted from the time series to better understand and analyze the speed changes of the color sorter.

[0038] Accordingly, the velocity extractor based on the multi-scale neighborhood feature extraction module is a module or algorithm for extracting velocity features from time-series data. The main function of the velocity extractor is to transform the velocity data in the time series into feature vectors with representative and useful information. These feature vectors can be used to describe and analyze the operating status, performance, and other relevant characteristics of the color sorter. Specifically, an appropriate scale range or scale level is selected based on the characteristics of the velocity data and task requirements. For each scale, a multi-scale neighborhood feature extraction method is used to extract features from the velocity data. These methods can include sliding window, wavelet transform, Fourier transform, etc., to capture velocity change information at different scales. Finally, the velocity features extracted from different scales are fused, which can be achieved through simple concatenation, weighted summation, or other fusion strategies.

[0039] It is worth mentioning that, in other specific examples of this application, the speed values ​​of the color sorters at multiple predetermined time points can also be processed by a speed extractor to obtain speed feature vectors in other ways. For example, the speed values ​​of the color sorters at multiple predetermined time points can be arranged into a speed input sequence according to the time dimension. The speed sequence can be represented as a two-dimensional matrix, where time is one dimension and speed value is another dimension. A convolutional neural network structure is designed, including convolutional layers, pooling layers, and fully connected layers. The convolutional layers are used to extract local features in the speed sequence, the pooling layers are used to reduce the feature dimension, and the fully connected layers are used to generate the final speed feature vector. The speed input sequence is input into the convolutional neural network, and a feature map is obtained through forward propagation. The feature map represents the speed features at different scales in the input sequence. The feature map is converted into a speed feature vector. The overall speed features can be obtained by performing average pooling or global pooling on the feature map. The final speed feature vector represents the feature information of the input speed sequence and can be used for subsequent analysis and application. The advantage of using a speed extractor based on a convolutional neural network model is that it can automatically learn and extract local features in the speed sequence without the need for manually designing feature extraction methods. Meanwhile, convolutional neural networks possess excellent scalability and adaptability, enabling them to handle velocity sequence data of varying lengths and resolutions. Of course, velocity extractors based on multi-scale neighborhood feature extraction modules are suitable for scenarios requiring attention to the changing characteristics of velocity data across different time scales. Through multi-scale neighborhood feature extraction, local variations and overall trends in velocity data at different time scales can be captured, yielding more comprehensive velocity feature information. This method is more suitable for tasks requiring time series analysis and pattern recognition of velocity data.

[0040] Specifically, the speed values ​​of the color sorters at multiple predetermined time points are arranged into a speed input vector according to the time dimension, and then a speed feature vector is obtained by a speed extractor based on a multi-scale neighborhood feature extraction module. This includes: a first convolutional layer, a second convolutional layer parallel to the first convolutional layer, and a cascaded layer connected to the first and second convolutional layers. The first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional layer uses a one-dimensional convolutional kernel with a second scale. First, the first convolutional layer of the multi-scale neighborhood feature extraction module performs one-dimensional convolutional encoding on the speed input vector using the following first convolution formula to obtain the first-scale feature vector; wherein the first convolution formula is:

[0041]

[0042] Where a is the width of the first convolutional kernel in the x-direction, F(a) is the parameter vector of the first convolutional kernel, G(xa) is the local vector matrix operated with the first convolutional kernel function, w is the size of the first convolutional kernel, X represents the velocity input vector, and Cov1(X) represents the first scale feature vector; then, the second convolutional layer of the multi-scale neighborhood feature extraction module performs one-dimensional convolutional encoding on the velocity input vector using the following second convolution formula to obtain the second scale feature vector; wherein, the second convolution formula is:

[0043]

[0044] Where b is the width of the second convolution kernel in the x direction, F(b) is the parameter vector of the second convolution kernel, G(xb) is the local vector matrix of the operation with the function of the second convolution kernel, m is the size of the second convolution kernel, X represents the velocity input vector, and Cov2(X) represents the second scale feature vector; finally, the first scale feature vector and the second scale feature vector are concatenated to obtain the velocity feature vector.

[0045] In this embodiment, the fusion module 130 is used to fuse the tea color feature vector and the speed feature vector to obtain a color-speed correlation feature vector. It should be understood that the tea color feature vector can represent the color distribution and characteristics of tea in different color spaces (such as RGB, HSV, etc.). These features can include the average value, variance, and color histogram of the color. The tea color feature vector reflects the characteristic information of the tea color. The speed feature vector reflects the speed value of the color sorter at different time points. These speed values ​​can represent the operating state of the color sorter, the speed at which objects pass through, etc. The speed feature vector can include the average value, maximum value, minimum value, and variance of the speed. By constructing the color-speed correlation feature vector, the tea color features and speed features can be combined to explore the correlation between tea color and speed. It is also known that the color and speed of tea are often related; they may reflect information about the quality, maturity, and processing of the tea. By fusing color features and speed features, this correlation can be strengthened and highlighted, making the relationship between color and speed more obvious and easily captured. When the color feature vector and speed feature vector are each represented as high-dimensional vectors, directly connecting them may lead to the curse of dimensionality. However, by using appropriate fusion methods, high-dimensional feature vectors can be transformed into color-velocity associated feature vectors with more suitable dimensions, which helps to reduce computational and storage complexity.

[0046] In this embodiment, the classification result module 140 is used to determine whether the speed value of the color sorter should be increased or decreased based on the color-speed correlation feature vector. It should be understood that the increase or decrease of the speed value can be determined according to the quality requirements of the tea. For example, if increasing the speed value can improve the quality of the tea, then the speed value should be increased. Conversely, if decreasing the speed value can improve the quality of the tea, then the speed value should be decreased. By classifying the color-speed correlation feature vector using a classifier, the speed adjustment decision can be automated. The classifier can learn the mapping pattern from the feature vector to speed adjustment, and based on the classification result of the current feature vector, it can automatically determine whether the speed value should be increased or decreased. The color-speed correlation feature vector captures the correlation between color and speed. By inputting these feature vectors into the classifier, machine learning algorithms can be used to learn the relationship between color features and speed features. The classifier can identify the speed adjustment direction corresponding to different color features and speed features, thereby adjusting the speed value according to the classification result of the current feature vector.

[0047] Specifically, in one embodiment of this application, the classification module is configured to: pass the color-speed associated feature vector through a classifier to obtain a classification result, the classification result being used to indicate whether the speed value of the color sorter at the current time point should increase or decrease. Specifically, the color-speed associated feature vector is fully connected and encoded using the fully connected layer of the classifier to obtain an encoded classification feature vector; the encoded classification feature vector is then passed through the Softmax classification function of the classifier to obtain a first probability that the speed value of the color sorter at the current time point should increase and a second probability that the speed value of the color sorter at the current time point should decrease; based on the comparison between the first probability and the second probability, the classification result is determined.

[0048] It should be understood that before using the aforementioned neural network model for inference, the color extractor based on the first convolutional neural network model, the speed extractor based on the multi-scale neighborhood feature extraction module, and the classifier need to be trained. That is, the intelligent tea sorting equipment control system 100 according to this application also includes a training phase 200 for training the color extractor based on the first convolutional neural network model, the speed extractor based on the multi-scale neighborhood feature extraction module, and the classifier.

[0049] Specifically, the training phase 200 includes: a training data acquisition unit for acquiring training data, which includes color sorter speed values ​​at multiple predetermined time points within a predetermined training time period and monitoring videos of tea leaves to be tested during the predetermined training time period; a training video extraction unit for extracting training keyframes for multiple tea leaf regions from the monitoring videos of tea leaves to be tested during the predetermined training time period; a training keyframe extraction unit for extracting training color feature maps from the training keyframes of the multiple tea leaf regions; a training color feature extraction unit for processing the training color feature maps using a color extractor based on a first convolutional neural network model to obtain training tea leaf color feature maps; a training color feature dimensionality reduction unit for performing global mean pooling on each feature matrix along the channel dimension of the training tea leaf color feature map to obtain training tea leaf color feature vectors; and a training speed multi-scale feature extraction unit for processing the color features at multiple predetermined time points within the predetermined training time period. The speed values ​​of the selected machine are arranged according to the time dimension to form a training speed input vector, which is then processed by a speed extractor based on a multi-scale neighborhood feature extraction module to obtain a training speed feature vector. A training fusion unit is used to fuse the training tea color feature vector and the training speed feature vector to obtain a training color-speed correlation feature vector. A geometric rigidity uniformity factor calculation unit is used to calculate a geometric rigidity uniformity factor based on an order-prior between the training tea color feature vector and the training speed feature vector. A classification loss unit is used to pass the training color-speed correlation feature vector through a classifier to obtain a classification loss function value. A model training unit is used to train the color extractor based on the first convolutional neural network model, the speed extractor based on the multi-scale neighborhood feature extraction module, and the classifier using a weighted sum of the classification loss function value and the geometric rigidity uniformity factor based on an order-prior as the loss function value.

[0050] Specifically, in the technical solution of this application, the tea color feature vector and the speed feature vector represent different aspects of information. The tea color feature vector reflects the color characteristics of the tea, while the speed feature vector reflects the operating speed of the color sorter. The stronger the dependency between these two feature vectors, the better their complementarity can be utilized, thus providing a more comprehensive and accurate feature representation. Simultaneously, the relationship between tea color and color sorter speed may be dynamic. By further enhancing the dependency between the tea color feature vector and the speed feature vector, their dynamic relationship can be better captured, thereby more accurately determining the speed adjustment needs of the color sorter at the current time point. Furthermore, by strengthening the dependency between the tea color feature vector and the speed feature vector, the model's generalization ability for different tea batches and different color sorter operating states can be improved. Increased dependency allows the model to better adapt to feature changes under different conditions, thereby improving the accuracy and stability of the classifier. The fusion of the tea color feature vector and the speed feature vector is to comprehensively utilize their information to determine the speed adjustment needs of the color sorter. By further enhancing the dependency between the two feature vectors, the feature fusion strategy can be optimized, and a more suitable fusion method and weight allocation can be selected, thereby improving the accuracy and reliability of the final classification result.

[0051] In other words, further enhancing the dependency between the tea color feature vector and the speed feature vector can better utilize their complementarity and dynamic relationship, improve the model's generalization ability, and optimize the feature fusion strategy. This enhancement allows for a more accurate determination of the speed adjustment needs of the color sorter at the current time point, thereby optimizing the color sorting process and improving the quality of the tea. Specifically, the dependency between the tea color feature vector and the speed feature vector is enhanced by calculating a geometrically rigid uniformity factor based on an order-prior prior between the trained tea color feature vector and the trained speed feature vector as a loss function.

[0052] In one embodiment of this application, a geometric rigidity uniformity factor calculation unit is used to: calculate the geometric rigidity uniformity factor based on order prior between the training tea color feature vector and the training speed feature vector using the following formula;

[0053]

[0054] Wherein, V1 represents the training tea color feature vector, V2 represents the training tea color feature vector, and ||·|| FLet Frobenius norm represent the feature vector, α represent the hyperparameter, cos(V1,V2) represent the cosine distance between the training tea color feature vector and the training speed feature vector, ||·|2 represents the second norm of the feature vector, and Loss represents the geometric rigidity uniformity factor based on the order prior.

[0055] In other words, to better establish the dependency between the training tea color feature vector and the training velocity feature vector in the high-dimensional feature space, the technical solution of this application utilizes the manifold structure observed from different dimensional perspectives within the high-dimensional feature space. This manifold structure can reflect the differences and similarities in the local feature descriptions associated with the feature matrices. To optimize the local feature descriptions, a geometrically rigid uniformity factor based on an order-priority premise is used to measure the geometric similarity of the manifold structure from different dimensional perspectives. This factor serves as a constraint to adjust the arrangement and transformation of the feature vectors, making them more consistent with the expression and matching of the training tea color feature vector and the training velocity feature vector.

[0056] In summary, this application also provides a system architecture diagram, as shown in the following figure. Figure 4 As shown. Figure 4 This is a schematic diagram of the architecture of a tea intelligent sorting equipment control system according to an embodiment of this application. In this system architecture, firstly, the speed values ​​of the color sorter at multiple predetermined time points within a predetermined time period and the monitoring video of the tea to be inspected during the predetermined time period are acquired. Then, key frames of multiple tea regions are extracted from the monitoring video of the tea to be inspected during the predetermined time period. Next, color feature maps are extracted from the key frames of the multiple tea regions. Then, the color feature maps are processed by a color extractor based on a first convolutional neural network model to obtain a tea color feature map. Next, the feature matrices along the channel dimension of the tea color feature map are subjected to global mean pooling to obtain a tea color feature vector. Then, the speed values ​​of the color sorter at the multiple predetermined time points are arranged according to the time dimension into a speed input vector and processed by a speed extractor based on a multi-scale neighborhood feature extraction module to obtain a speed feature vector. Then, the tea color feature vector and the speed feature vector are fused to obtain a color-speed associated feature vector. Finally, the color-speed associated feature vector is processed by a classifier to obtain a classification result, which indicates whether the speed value of the color sorter at the current time point should increase or decrease.

[0057] As described above, the intelligent tea sorting equipment control system 100 according to the embodiments of this application can be implemented in various terminal devices, such as servers for intelligent tea sorting equipment control systems. In one example, the intelligent tea sorting equipment control system 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the intelligent tea sorting equipment control system 100 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the intelligent tea sorting equipment control system 100 can also be one of many hardware modules of the terminal device.

[0058] Alternatively, in another example, the intelligent tea sorting equipment control system 100 and the terminal device can also be separate devices, and the intelligent tea sorting equipment control system 100 can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0059] Exemplary methods

[0060] Figure 5 This is a flowchart of a control method for an intelligent tea sorting device according to an embodiment of this application. Figure 5 As shown, the intelligent tea sorting equipment control method according to an embodiment of this application includes: S110, acquiring the speed values ​​of a color sorter at multiple predetermined time points within a predetermined time period and a monitoring video of the tea to be tested during the predetermined time period; S120, extracting features from the speed values ​​of the color sorter at the multiple predetermined time points and the monitoring video of the tea to be tested during the predetermined time period to obtain a tea color feature vector and a speed feature vector; S130, fusing the tea color feature vector and the speed feature vector to obtain a color-speed correlation feature vector; S140, determining whether the speed value of the color sorter should be increased or decreased based on the color-speed correlation feature vector.

[0061] Here, those skilled in the art will understand that the specific operations of each step in the above-described intelligent tea sorting equipment control method have been referenced above. Figures 1 to 4 The description of the intelligent tea sorting equipment control system is detailed here, and therefore, its repeated description will be omitted.

[0062] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0063] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0064] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0065] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0066] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A control system for an intelligent tea sorting device, characterized in that, include: The data acquisition module is used to acquire the speed values ​​of the color sorter at multiple predetermined time points within a predetermined time period, as well as the monitoring video of the tea leaves to be tested during the predetermined time period. The data processing module is used to extract features from the speed values ​​of the color sorter at the multiple predetermined time points and the monitoring video of the tea to be tested during the predetermined time period to obtain the tea color feature vector and speed feature vector. The fusion module is used to fuse the tea color feature vector and the speed feature vector to obtain a color-speed associated feature vector; The classification result module is used to determine whether the speed value of the color sorter should be increased or decreased based on the color-speed correlation feature vector.

2. The intelligent tea sorting equipment control system according to claim 1, characterized in that, The data processing module includes: The monitoring video processing unit is used to perform convolutional encoding on the monitoring video of the tea to be detected during the predetermined time period to obtain the tea color feature vector. The speed processing unit is used to process the speed values ​​of the color sorters at the multiple predetermined time points through a speed extractor to obtain a speed feature vector.

3. The intelligent tea sorting equipment control system according to claim 2, characterized in that, The surveillance video processing unit includes: The keyframe extraction subunit is used to extract keyframes of multiple tea areas from the monitoring video of the tea to be detected during the predetermined time period. The color feature extraction subunit is used to extract colors from keyframes of the multiple tea leaf regions to obtain tea leaf color feature vectors.

4. The intelligent tea sorting equipment control system according to claim 3, characterized in that, The keyframe extraction subunit includes: extracting keyframes of the multiple tea areas from the monitoring video of the tea to be detected during the predetermined time period at a predetermined sampling frequency.

5. The intelligent tea sorting equipment control system according to claim 4, characterized in that, The color feature extraction subunit includes: A secondary subunit for feature map extraction is used to extract color feature maps from keyframes of the multiple tea leaf regions; The first convolutional coding secondary subunit is used to pass the color feature map through a color extractor based on the first convolutional neural network model to obtain the tea color feature map; The dimension reduction second-level subunit is used to perform global mean pooling on each feature matrix along the channel dimension of the tea color feature map to obtain the tea color feature vector.

6. The intelligent tea sorting equipment control system according to claim 5, characterized in that, The speed processing unit is used to: arrange the speed values ​​of the color sorters at the multiple predetermined time points into a speed input vector according to the time dimension, and then use a speed extractor based on a multi-scale neighborhood feature extraction module to obtain a speed feature vector.

7. The intelligent tea sorting equipment control system according to claim 6, characterized in that, The classification module is used to: pass the color-speed associated feature vector through a classifier to obtain a classification result, the classification result being used to indicate whether the speed value of the color sorter at the current time point should increase or decrease.

8. The intelligent tea sorting equipment control system according to claim 7, characterized in that, It also includes a training module for training the color extractor based on the first convolutional neural network model, the speed extractor based on the multi-scale neighborhood feature extraction module, and the classifier; The training module includes: The training data acquisition unit is used to acquire training data, which includes the speed values ​​of the color sorter at multiple predetermined time points within a predetermined training time period and the monitoring video of the tea leaves to be tested within the predetermined training time period. The training video extraction unit is used to extract training keyframes for multiple tea regions from the monitoring video of the tea to be detected during the predetermined training time period. A training keyframe extraction unit is used to extract training color feature maps from training keyframes of the multiple tea leaf regions; A training color feature extraction unit is used to pass the training color feature map through a color extractor based on a first convolutional neural network model to obtain a training tea color feature map; A color feature dimensionality reduction unit is used to perform global mean pooling on each feature matrix along the channel dimension of the trained tea color feature map to obtain the trained tea color feature vector. The training speed multi-scale feature extraction unit is used to arrange the speed values ​​of the color sorter at multiple predetermined time points according to the time dimension into a training speed input vector, and then pass it through the speed extractor based on the multi-scale neighborhood feature extraction module to obtain the training speed feature vector. The training fusion unit is used to fuse the training tea color feature vector and the training speed feature vector to obtain a training color-speed correlation feature vector. A geometric rigidity uniformity factor calculation unit is used to calculate the geometric rigidity uniformity factor based on the order prior between the training tea color feature vector and the training velocity feature vector. The classification loss unit is used to pass the trained color-velocity associated feature vector through a classifier to obtain a classification loss function value; The model training unit is used to train the color extractor based on the first convolutional neural network model, the speed extractor based on the multi-scale neighborhood feature extraction module, and the classifier using the weighted sum between the classification loss function value and the geometric rigidity uniformity factor based on the order prior as the loss function value.

9. The intelligent tea sorting equipment control system according to claim 8, characterized in that, The computational geometric stiffness uniformity factor unit is used for: The geometric rigidity uniformity factor based on the order prior is calculated between the training tea color feature vector and the training speed feature vector using the following formula; Where V1 represents the training tea color feature vector, V2 represents the training tea color feature vector, ||·|| F Let Frobenius norm represent the feature vector, α represent the hyperparameter, cos(V1,V2) represent the cosine distance between the training tea color feature vector and the training speed feature vector, ||·||2 represents the second norm of the feature vector, and Loss represents the geometric rigidity uniformity factor based on the order prior.

10. A control method for an intelligent tea sorting device, characterized in that, include: Obtain the speed values ​​of the color sorter at multiple predetermined time points within a predetermined time period, as well as the monitoring video of the tea leaves to be tested during the predetermined time period; Feature extraction is performed on the speed values ​​of the color sorter at the multiple predetermined time points and the monitoring video of the tea to be tested during the predetermined time period to obtain the tea color feature vector and speed feature vector; The tea color feature vector and the velocity feature vector are fused to obtain a color-velocity related feature vector; Based on the color-speed correlation feature vector, the speed value of the color sorter should be increased or decreased.

Images