Grain and oil quality detection method and system

Through deep learning and artificial intelligence technology, the monitoring images in grain and oil quality detection are preprocessed and feature extraction, which solves the objectivity and accuracy of traditional detection methods, and achieves efficient and reliable grain and oil quality evaluation, improving product quality and Market competitiveness.

CN118918379AInactive Publication Date: 2024-11-08QINGDAO QIANQIU AGRI DEV CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411055539.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure CN118918379A_ABST
    Figure CN118918379A_ABST
Patent Text Reader

Abstract

The invention discloses a grain and oil quality detection method and system. The method comprises the following steps: firstly, preprocessing a monitoring image to obtain a preprocessed monitoring image, then enabling the monitoring image to pass through a sample feature extractor comprising a deep and shallow feature fusion module to obtain a sample monitoring feature map, and finally, obtaining a sample monitoring feature map; expanding the sample monitoring feature map along each sample monitoring feature matrix of a channel dimension to obtain a plurality of sample monitoring feature vectors, then enabling the plurality of sample monitoring feature vectors to pass through a contextual encoder based on a converter to obtain a sample global semantic association feature vector, and finally obtaining a sample global semantic association feature vector; and enabling the sample global semantic association feature vector to pass through a classifier to obtain a classification result used for representing whether the to-be-detected sample meets a preset requirement or not. Therefore, the quality characteristics of the sample to be detected can be accurately reflected.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent detection, and more specifically, to a grain and oil quality detection method and system thereof. Background Art

[0002] Grain and oil is a general term for grains such as cereals and beans, as well as their processed products and semi-finished products. Common ones include grains such as rice, wheat, corn, sorghum, and oils such as peanuts, palm fruits, and sunflower seeds.

[0003] Grain and oil quality testing is a process of systematically analyzing and evaluating the chemical, physical, and biological properties of the above-mentioned grains or oils. Traditional grain and oil quality testing usually evaluates sensory indicators (such as color, etc.) in an intuitive way, which is affected by individual differences, environmental factors, psychological factors, and other factors, resulting in the evaluation results being not objective and accurate.

[0004] Therefore, an optimized grain and oil quality testing solution is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a grain and oil quality detection method and system. It first preprocesses the monitoring image to obtain a preprocessed monitoring image, then passes the monitoring image through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map, then expands the sample monitoring feature map along each sample monitoring feature matrix of the channel dimension to obtain a plurality of sample monitoring feature vectors, then passes the plurality of sample monitoring feature vectors through a converter-based context encoder to obtain a sample global semantic association feature vector, and finally, passes the sample global semantic association feature vector through a classifier to obtain a classification result indicating whether the sample to be detected meets the predetermined requirements. In this way, the quality characteristics of the sample to be detected can be accurately reflected.

[0006] According to one aspect of the present application, a method for detecting grain and oil quality is provided, which comprises:

[0007] Acquire a monitoring image of the sample to be tested;

[0008] Preprocessing the monitoring image to obtain a preprocessed monitoring image;

[0009] Passing the monitoring image through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map;

[0010] Expanding each sample monitoring feature matrix of the sample monitoring feature graph along the channel dimension to obtain a plurality of sample monitoring feature vectors;

[0011] Passing the plurality of sample monitoring feature vectors through a transformer-based context encoder to obtain a sample global semantic association feature vector; and

[0012] The global semantic association feature vector of the sample is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements.

[0013] In the above-mentioned grain and oil quality detection method, the monitoring image is passed through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map, including:

[0014] Inputting the monitoring image into the sample feature extractor to extract a shallow feature map from a shallow layer of the sample feature extractor and a deep feature map from a deep layer of the sample feature extractor; and

[0015] The shallow-deep feature fusion module is used to cascade the shallow feature map and the deep feature map to obtain the sample monitoring feature map.

[0016] In the above-mentioned grain and oil quality detection method, the plurality of sample monitoring feature vectors are passed through a converter-based context encoder to obtain a sample global semantic association feature vector, including:

[0017] Passing the plurality of sample monitoring feature vectors through the context encoder to obtain a plurality of sample monitoring semantic feature vectors; and

[0018] The multiple sample monitoring semantic feature vectors are cascaded to obtain the sample global semantic association feature vector.

[0019] In the above-mentioned grain and oil quality detection method, the global semantic association feature vector of the sample is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements, including:

[0020] Using multiple fully connected layers of the classifier to perform fully connected encoding on the sample global semantic association feature vector to obtain an encoded classification feature vector; and

[0021] The encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the classification result.

[0022] The above-mentioned grain and oil quality detection method also includes a training step: training the sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier.

[0023] In the above-mentioned grain and oil quality detection method, the training step includes:

[0024] Acquire training data, wherein the training data includes a training monitoring image of a sample to be detected and a true value of whether the sample to be detected meets a predetermined requirement;

[0025] Preprocessing the training monitoring image to obtain a training preprocessed monitoring image;

[0026] Passing the training monitoring image through the sample feature extractor including the deep and shallow feature fusion module to obtain a training sample monitoring feature map;

[0027] Expanding each training sample monitoring feature matrix of the training sample monitoring feature graph along the channel dimension to obtain a plurality of training sample monitoring feature vectors;

[0028] Passing the plurality of training sample monitoring feature vectors through the transformer-based context encoder to obtain a training sample global semantic association feature vector;

[0029] Passing the global semantic association feature vector of the training sample through a classifier to obtain a classification loss function value; and

[0030] The sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier are trained based on the classification loss function value and propagated in the direction of gradient descent, wherein the training sample monitoring feature map is optimized at each iteration during the training process.

[0031] In the above-mentioned grain and oil quality detection method, the training sample monitoring feature map is optimized at each iteration in the training process, including: first, the training sample monitoring feature map is expanded into a training sample monitoring feature vector, the autocorrelation matrix of the training sample monitoring feature vector and its own transposition is calculated, and the inner product of the i-th and j-th row vectors of the autocorrelation matrix is ​​calculated as the matrix value of the (i, j)-th position of the weight matrix to obtain the weight matrix, and then, after matrix multiplication of the weight matrix and the autocorrelation matrix, further matrix-vector multiplication is performed with the training sample monitoring feature vector to obtain a correction vector, and finally, the correction vector is dot-multiplied with the training sample monitoring feature vector to obtain an optimized training sample monitoring feature vector, and the optimized training sample monitoring feature vector is restored to the optimized training sample monitoring feature map.

[0032] According to another aspect of the present application, a grain and oil quality detection system is provided, which includes:

[0033] An image acquisition module is used to acquire a monitoring image of the sample to be detected;

[0034] An image preprocessing module, used for preprocessing the monitoring image to obtain a preprocessed monitoring image;

[0035] A deep and shallow feature fusion module, used for passing the monitoring image through a sample feature extractor including the deep and shallow feature fusion module to obtain a sample monitoring feature map;

[0036] A feature matrix expansion module, used for expanding each sample monitoring feature matrix of the sample monitoring feature graph along the channel dimension to obtain a plurality of sample monitoring feature vectors;

[0037] A context encoding module, configured to pass the plurality of sample monitoring feature vectors through a transformer-based context encoder to obtain a sample global semantic association feature vector; and

[0038] The classification module is used to pass the global semantic association feature vector of the sample through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements.

[0039] In the above-mentioned grain and oil quality detection system, the depth and shallowness feature fusion module is used to:

[0040] Inputting the monitoring image into the sample feature extractor to extract a shallow feature map from a shallow layer of the sample feature extractor and a deep feature map from a deep layer of the sample feature extractor; and

[0041] The shallow-deep feature fusion module is used to cascade the shallow feature map and the deep feature map to obtain the sample monitoring feature map.

[0042] In the above-mentioned grain and oil quality detection system, the context encoding module is used to:

[0043] Passing the plurality of sample monitoring feature vectors through the context encoder to obtain a plurality of sample monitoring semantic feature vectors; and

[0044] The multiple sample monitoring semantic feature vectors are cascaded to obtain the sample global semantic association feature vector.

[0045] Compared with the prior art, the grain and oil quality detection method and system provided by the present application first preprocesses the monitoring image to obtain a preprocessed monitoring image, then passes the monitoring image through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map, then expands the sample monitoring feature map along each sample monitoring feature matrix of the channel dimension to obtain a plurality of sample monitoring feature vectors, then passes the plurality of sample monitoring feature vectors through a converter-based context encoder to obtain a sample global semantic association feature vector, and finally passes the sample global semantic association feature vector through a classifier to obtain a classification result indicating whether the sample to be detected meets the predetermined requirements. In this way, the quality characteristics of the sample to be detected can be accurately reflected. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The following drawings are not deliberately scaled according to the actual size, and the focus is on illustrating the main purpose of the present application.

[0047] Figure 1 This is an application scenario diagram of the grain and oil quality detection method according to an embodiment of the present application.

[0048] Figure 2 Flow chart of a grain and oil quality detection method according to an embodiment of the present application.

[0049] Figure 3 Schematic diagram of the structure of the grain and oil quality detection method according to the embodiment of the present application.

[0050] Figure 4 This is a flowchart of sub-step S130 of the grain and oil quality detection method according to an embodiment of the present application.

[0051] Figure 5 This is a flowchart of sub-step S150 of the grain and oil quality detection method according to an embodiment of the present application.

[0052] Figure 6 This is a flowchart of sub-step S160 of the grain and oil quality detection method according to an embodiment of the present application.

[0053] Figure 7 This is a flowchart of the training steps further included in the grain and oil quality detection method according to the embodiment of the present application.

[0054] Figure 8 It is a block diagram of a grain and oil quality detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0056] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0057] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0058] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0059] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0060] In response to the above technical problems, the technical concept of this application is to obtain monitoring images of samples to be tested, and use deep learning and artificial intelligence technology to perform automated grain and oil quality testing. In this way, the accuracy and reliability of grain and oil quality testing are improved, and the influence of human subjective factors on the evaluation results is avoided.

[0061] Specifically, in the technical solution of the present application, first, a monitoring image of the sample to be detected is obtained. In the actual operation process, a camera or an industrial camera can be used to monitor and collect the sample to be detected.

[0062] Since the monitoring image may have some problems such as noise, blur, light, etc., in the technical solution of the present application, the monitoring image is preprocessed to solve or alleviate these problems, thereby obtaining a preprocessed monitoring image. Specifically, the preprocessing may include operations such as image denoising, filtering, and enhancement, as well as operations such as image normalization and resizing. The preprocessed monitoring image can retain more useful information, remove some irrelevant information, and better reflect the true situation of the sample to be tested.

[0063] In order to capture various feature information of the sample to be detected, including color, texture, shape, etc., in the technical solution of the present application, the monitoring image is passed through a sample feature extractor including a deep and shallow feature fusion module to learn to extract feature information of different levels in the image, and abstract a higher-level feature representation, thereby obtaining a sample monitoring feature map. Here, the sample feature extractor is a convolutional neural network model, and the deep and shallow features generated by different convolutional layers can be fused by using a deep and shallow feature fusion module, thereby taking into account the extraction of multi-scale features. The advantage of this is that it can effectively reduce the loss of feature information and increase the capture of multi-level features of the image.

[0064] Although convolutional neural networks have excellent performance in image feature extraction, they are restricted by the limitations of convolutional coding. The features extracted by the sample feature extractor including the deep and shallow feature fusion module will have a relatively small feature receptive field. In other words, the sample monitoring feature map is relatively weak in terms of global features.

[0065] Based on this, in the technical solution of the present application, first, the sample monitoring feature matrices of the sample monitoring feature map along the channel dimension are expanded to convert each feature matrix in the sample monitoring feature map into a feature vector, thereby obtaining a plurality of sample monitoring feature vectors. Then, the plurality of sample monitoring feature vectors are passed through a converter-based context encoder to obtain a sample global semantic association feature vector. Here, the converter-based context encoder is based on the Transformer mechanism, which can make full use of the intrinsic relationship between the sample monitoring feature vectors to realize the interaction and information transfer between the feature vectors. In this way, the feature information of the global semantic association of the sample can be effectively extracted, thereby more accurately reflecting the quality characteristics of the sample to be detected.

[0066] After obtaining the global semantic association feature vector of the sample, it is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be tested meets the predetermined requirements. Specifically, the classifier can be trained and optimized using an existing training data set so that it can better identify and classify samples of different qualities. By classifying the sample to be tested, the quality evaluation result of the sample can be obtained, that is, whether the sample to be tested meets the predetermined requirements and whether the sample to be tested does not meet the predetermined requirements. In actual application, the classification result can be used as one of the important indicators for grain and oil quality testing, which is of great significance for ensuring product quality and market competitiveness.

[0067] Here, when the multiple sample monitoring feature vectors are passed through a converter-based context encoder to obtain a sample global semantic association feature vector, it is considered that the converter-based context encoder will perform context image semantic association encoding on the multiple sample monitoring feature vectors, and each sample monitoring feature vector is obtained by expanding each sample monitoring feature matrix of the sample monitoring feature map along the channel dimension. Therefore, if the feature redundancy between the various sample monitoring feature matrices is low, the context encoding effect of the converter-based context encoder can be improved, thereby improving the feature expression effect of the sample global semantic association feature vector, so as to improve the accuracy of the classification result obtained by the classifier.

[0068] Therefore, the applicant of the present application performs feature redundancy optimization based on low-cost bottleneck mechanism stacking on the sample monitoring feature graph, for example, denoted as F, during the training process to obtain a trained sample monitoring feature graph, for example, denoted as F. ′ , specifically expressed as:

[0069]

[0070] F b =Cov(F a )

[0071] Cov represents a single-layer convolution operation. and ⊙ respectively represent the position-by-position addition, subtraction and multiplication of the feature maps, and B1 and B2 are biased feature maps, for example, they can be initially set to the global mean feature map or unit feature map of the sample monitoring feature map, where the initial biased feature maps B1 and B2 are different.

[0072] Here, the feature redundancy optimization based on stacking of cheap bottleneck mechanisms can use a cheap bottleneck mechanism of multiplication and addition stacking of two cheap transformation features to perform feature expansion, and match the residual path by stacking channels with a uniform value bias, so as to reveal the implicit distribution information under the intrinsic essential features in the redundant features through a cheap operation transformation with low cost similar to the basic residual module, so as to obtain a more intrinsic expression of the features through a simple and effective convolution operation architecture, thereby optimizing the redundant feature expression of the sample monitoring feature map, improving the context encoding effect of the converter-based context encoder, thereby improving the feature expression effect of the sample global semantic association feature vector, so as to improve the accuracy of the classification results obtained by the classifier.

[0073] This application has the following technical effects:

[0074] 1. Provides an intelligent grain and oil quality testing solution.

[0075] 2. The above-mentioned intelligent grain and oil quality detection solution can process and analyze the monitoring images of the samples to be tested to realize the automatic evaluation of grain and oil quality. In this way, the accuracy and reliability of grain and oil quality detection are improved, and the influence of human subjective factors on the evaluation results is avoided. At the same time, the grain and oil quality detection time and labor costs can be greatly shortened, and work efficiency can be improved, thereby effectively improving product quality and market competitiveness.

[0076] Figure 1 FIG. 1 is an application scenario diagram of the grain and oil quality detection method according to an embodiment of the present application. Figure 1 As shown, in this application scenario, first, a camera (for example, Figure 1 C) as shown in the figure, obtaining a monitoring image of the sample to be detected (for example, Figure 1 D), and then, the monitoring image is input into a server deployed with a grain and oil quality detection algorithm (for example, Figure 1 S) as shown in , wherein the server can use the grain and oil quality detection algorithm to process the monitoring image to obtain a classification result indicating whether the sample to be detected meets the predetermined requirements.

[0077] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0078] Figure 2 Flow chart of the grain and oil quality detection method according to the embodiment of the present application. Figure 2As shown, the grain and oil quality detection method according to the embodiment of the present application includes the steps of: S110, acquiring a monitoring image of the sample to be detected; S120, preprocessing the monitoring image to obtain a preprocessed monitoring image; S130, passing the monitoring image through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map; S140, expanding the sample monitoring feature map along each sample monitoring feature matrix of the channel dimension to obtain a plurality of sample monitoring feature vectors; S150, passing the plurality of sample monitoring feature vectors through a converter-based context encoder to obtain a sample global semantic association feature vector; and, S160, passing the sample global semantic association feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements.

[0079] Figure 3 FIG. 1 is a schematic diagram of the structure of the grain and oil quality detection method according to the embodiment of the present application. Figure 3 As shown, in the network architecture, first, a monitoring image of the sample to be detected is obtained; then, the monitoring image is preprocessed to obtain a preprocessed monitoring image; then, the monitoring image is passed through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map; then, the sample monitoring feature map is expanded along each sample monitoring feature matrix of the channel dimension to obtain a plurality of sample monitoring feature vectors; then, the plurality of sample monitoring feature vectors are passed through a converter-based context encoder to obtain a sample global semantic association feature vector; finally, the sample global semantic association feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements.

[0080] More specifically, in step S110, a monitoring image of the sample to be detected is obtained. In the actual operation process, a camera or an industrial camera can be used to monitor and collect the sample to be detected.

[0081] More specifically, in step S120, the monitoring image is preprocessed to obtain a preprocessed monitoring image. Since the monitoring image may have some noise, blur, light and other problems, in the technical solution of the present application, the monitoring image is preprocessed to solve or alleviate these problems, thereby obtaining a preprocessed monitoring image. Specifically, the preprocessing may include operations such as image denoising, filtering, and enhancement, as well as operations such as image normalization and resizing. The preprocessed monitoring image can retain more useful information, remove some irrelevant information, and better reflect the true situation of the sample to be detected.

[0082] More specifically, in step S130, the monitoring image is passed through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map. In order to capture various feature information of the sample to be detected, including color, texture, shape, etc., in the technical solution of the present application, the monitoring image is passed through a sample feature extractor including a deep and shallow feature fusion module to learn to extract feature information of different levels in the image, and abstract a higher-level feature representation, thereby obtaining a sample monitoring feature map. Here, the sample feature extractor is a convolutional neural network model. The deep and shallow feature fusion module can be used to fuse the deep and shallow features generated by different convolutional layers, thereby taking into account the extraction of multi-scale features. The advantage of this is that it can effectively reduce the loss of feature information and increase the capture of multi-level features of the image.

[0083] Accordingly, in a specific example, Figure 4 As shown, the monitoring image is passed through a sample feature extractor including a shallow-deep feature fusion module to obtain a sample monitoring feature map, including: S131, inputting the monitoring image into the sample feature extractor to extract a shallow feature map from the shallow layer of the sample feature extractor and to extract a deep feature map from the deep layer of the sample feature extractor; and, S132, using the shallow-deep feature fusion module to cascade the shallow feature map and the deep feature map to obtain the sample monitoring feature map.

[0084] It should be understood that Convolutional Neural Network (CNN) is an artificial neural network that is widely used in image recognition and other fields. Convolutional neural network may include input layer, hidden layer and output layer, wherein the hidden layer may include convolution layer, pooling layer, activation layer and fully connected layer, etc. The previous layer performs corresponding operations according to the input data and outputs the operation results to the next layer. The input initial data is processed through multiple layers of operations to obtain a final result.

[0085] More specifically, in step S140, the sample monitoring feature matrices along the channel dimension of the sample monitoring feature map are expanded to obtain multiple sample monitoring feature vectors. Although convolutional neural networks have excellent performance in image feature extraction, they are limited by the limitations of convolutional coding. The features extracted by the sample feature extractor including the deep and shallow feature fusion module will have a relatively small feature receptive field, that is, the sample monitoring feature map is weak in global features. Based on this, the sample monitoring feature matrices along the channel dimension of the sample monitoring feature map can be expanded to convert each feature matrix in the sample monitoring feature map into a feature vector, thereby obtaining multiple sample monitoring feature vectors.

[0086] More specifically, in step S150, the plurality of sample monitoring feature vectors are passed through a transformer-based context encoder to obtain a sample global semantic association feature vector. The transformer-based context encoder is based on the Transformer mechanism and can make full use of the intrinsic relationship between the sample monitoring feature vectors to achieve interaction and information transfer between feature vectors. In this way, feature information of the sample global semantic association can be effectively extracted, thereby more accurately reflecting the quality characteristics of the sample to be detected.

[0087] It should be understood that through the context encoder, the relationship between a certain word segmentation and other word segmentations in the vector representation sequence can be analyzed to obtain the corresponding feature information. The context encoder aims to mine the hidden patterns between the contexts in the word sequence. Optionally, the encoder includes: CNN (Convolutional Neural Network), Recursive NN (Recursive Neural Network), Language Model, etc. The CNN-based method has a better extraction effect on local features, but it is not effective for the long-term dependency problem in sentences. Therefore, the encoder based on Bi-LSTM (Long Short-Term Memory) is widely used. Recursive NN treats sentences as tree structures rather than sequences. In theory, it has stronger representation capabilities, but it has weaknesses such as difficulty in sample annotation, easy gradient disappearance in deep layers, and difficulty in parallel calculation. Therefore, it is rarely used in practical applications. Transformer is a widely used network structure that has the characteristics of both CNN and RNN. It has a good extraction effect on global features and has certain advantages in parallel computing compared to RNN (recurrent neural network).

[0088] Accordingly, in a specific example, Figure 5 As shown, the multiple sample monitoring feature vectors are passed through a converter-based context encoder to obtain a sample global semantic association feature vector, including: S151, passing the multiple sample monitoring feature vectors through the context encoder to obtain multiple sample monitoring semantic feature vectors; and, S152, cascading the multiple sample monitoring semantic feature vectors to obtain the sample global semantic association feature vector.

[0089] More specifically, in step S160, the sample global semantic association feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be tested meets the predetermined requirements. The classification result can be used as one of the important indicators for grain and oil quality testing, which is of great significance for ensuring product quality and market competitiveness.

[0090] That is, in the technical solution of the present application, the label of the classifier includes that the sample to be detected meets the predetermined requirements (first label), and that the sample to be detected does not meet the predetermined requirements (second label), wherein the classifier determines which classification label the classification feature vector belongs to through a soft maximum function. It is worth noting that the first label p1 and the second label p2 here do not contain artificially set concepts. In fact, during the training process, the computer model does not have the concept of "whether the sample to be detected meets the predetermined requirements". It only has two classification labels and the probability of the output feature under these two classification labels, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the sample to be detected meets the predetermined requirements is actually converted into a binary class probability distribution that conforms to the laws of nature through the classification label. In essence, the physical meaning of the natural probability distribution of the label is used, rather than the linguistic text meaning of "whether the sample to be detected meets the predetermined requirements".

[0091] It should be understood that the role of the classifier is to use the given categories and known training data to learn classification rules and classifiers, and then classify (or predict) unknown data. Logistic regression and SVM are often used to solve binary classification problems. For multi-class classification problems, logistic regression or SVM can also be used, but multiple binary classifications are required to form a multi-classification. However, this is prone to errors and is not efficient. Commonly used multi-classification methods include the Softmax classification function.

[0092] Accordingly, in a specific example, Figure 6 As shown, the sample global semantic association feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be tested meets predetermined requirements, including: S161, using multiple fully connected layers of the classifier to perform fully connected encoding on the sample global semantic association feature vector to obtain an encoded classification feature vector; and, S162, passing the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0093] Accordingly, in a specific example, the grain and oil quality detection method further includes a training step: training the sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier. Figure 7 As shown, the training step includes: S210, acquiring training data, the training data including a training monitoring image of a sample to be detected, and a true value of whether the sample to be detected meets a predetermined requirement; S220, preprocessing the training monitoring image to obtain a preprocessed training monitoring image; S230, passing the training monitoring image through the sample feature extractor including the deep and shallow feature fusion module to obtain a training sample monitoring feature map; S240, expanding the training sample monitoring feature map along each training sample monitoring feature matrix of the channel dimension to obtain a plurality of training sample monitoring feature vectors; S250, passing the multiple training sample monitoring feature vectors through the converter-based context encoder to obtain the training sample global semantic association feature vector; S260, passing the training sample global semantic association feature vector through the classifier to obtain the classification loss function value; and, S270, training the sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier based on the classification loss function value and by directional propagation of gradient descent, wherein the training sample monitoring feature map is optimized at each iteration in the training process.

[0094] Here, here, when the multiple training sample monitoring feature vectors are passed through the converter-based context encoder to obtain the training sample global semantic association feature vector, it is considered that the converter-based context encoder will perform context image semantic association encoding on the multiple training sample monitoring feature vectors, and each training sample monitoring feature vector is obtained by expanding each sample monitoring feature matrix of the training sample monitoring feature map along the channel dimension. Therefore, if the predetermined category harmony based on feature redundancy between each training sample monitoring feature matrix is ​​high, the context class probability constraint coding effect of the converter-based context encoder can be improved, thereby improving the category feature expression effect of the training sample global semantic association feature vector, so as to improve the accuracy of the classification result obtained by the classifier.

[0095] Therefore, the present application optimizes the training sample monitoring feature map at each iteration during the training process, specifically including the following optimization steps: first, the training sample monitoring feature map is expanded into a training sample monitoring feature vector, the autocorrelation matrix of the training sample monitoring feature vector and its own transposition is calculated, and the inner product of the i-th and j-th row vectors of the autocorrelation matrix is ​​calculated as the matrix value of the (i, j)-th position of the weight matrix to obtain the weight matrix, and then, after matrix multiplication of the weight matrix and the autocorrelation matrix, further matrix-vector multiplication is performed with the training sample monitoring feature vector to obtain a correction vector, and finally, the correction vector is dot-multiplied with the training sample monitoring feature vector to obtain an optimized training sample monitoring feature vector, and the optimized training sample monitoring feature vector is restored to the optimized training sample monitoring feature map.

[0096] The formula of the specific optimization steps is expressed as:

[0097]

[0098] Where V represents the monitoring feature vector of the training sample, T represents the transpose of the vector, M represents the autocorrelation matrix, and V i and V j represents the i-th and j-th row vectors of the autocorrelation matrix, M′ represents the weight matrix, m i,j represents the matrix value of the (i, j)th position of the weight matrix, L represents the height of the autocorrelation matrix, and R represents a set of real numbers. represents matrix multiplication, ⊙ represents dot product, and V′ represents the optimized monitoring feature vector of training samples.

[0099] That is, based on the self-correlation dimension of the training sample monitoring feature vector as the object to be modulated, the correlation expansion is performed based on the spatial sub-dimensional complexity of the high-dimensional feature space of the feature distribution, thereby introducing the decomposable correlation dimension offset of the training sample monitoring feature vector into the heterogeneous correlation embedding space, thereby enhancing the correlation self-consistency through the joint fine-tuning of the decomposable dimension set represented by the heterogeneous correlation embedding space, so as to improve the predetermined category harmony of the feature set of the training sample monitoring feature map in the associated target classification domain, thereby improving the context class probability constraint coding effect of the converter-based context encoder, and then improving the category feature expression effect of the global semantic association feature vector of the training sample, so as to improve the accuracy of the classification result obtained by the classifier.

[0100] In summary, based on the grain and oil quality detection method of the embodiment of the present application, the monitoring image is first preprocessed to obtain a preprocessed monitoring image, then the monitoring image is passed through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map, then the sample monitoring feature map is expanded along each sample monitoring feature matrix of the channel dimension to obtain a plurality of sample monitoring feature vectors, then the plurality of sample monitoring feature vectors are passed through a converter-based context encoder to obtain a sample global semantic association feature vector, and finally, the sample global semantic association feature vector is passed through a classifier to obtain a classification result indicating whether the sample to be detected meets the predetermined requirements. In this way, the quality characteristics of the sample to be detected can be accurately reflected.

[0101] Figure 8 FIG. 1 is a block diagram of a grain and oil quality detection system 100 according to an embodiment of the present application. Figure 8 As shown, the grain and oil quality detection system 100 according to the embodiment of the present application includes: an image acquisition module 110, which is used to acquire a monitoring image of a sample to be detected; an image preprocessing module 120, which is used to preprocess the monitoring image to obtain a preprocessed monitoring image; a depth feature fusion module 130, which is used to pass the monitoring image through a sample feature extractor including a depth feature fusion module to obtain a sample monitoring feature map; a feature matrix expansion module 140, which is used to expand each sample monitoring feature matrix of the sample monitoring feature map along the channel dimension to obtain a plurality of sample monitoring feature vectors; a context encoding module 150, which is used to pass the plurality of sample monitoring feature vectors through a converter-based context encoder to obtain a sample global semantic association feature vector; and a classification module 160, which is used to pass the sample global semantic association feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements.

[0102] In one example, in the above-mentioned grain and oil quality detection system 100, the deep and shallow feature fusion module 130 is used to: input the monitoring image into the sample feature extractor to extract a shallow feature map from the shallow layer of the sample feature extractor and to extract a deep feature map from the deep layer of the sample feature extractor; and, use the deep and shallow feature fusion module to cascade the shallow feature map and the deep feature map to obtain the sample monitoring feature map.

[0103] In one example, in the above-mentioned grain and oil quality detection system 100, the context encoding module 150 is used to: pass the multiple sample monitoring feature vectors through the context encoder to obtain multiple sample monitoring semantic feature vectors; and cascade the multiple sample monitoring semantic feature vectors to obtain the sample global semantic association feature vector.

[0104] In one example, in the above-mentioned grain and oil quality detection system 100, the classification module 160 is used to: use multiple fully connected layers of the classifier to fully connect the global semantic association feature vector of the sample to obtain an encoded classification feature vector; and pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0105] In one example, in the above-mentioned grain and oil quality detection system 100, a training module for training the sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier is also included. Wherein, the training module is used to: obtain training data, the training data includes a training monitoring image of the sample to be detected, and whether the sample to be detected meets the real value of the predetermined requirement; preprocess the training monitoring image to obtain a training preprocessed monitoring image; pass the training monitoring image through the sample feature extractor including the deep and shallow feature fusion module to obtain a training sample monitoring feature map; expand the training sample monitoring feature map along each training sample monitoring feature matrix of the channel dimension to obtain a plurality of training sample monitoring feature vectors; pass the plurality of training sample monitoring feature vectors through the converter-based context encoder to obtain a training sample global semantic association feature vector; pass the training sample global semantic association feature vector through the classifier to obtain a classification loss function value; and, based on the classification loss function value and propagated in the direction of gradient descent, train the sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier.

[0106] In one example, in the grain and oil quality detection system 100, the training sample monitoring feature graph is subjected to feature redundancy optimization to obtain the training sample monitoring feature graph, including: performing feature redundancy optimization on the training sample monitoring feature graph according to the following optimization formula to obtain the training sample monitoring feature graph; wherein the optimization formula is:

[0107]

[0108] F b =Cov(F a )

[0109] Wherein, F represents the monitoring feature map of the training sample, F ′ represents the monitoring feature map of the training sample, Cov represents a single-layer convolution operation, represents the position-by-position addition of feature maps, represents the position-by-position subtraction of feature maps, ⊙ represents the position-by-position multiplication of feature maps, and B1 and B2 are bias feature maps, where the initial bias feature maps B1 and B2 are different.

[0110] Here, those skilled in the art will appreciate that the specific functions and operations of each module in the grain and oil quality detection system 100 have been described in detail above. Figures 1 to 7 The description of the grain and oil quality detection method has been introduced in detail, and therefore, its repeated description will be omitted.

[0111] As described above, the grain and oil quality detection system 100 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a grain and oil quality detection algorithm. In one example, the grain and oil quality detection system 100 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the grain and oil quality detection system 100 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the grain and oil quality detection system 100 can also be one of the many hardware modules of the wireless terminal.

[0112] Alternatively, in another example, the grain and oil quality detection system 100 and the wireless terminal may also be separate devices, and the grain and oil quality detection system 100 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0113] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer-readable instructions are stored. When the instructions are executed by a computer, the above-mentioned method can be executed.

[0114] The program part of the technology can be considered as a "product" or "manufactured product" in the form of executable code and / or related data, which is participated in or realized by computer-readable media. Tangible and permanent storage media can include any memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives or any similar devices that can provide storage functions for software.

[0115] All or part of the software may sometimes communicate over a network, such as the Internet or other communications network. Such communications can load software from one computer device or processor to another. For example: from a server or host computer of a video target detection device to a hardware platform of a computer environment, or other computer environment that implements the system, or a system with similar functions related to providing information required for target detection. Therefore, another medium capable of transmitting software elements can also be used as a physical connection between local devices, such as light waves, radio waves, electromagnetic waves, etc., transmitted through cables, optical cables or air. Physical media used to carry carriers, such as cables, wireless connections or optical cables and the like, can also be considered as media that carry software. As used herein, unless limited to tangible "storage" media, other terms referring to computer or machine "readable media" refer to media involved in the process of the processor executing any instructions.

[0116] The present application uses specific words to describe the embodiments of the present application. For example, "first / second embodiment", "one embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0117] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0118] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0119] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. A method for detecting grain and oil quality, It is characterized in that include: Acquire a monitoring image of the sample to be tested; Preprocessing the monitoring image to obtain a preprocessed monitoring image; Passing the monitoring image through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map; Expanding each sample monitoring feature matrix of the sample monitoring feature graph along the channel dimension to obtain a plurality of sample monitoring feature vectors; Passing the plurality of sample monitoring feature vectors through a transformer-based context encoder to obtain a sample global semantic association feature vector; as well as The global semantic association feature vector of the sample is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements.

2. The method for detecting the quality of grain and oil according to claim 1, It is characterized in that The monitoring image is passed through a sample feature extractor including a deep and shallow feature fusion module to obtain a sample monitoring feature map, including: Inputting the monitoring image into the sample feature extractor to extract a shallow feature map from a shallow layer of the sample feature extractor and a deep feature map from a deep layer of the sample feature extractor; and The shallow-deep feature fusion module is used to cascade the shallow feature map and the deep feature map to obtain the sample monitoring feature map.

3. The method for detecting the quality of grain and oil according to claim 2, It is characterized in that Passing the plurality of sample monitoring feature vectors through a converter-based context encoder to obtain a sample global semantic association feature vector, comprising: Passing the plurality of sample monitoring feature vectors through the context encoder to obtain a plurality of sample monitoring semantic feature vectors; and The multiple sample monitoring semantic feature vectors are cascaded to obtain the sample global semantic association feature vector.

4. The method for detecting grain and oil quality according to claim 3, It is characterized in that The global semantic association feature vector of the sample is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be tested meets the predetermined requirements, including: Using multiple fully connected layers of the classifier to perform fully connected encoding on the sample global semantic association feature vector to obtain an encoded classification feature vector; and The encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the classification result.

5. The method for detecting grain and oil quality according to claim 4, It is characterized in that It also includes a training step: training the sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier.

6. The method for detecting the quality of grain and oil according to claim 5, It is characterized in that The training step comprises: Acquire training data, wherein the training data includes a training monitoring image of a sample to be detected and a true value of whether the sample to be detected meets a predetermined requirement; Preprocessing the training monitoring image to obtain a training preprocessed monitoring image; Passing the training monitoring image through the sample feature extractor including the deep and shallow feature fusion module to obtain a training sample monitoring feature map; Expanding each training sample monitoring feature matrix of the training sample monitoring feature graph along the channel dimension to obtain a plurality of training sample monitoring feature vectors; Passing the plurality of training sample monitoring feature vectors through the transformer-based context encoder to obtain a training sample global semantic association feature vector; Passing the global semantic association feature vector of the training sample through a classifier to obtain a classification loss function value; and The sample feature extractor including the deep and shallow feature fusion module, the converter-based context encoder and the classifier are trained based on the classification loss function value and propagated in the direction of gradient descent, wherein the training sample monitoring feature map is optimized at each iteration during the training process.

7. The method for detecting the quality of grain and oil according to claim 6, It is characterized in that At each iteration in the training process, the training sample monitoring feature map is optimized, including: first, the training sample monitoring feature map is expanded into a training sample monitoring feature vector, the autocorrelation matrix of the training sample monitoring feature vector and its own transposition is calculated, and the inner product of the i-th and j-th row vectors of the autocorrelation matrix is ​​calculated as the matrix value of the (i, j)-th position of the weight matrix to obtain the weight matrix, then, after matrix multiplication of the weight matrix and the autocorrelation matrix, further matrix-vector multiplication is performed with the training sample monitoring feature vector to obtain a correction vector, and finally, the correction vector is dot-multiplied with the training sample monitoring feature vector to obtain an optimized training sample monitoring feature vector, and the optimized training sample monitoring feature vector is restored to the optimized training sample monitoring feature map.

8. A grain and oil quality detection system, It is characterized in that include: An image acquisition module is used to acquire a monitoring image of the sample to be detected; An image preprocessing module, used for preprocessing the monitoring image to obtain a preprocessed monitoring image; A deep and shallow feature fusion module, used for passing the monitoring image through a sample feature extractor including the deep and shallow feature fusion module to obtain a sample monitoring feature map; A feature matrix expansion module, used for expanding each sample monitoring feature matrix of the sample monitoring feature graph along the channel dimension to obtain a plurality of sample monitoring feature vectors; A context encoding module, configured to pass the plurality of sample monitoring feature vectors through a converter-based context encoder to obtain a sample global semantic association feature vector; as well as The classification module is used to pass the global semantic association feature vector of the sample through a classifier to obtain a classification result, and the classification result is used to indicate whether the sample to be detected meets the predetermined requirements.

9. The grain and oil quality detection system according to claim 8, It is characterized in that The deep and shallow feature fusion module is used to: Inputting the monitoring image into the sample feature extractor to extract a shallow feature map from a shallow layer of the sample feature extractor and a deep feature map from a deep layer of the sample feature extractor; as well as The shallow-deep feature fusion module is used to cascade the shallow feature map and the deep feature map to obtain the sample monitoring feature map.

10. The grain and oil quality detection system according to claim 9, It is characterized in that The context encoding module is used to: Passing the plurality of sample monitoring feature vectors through the context encoder to obtain a plurality of sample monitoring semantic feature vectors; and The multiple sample monitoring semantic feature vectors are cascaded to obtain the sample global semantic association feature vector.