Forage information identification and acquisition system and method based on machine vision

Abstract

The invention relates to the technical field of information perception, and particularly discloses a pasture information identification and acquisition system and method based on machine vision, which adopts an image processing technology based on deep learning to carry out target detection on the ingestion image of a sheep so as to focus on a pasture object, and meanwhile, carries out identification and acquisition on the pasture information. The method comprises the following steps: extracting a set of pasture prototype images marked with pasture types from a database, respectively extracting image features of each pasture prototype image and a to-be-detected pasture object, and further performing query response coding on the image features of the to-be-detected pasture object and each pasture prototype image feature to intelligently identify the type of the pasture object. In this way, the generalization ability of the model can be effectively improved, the diversity of forage grass under different environments and conditions can be better handled, and the accuracy and efficiency of forage grass information identification are further improved.

Description

Technical Field

[0001] The present application relates to the field of information perception technology, and more specifically, to a forage information recognition and collection system and method based on machine vision. Background Art

[0002] With the development of modern agricultural technology, accurate identification and collection of forage information is of great significance to improving the management level of animal husbandry. Accurate forage information can not only help farmers arrange grazing plans reasonably, but also be used to monitor the ecological health of grasslands and improve the production efficiency of animal husbandry. Traditionally, the collection of forage information mainly relies on manual observation and recording, which is not only time-consuming and labor-intensive, but also prone to errors and difficult to meet the management needs of large-scale pastures.

[0003] In recent years, with the development of computer vision technology, image-based forage information recognition methods have gradually become a research hotspot. For example, the invention patent with publication number CN118230419A discloses a method for identifying forage information of grazing sheep, which fixes the camera on the front chest of the sheep by wearing a back clip, collects the sheep's feeding pictures and audio, and processes the captured sheep feeding pictures based on the recognition algorithm, and then integrates them with the recorded audio to identify and judge the type of forage, growth stage and other information.

[0004] However, although the above method can automatically collect and process data and reduce manual intervention, the accuracy of forage information recognition still needs to be improved. Specifically, although this method can realize the recognition of forage species in specific scenarios, the generalization ability of the model still needs to be further verified. For example, if the model is tested in different geographical areas or different types of pastures, the performance of the model may vary, and more training data support is needed to enhance the robustness and generalization ability of the model. In addition, this method introduces sound signals as auxiliary information when identifying forage species, but in an open natural environment, there is a lot of background noise, which may affect the extraction and analysis of sound signals, and thus affect the final recognition results. Moreover, the contribution of sound signals to forage information recognition is relatively low, and data processing and analysis require additional computing resources, which further increases the complexity and cost of the system.

[0005] Therefore, an optimized machine vision-based forage information recognition and collection system and method is expected. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a forage information recognition and collection system and method based on machine vision, which uses deep learning-based image processing technology to perform target detection on sheep feeding images to focus on forage objects. At the same time, a collection of forage type images marked with forage types is extracted from the database, and the image features of each forage type image and the forage object to be detected are extracted respectively. Then, the image features of the forage object to be detected and the image features of each forage type are queried and encoded to intelligently identify the type of forage object. In this way, the generalization ability of the model can be effectively improved, the diversity of forage under different environments and conditions can be better coped with, and the accuracy and efficiency of forage information recognition can be further improved.

[0007] According to one aspect of the present application, a method for identifying and collecting forage information based on machine vision is provided, which includes: wearing a wearable back clip on the front of a sheep to cover the front chest of the sheep; fixing a camera protective cover on the front chest of the sheep through the wearable back clip; placing a high-definition anti-shake camera in the camera protective cover to capture a complete picture of the sheep when feeding; processing the captured feeding image based on a recognition algorithm to identify the sheep's feeding forage information.

[0008] In the above-mentioned forage information recognition and collection method based on machine vision, the captured feeding images are processed based on the recognition algorithm to identify the forage feeding information of the sheep, including: performing target detection on the feeding images to obtain a local focused image of the forage object; extracting a set of forage type images labeled with forage types from a database; performing forage feature extraction on each forage type image in the set of forage type images and the local focused image of the forage object to obtain a set of forage type image feature vectors and a feature vector of the image of the forage object to be detected; performing query encoding on the feature vectors of the image of the forage object to be detected and the set of the forage type image feature vectors to obtain an optimized encoding vector for query response of the forage to be detected; and determining the type label of the forage object based on the optimized encoding vector for query response of the forage to be detected.

[0009] In the above-mentioned machine vision-based forage information identification and collection method, the set of the image feature vectors of the forage object to be detected and the set of the forage type image feature vectors are query encoded, including: extracting the significant features of the set of the forage type image feature vectors as a prompt template; based on the prompt template, performing cross-domain optimized query encoding guided by significant features on the set of the image feature vectors of the forage object to be detected and the set of the forage type image feature vectors to obtain the query response optimized encoding vector of the forage to be detected.

[0010] In the above-mentioned method for identifying and collecting forage information based on machine vision, target detection is performed on the grazing image to obtain a local focused image of the forage object, including: inputting the grazing image into a forage object target detection network based on the YOLO model to obtain the local focused image of the forage object.

[0011] In the above-mentioned method for identifying and collecting forage information based on machine vision, grass features are extracted from each pasture type image in the set of pasture type images and the local focused image of the forage object to obtain a set of pasture type image feature vectors and feature vectors of the image of the forage object to be detected, including: inputting each pasture type image in the set of pasture type images and the local focused image of the forage object into a forage image feature extractor based on a deep separable convolutional neural network model to obtain a set of pasture type image feature vectors and feature vectors of the image of the forage object to be detected.

[0012] In the above-mentioned machine vision-based forage information identification and collection method, the significant features of the set of feature vectors of the pasture type image are extracted as a prompt template, including: constructing a key matrix based on the set of feature vectors of the pasture type image; extracting the maximum value of each key vector in the key matrix to obtain a significant feature vector of the pasture type image as the prompt template.

[0013] In the above-mentioned machine vision-based forage information identification and collection method, a key matrix is ​​constructed based on the set of the forage type image feature vectors, including: using a key embedding matrix to linearly embed code each forage type image feature vector in the set of the forage type image feature vectors to obtain a set of linearly transformed forage type image feature vectors; using the linearly transformed forage type image feature vectors as the key vectors, and arranging the set of linearly transformed forage type image feature vectors in a matrix to obtain the key matrix.

[0014] In the above-mentioned machine vision-based forage information identification and collection method, based on the prompt template, the set of the image feature vectors of the forage object to be detected and the image feature vectors of the forage type are subjected to cross-domain optimized query encoding based on significant feature guidance to obtain the query response optimized encoding vector of the forage object to be detected, including: using the query embedding matrix and the value embedding matrix to perform linear embedding encoding on the image feature vectors of the forage object to be detected respectively to obtain a value vector and a query vector; the query vector, the value vector, each key vector in the key matrix and the prompt template are respectively input into the heterogeneous converter structure based on template prompt optimization to obtain a set of cross-domain optimized query encoding feature vectors of the forage object to be detected; the positional mean vector of the set of cross-domain optimized query encoding feature vectors of the forage object to be detected is calculated to obtain the query response optimized encoding vector of the forage object to be detected.

[0015] In the above-mentioned machine vision-based forage information identification and collection method, the query vector, the value vector, each key vector in the key matrix and the prompt template are respectively input into a heterogeneous converter structure based on template prompt optimization to obtain a set of cross-domain optimized query encoding feature vectors of the forage object to be detected, including: multiplying the query vector by the transposed vector of the key vector and then dividing it by the bi-norm of the prompt template to obtain an attention score matrix; multiplying the attention score matrix by the prompt template after passing it through a softmax function to obtain a template prompt optimized attention weight vector; calculating the position point multiplication between the template prompt optimized attention weight vector and the value vector to obtain the cross-domain optimized query encoding feature vector of the forage object to be detected.

[0016] In the above-mentioned machine vision-based forage information identification and collection method, the type label of the forage object is determined based on the optimized coding vector of the query response of the forage to be detected, including: inputting the optimized coding vector of the query response of the forage to be detected into a classifier-based forage identification module to obtain an identification result, and the identification result is used to represent the type label of the forage object.

[0017] According to another aspect of the present application, a forage information recognition and collection system based on machine vision is provided, which includes:

[0018] A target detection module is used to perform target detection on the grazing image to obtain a local focused image of the forage object;

[0019] A database query module, used for extracting a set of grassland type images labeled with grassland types from a database;

[0020] A feature extraction module, used for extracting grass features from each pasture type image in the pasture type image set and the pasture object local focus image to obtain a pasture type image feature vector set and a pasture object image feature vector to be detected;

[0021] A feature encoding module, used for performing query encoding on a set of the forage object image feature vector to be detected and the forage type image feature vector to obtain a query response optimized encoding vector of the forage to be detected;

[0022] The label generation module is used to determine the type label of the forage object based on the query response optimization encoding vector of the forage to be detected.

[0023] Compared with the prior art, the machine vision-based forage information recognition and collection system and method provided by the present application uses deep learning-based image processing technology to perform target detection on sheep feeding images to focus on forage objects. At the same time, a collection of forage type images marked with forage types is extracted from the database, and the image features of each forage type image and the forage object to be detected are extracted respectively. Then, the image features of the forage object to be detected and the image features of each forage type are coded for query response to intelligently identify the type of forage object. In this way, the generalization ability of the model can be effectively improved, the diversity of forage under different environments and conditions can be better coped with, and the accuracy and efficiency of forage information recognition can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0025] Figure 1 This is a flow chart of a method for identifying and collecting forage information based on machine vision according to an embodiment of the present application.

[0026] Figure 2 This is a data flow diagram of a method for identifying and collecting forage information based on machine vision according to an embodiment of the present application.

[0027] Figure 3 This is a flowchart of sub-step S4 of the method for identifying and collecting forage information based on machine vision according to an embodiment of the present application.

[0028] Figure 4 It is a block diagram of a forage information recognition and collection system based on machine vision according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0030] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0031] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0032] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0033] As mentioned in the background technology, patent CN118230419A proposes a method for identifying the grass-eating information of grazing sheep, which includes: wearing a wearable back clip on the front of the sheep to cover the front chest of the sheep; fixing the camera protective cover on the front chest of the sheep through the wearable back clip; placing a high-definition anti-shake camera in the camera protective cover to capture a complete picture of the sheep when eating; processing the captured eating image based on the recognition algorithm to identify the grass-eating information of the sheep.

[0034] Although the above method can automatically collect and process data and reduce manual intervention, the accuracy of forage information recognition still needs to be improved. Specifically, although this method can realize the recognition of forage species in specific scenarios, the generalization ability of the model still needs to be further verified, and a large amount of training data support is required to enhance the robustness and generalization ability of the model. In addition, this method introduces sound signals as auxiliary information when identifying forage species, but in an open natural environment, there is a lot of background noise, which may affect the extraction and analysis of sound signals, and thus affect the final recognition results. Moreover, the contribution of sound signals to forage information recognition is relatively low, and data processing and analysis require additional computing resources, which further increases the complexity and cost of the system.

[0035] In response to the above technical problems, this application proposes a forage information recognition and collection method based on machine vision, which uses deep learning-based image processing technology to perform target detection on sheep feeding images to focus on forage objects. At the same time, a collection of forage type images marked with forage types is extracted from the database, and the image features of each forage type image and the forage object to be detected are extracted respectively. Then, the image features of the forage object to be detected and the image features of each forage type are coded for query response to intelligently identify the type of forage object. In this way, the generalization ability of the model can be effectively improved, the diversity of forage under different environments and conditions can be better coped with, and the accuracy and efficiency of forage information recognition can be further improved.

[0036] Figure 1 This is a flow chart of a method for identifying and collecting forage information based on machine vision according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the method for recognizing and collecting forage information based on machine vision according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the machine vision-based forage information recognition and collection method of the embodiment of the present application, the steps include: S1, performing target detection on the grazing image to obtain a local focused image of the forage object; S2, extracting a set of forage type images marked with forage types from a database; S3, performing forage feature extraction on each forage type image in the set of forage type images and the local focused image of the forage object to obtain a set of forage type image feature vectors and a feature vector of the image of the forage object to be detected; S4, performing query encoding on the feature vector of the image of the forage object to be detected and the set of the forage type image feature vector to obtain an optimized encoding vector for query response of forage to be detected; S5, determining the type label of the forage object based on the optimized encoding vector for query response of the forage to be detected.

[0037] In the above-mentioned method for identifying and collecting grass information based on machine vision, the step S1 performs target detection on the grazing image to obtain a local focused image of the grass object. It should be understood that considering that the original grazing image is a complex scene containing a large amount of background information, the sheep body and grass information in the image are mixed, and directly extracting and identifying features from the entire grazing image not only consumes a lot of computing resources, but is also easily interfered. Therefore, in order to improve the accuracy of grass type recognition, the present application uses a grass object target detection network based on the YOLO model to process the grazing image, so as to utilize the powerful target detection capability of the YOLO model to quickly locate and crop out the part of the image related to the grass, generate a local focused image of the grass object, thereby reducing background interference and improving data processing efficiency. It is worth mentioning that YOLO is a popular real-time target detection system that converts the target detection task into a regression problem, by dividing the input image into grids, and predicting the bounding box and its category probability in each grid, so as to intelligently identify the target object in the image.

[0038] In the above-mentioned method for identifying and collecting forage information based on machine vision, the step S2 extracts a set of forage type images labeled with forage types from the database. That is, considering that in practical applications, it is necessary to identify multiple different types of forage. Therefore, in order to improve the versatility and generalization ability of the forage recognition algorithm to support larger-scale forage recognition tasks, the present application constructs a forage type image database containing multiple forage types. By extracting a set of forage type images labeled with forage types from the database, rich reference data is provided for subsequent forage recognition.

[0039] In the above-mentioned method for identifying and collecting forage information based on machine vision, in step S3, forage features are extracted for each forage type image in the set of forage type images and the local focused image of the forage object to obtain a set of forage type image feature vectors and a feature vector of the forage object image to be detected. It should be understood that in order to fully extract the forage information in the local focused image of the forage object and each forage type image to achieve query matching between the forage object and the forage type image, the present application further adopts a deep separable convolutional neural network model to construct a forage image feature extractor, and processes each forage type image in the set of forage type images and the local focused image of the forage object to capture the shape, color, texture and other detailed features of the forage in the image, thereby generating a set of forage type image feature vectors and a feature vector of the forage object image to be detected. It is worth mentioning that the deep separable convolutional neural network model is an efficient convolutional neural network structure. By decomposing the standard convolution operation into deep convolution and point-by-point convolution, it not only significantly reduces the number of parameters and computational complexity of the model, but also maintains a high feature extraction capability. It is very suitable for processing and analysis of large-scale image data.

[0040] In the above-mentioned machine vision-based forage information identification and collection method, the step S4 performs query encoding on the set of the forage object image feature vector to be detected and the forage type image feature vector to obtain the query response optimized encoding vector of the forage to be detected. It should be understood that in order to further improve the accuracy and efficiency of recognition, the present application proposes a cross-domain optimized query encoding method based on significant feature guidance, which is guided by the significant features in the set of the forage type image feature vector to strengthen the feature part of the forage object image feature vector to be detected that is significantly related to the forage type image, thereby optimizing the query process and improving the efficiency and accuracy of query encoding. It is worth mentioning that significant feature guidance is a technology for improving model performance in computer vision tasks, especially in tasks such as significant target detection, image segmentation, and target detection.

[0041] Figure 3 FIG. 4 is a flowchart of sub-step S4 of the method for recognizing and collecting forage information based on machine vision according to an embodiment of the present application. Figure 3 As shown, the step S4, query encoding the set of the image feature vectors of the forage object to be detected and the pasture type image feature vectors, includes: S41, extracting the significant features of the set of the pasture type image feature vectors as a prompt template; S42, based on the prompt template, performing cross-domain optimized query encoding guided by significant features on the set of the image feature vectors of the forage object to be detected and the pasture type image feature vectors to obtain the query response optimized encoding vector of the forage to be detected.

[0042] Specifically, the step S41 includes: constructing a key matrix based on the set of the pasture-type image feature vectors; extracting the maximum value of each key vector in the key matrix to obtain a pasture-type image salient feature vector as the prompt template. Wherein, constructing a key matrix based on the set of the pasture-type image feature vectors includes: using a key embedding matrix to linearly embed code each pasture-type image feature vector in the set of the pasture-type image feature vectors to obtain a set of pasture-type image feature vectors after linear transformation; using the pasture-type image feature vector after linear transformation as the key vector, and arranging the set of pasture-type image feature vectors after linear transformation into a matrix to obtain the key matrix.

[0043] Specifically, the step S42 includes: using a query embedding matrix and a value embedding matrix to perform linear embedding coding on the image feature vector of the forage object to be detected to obtain a value vector and a query vector; inputting the query vector, the value vector, each key vector in the key matrix and the prompt template into a heterogeneous converter structure based on template prompt optimization to obtain a set of cross-domain optimized query coding feature vectors of the forage object to be detected; calculating the positional mean vector of the set of cross-domain optimized query coding feature vectors of the forage object to be detected to obtain the query response optimized coding vector of the forage object to be detected. Among them, the query vector, the value vector, each key vector in the key matrix and the prompt template are respectively input into a heterogeneous converter structure based on template prompt optimization to obtain a set of cross-domain optimized query encoding feature vectors of the forage object to be detected, including: multiplying the query vector by the transposed vector of the key vector and then dividing it by the bi-norm of the prompt template to obtain an attention score matrix; multiplying the attention score matrix by the prompt template after passing it through a softmax function to obtain a template prompt optimized attention weight vector; calculating the position point multiplication between the template prompt optimized attention weight vector and the value vector to obtain the cross-domain optimized query encoding feature vector of the forage object to be detected.

[0044] Accordingly, the set of the grassland type image feature vectors and the feature vector of the to-be-detected grassland object image are processed by the following query encoding formula to obtain the to-be-detected grassland query response optimized encoding vector, wherein the query encoding formula is:

[0045] K={k1,k2,...,k n}

[0046] M o ={k1′; k2′; ...; k n ′}

[0047] k i ′=f(k i ,Wk ) = k i W k +b k

[0048] v tip ={max(k1′);max(k2′);...;max(k n ′)}

[0049] v q =x1W q +b q

[0050] v v =x1W v +b v

[0051]

[0052] Where K represents the set of grassland type image feature vectors, k1, k2, k i and k n are the first, second, i-th and n-th pasture type image feature vectors in the set of pasture type image feature vectors, respectively, and the value of n is the number of pasture type image feature vectors, W k , W q and W v denote the key embedding matrix, query embedding matrix and value embedding matrix respectively, and b k , b q and b v Represent different bias terms, M o represents the key matrix, k1', k2', k i ' and k n ' are the first, second, i-th and n-th key vectors in the key matrix, i.e., the grassland type image feature vector after the linear transformation, max(·) represents the maximum value function, v tip represents the prompt template, x1 represents the image feature vector of the forage object to be detected, and v q and v v denote the query vector and value vector respectively, (·) T represents the transpose of the vector, ||·||2 represents the bi-norm of the vector, and softmax is the normalized exponential function. represents matrix multiplication operation, ⊙ represents point multiplication by position, v pi represents the i-th cross-domain optimization query encoding feature vector of the forage object to be detected in the set of cross-domain optimization query encoding feature vectors of the forage object to be detected, v p Represents the optimized encoding vector of the query response of the forage to be detected.

[0053] That is, first, each pasture type image feature vector in the set of pasture type image feature vectors is linearly transformed to map it to a new latent feature space so that it is suitable for the subsequent query encoding process. Then, each pasture type image feature vector after linear transformation is used as a key vector and arranged into a key matrix to facilitate subsequent parallel processing. Then, the maximum value of each key vector is selected from the key matrix to combine into a pasture type image significant feature vector to capture important pasture image feature information, and use it as a prompt template to guide the subsequent query encoding process to focus more on the key feature parts of each pasture type image to reduce noise interference. Then, the query embedding matrix and the value embedding matrix are respectively applied to the feature vector of the forage object image to be detected to generate the corresponding query vector and value vector, and the query vector, the value vector, each key vector in the key matrix and the prompt template are interactively encoded through a specially designed heterogeneous converter structure. By introducing an additional prompt template in the traditional attention mechanism to automatically adjust the weight allocation strategy, not only the correlation between each pasture type image feature and the pasture object image feature is considered, but also the overall characteristics of the pasture type images in the database are considered, which helps to generate more accurate and relevant query outputs. Finally, the cross-domain optimized query results of the forage object image feature to be detected and the pasture type image feature are fused by position mean to integrate all pasture type image query information and generate the query response optimized encoding vector of the forage to be detected, thereby effectively mining the feature correlation distribution information between the pasture object and each pasture type image.

[0054] In the above-mentioned forage information identification and collection method based on machine vision, the step S5 determines the type label of the forage object based on the optimized coding vector of the query response of the forage to be detected. That is, the optimized coding vector of the query response of the forage to be detected is input into the forage identification module based on the classifier for classification and identification, so as to utilize the powerful classification mapping ability of the classifier model to perform feature analysis and classification judgment on the optimized coding vector of the query response of the forage to be detected, thereby intelligently determining the type of the forage object. It is worth mentioning that the classifier is an algorithm or model used in the field of machine learning to classify data into known categories, which adjusts its internal parameters through back propagation and optimization algorithms (such as Adam, SGD, etc.) to improve the accuracy of classification judgment. In the actual application stage, the optimized coding vector of the query response of the forage to be detected is input into the trained classifier, and the classifier can parse and classify it according to the learned classification rules to determine the most likely type of forage object under the current conditions.

[0055] In particular, considering that the image feature vector of the forage object to be detected and the set of the forage type image feature vectors respectively represent the image semantic features of the forage object to be detected and the image semantic features of each forage type image in the set of the forage type images, when performing query encoding optimization using the key matrix salient features as the prompt template, the image source domain semantic differences will also make the image semantic query response representation of the forage query response optimization encoding vector to be detected have feature distribution diversity based on different query attention weight distribution structures, thereby expecting to improve its classification regression convergence and generalization effects under complex spatial structures. Therefore, when the forage query response optimization encoding vector to be detected is input into the forage recognition module based on the classifier for classification probability regression, it will affect the accuracy of the classification result.

[0056] Preferably, in an example of the present application, when the optimized encoding vector of the query response to be detected forage grass is input into a forage identification module based on a classifier for classification probability regression, the optimized encoding vector of the query response to be detected forage grass is first optimized, and the optimization process specifically includes:

[0057] Determine the mean and variance of the feature set composed of the feature values ​​of the forage query response optimization coding vector to be detected, and divide the mean by the variance to obtain the forage query response optimization coding probability weight value to be detected:

[0058]

[0059] Among them, μ and σ 2 represent the mean and variance of the feature set composed of the feature values ​​of the optimized coding vector of the query response to be detected grass, respectively, and θ represents the probability weight value of the optimized coding of the query response to be detected grass;

[0060] The optimized encoding vector of the query response to be detected is multiplied by the inverse of the maximum eigenvalue of the optimized encoding vector of the query response to be detected to obtain the optimized encoding probability constraint vector of the query response to be detected:

[0061] v1=V⊙v max -1

[0062] Among them, v max -1 represents the inverse of the maximum eigenvalue of the query response optimization coding vector of the forage to be detected, V represents the query response optimization coding vector of the forage to be detected, ⊙ represents the point multiplication by position, and V1 represents the query response optimization coding probability constraint vector of the forage to be detected;

[0063] The optimized coding probability constraint vector of the query response to be detected forage is added with the optimized coding probability weight value of the query response to be detected forage, and the logarithmic value with base 2 is calculated to obtain the optimized coding information interaction vector of the query response to be detected forage:

[0064] V2=log2(V1⊕θ)

[0065] Among them, V1 represents the probability constraint vector of the optimized coding of the query response to be detected forage grass, θ represents the weight value of the optimized coding of the query response to be detected forage grass, ⊕ represents adding by position point, and V2 represents the optimized coding information interaction vector of the query response to be detected forage grass;

[0066] After subtracting the query response optimization coding probability constraint vector of the forage to be detected from the query response optimization coding probability weight value of the forage to be detected, the inverse of each eigenvalue is calculated to obtain the query response optimization coding sequence constraint vector of the forage to be detected:

[0067]

[0068] Among them, V1 represents the probability constraint vector of the optimized coding of the query response to be detected forage grass, θ represents the weight value of the optimized coding probability of the query response to be detected forage grass, represents point-by-point subtraction, V3 represents the query response optimization encoding sequence constraint vector for the forage to be detected;

[0069] Finally, the query response optimization coding information interaction vector of the forage to be detected is added with the query response optimization coding sequence constraint vector of the forage to be detected to obtain an optimized query response optimization coding vector of the forage to be detected.

[0070] That is, considering that when the weight matrix during classification probability regression acts on the optimized coding vector of the query response to be detected, the distribution diversity of the optimized coding vector of the query response to be detected causes the uncertainty of the weight stimulus parameters of the weight matrix, which makes the posterior probability density inference of the optimized coding vector of the query response to be detected to be missing through the weight matrix, which will affect the accuracy of the probability regression result. The above optimization process simulates the mesoscale interaction structure under probability constraints between the eigenvalue scale and the eigenvector scale of the optimized coding vector of the query response to be detected through the probability statistical characteristics of the optimized coding vector of the query response to be detected, so as to construct a bidirectional hidden variable motif based on the mesoscale short sequence relative to the probability statistical value to perform mesoscale bidirectional migration, and perform posterior recovery based on the sequence constraints on the interactive information, so as to improve the convergence effect of the optimized coding vector of the query response to be detected in the probability density domain of the optimized coding vector of the query response to be detected, and improve the accuracy of the recognition result obtained by inputting the optimized coding vector of the query response to be detected into the grass recognition module based on the classifier.

[0071] In summary, the forage information recognition and collection method based on machine vision according to the embodiment of the present application is explained, which uses deep learning-based image processing technology to perform target detection on the sheep's feeding images to focus on the forage object. At the same time, a collection of forage type images marked with forage types is extracted from the database, and the image features of each forage type image and the forage object to be detected are extracted respectively. Then, the image features of the forage object to be detected and the image features of each forage type are queried and coded to intelligently identify the type of the forage object. In this way, the generalization ability of the model can be effectively improved, the diversity of forage under different environments and conditions can be better coped with, and the accuracy and efficiency of forage information recognition can be further improved.

[0072] Furthermore, a forage information recognition and collection system based on machine vision is also provided.

[0073] Figure 4 FIG. 1 is a block diagram of a forage information recognition and collection system based on machine vision according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the forage information recognition and collection system 100 based on machine vision includes: a target detection module 110, which is used to perform target detection on the forage image to obtain a local focused image of the forage object; a database query module 120, which is used to extract a set of forage type images marked with forage types from the database; a feature extraction module 130, which is used to perform forage feature extraction on each forage type image and the local focused image of the forage object in the set of forage type images to obtain a set of forage type image feature vectors and a feature vector of the forage object image to be detected; a feature encoding module 140, which is used to query and encode the set of the forage object image feature vector to be detected and the forage type image feature vector to obtain a query response optimized encoding vector of the forage to be detected; a label generation module 150, which is used to determine the type label of the forage object based on the query response optimized encoding vector of the forage to be detected.

[0074] Here, those skilled in the art can understand that the specific operations of each module in the forage information recognition and collection system based on machine vision have been referred to above. Figures 1 to 3 The description of the machine vision-based forage information recognition and collection method has been introduced in detail, and therefore, its repeated description will be omitted.

[0075] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0076] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0077] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0078] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0079] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for identifying and collecting forage information based on machine vision, comprising: Wear the back clip on the front of the sheep, covering the front chest area of ​​the sheep; Fix the camera protective cover on the sheep's chest through the wearing back clip; A high-definition anti-shake camera is placed in a camera protective cover to capture a complete picture of sheep feeding; the captured feeding image is processed based on a recognition algorithm to identify the feeding grass information of the sheep, which is characterized in that the captured feeding image is processed based on a recognition algorithm to identify the feeding grass information of the sheep, including: Performing target detection on the grazing image to obtain a local focused image of the forage object; Extracting a set of grassland type images labeled with grassland types from a database; Extracting grass features from each pasture type image in the pasture type image set and the pasture object local focus image to obtain a pasture type image feature vector set and a pasture object image feature vector to be detected; Performing query coding on the set of the forage object image feature vector to be detected and the forage type image feature vector to obtain a query response optimized coding vector of the forage to be detected; Determine the type label of the forage object based on the optimized encoding vector of the forage query response to be detected; Among them, query encoding is performed on the set of image feature vectors of the forage object to be detected and the set of image feature vectors of the pasture type, including: extracting the significant features of the set of image feature vectors of the pasture type as a prompt template; based on the prompt template, cross-domain optimized query encoding guided by significant features is performed on the set of image feature vectors of the forage object to be detected and the set of image feature vectors of the pasture type to obtain the query response optimized encoding vector of the forage to be detected.

2. The method for identifying and collecting forage information based on machine vision according to claim 1, characterized in that: Performing target detection on the grazing image to obtain a local focused image of the forage object includes: The grazing image is input into a forage object target detection network based on the YOLO model to obtain a local focused image of the forage object.

3. The method for identifying and collecting forage information based on machine vision according to claim 2, characterized in that: Extracting grass features from each pasture type image in the pasture type image set and the pasture object local focus image to obtain a pasture type image feature vector set and a pasture object image feature vector to be detected, including: Each pasture type image in the set of pasture type images and the local focused image of the pasture object are respectively input into a pasture image feature extractor based on a deep separable convolutional neural network model to obtain a set of pasture type image feature vectors and a feature vector of the pasture object image to be detected.

4. The method for identifying and collecting forage information based on machine vision according to claim 3 is characterized in that: Extracting the significant features of the set of grassland type image feature vectors as a prompt template includes: constructing a key matrix based on a set of grassland type image feature vectors; The maximum value of each key vector in the key matrix is ​​extracted to obtain a significant feature vector of the grassland type image as the prompt template.

5. The method for identifying and collecting forage information based on machine vision according to claim 4, characterized in that: Constructing a key matrix based on the set of grassland type image feature vectors, including: Using a key embedding matrix, linear embedding coding is performed on each pasture type image feature vector in the set of pasture type image feature vectors to obtain a set of pasture type image feature vectors after linear transformation; The grassland type image feature vector after the linear transformation is used as the key vector, and the set of grassland type image feature vectors after the linear transformation is arranged in a matrix to obtain the key matrix.

6. The method for identifying and collecting forage information based on machine vision according to claim 5, characterized in that: Based on the prompt template, a set of the forage object image feature vector to be detected and the forage type image feature vector is subjected to cross-domain optimized query encoding guided by significant features to obtain the forage query response optimized encoding vector to be detected, including: Using a query embedding matrix and a value embedding matrix, linear embedding coding is performed on the feature vector of the to-be-detected grass object image to obtain a value vector and a query vector respectively; The query vector, the value vector, each key vector in the key matrix and the prompt template are respectively input into a heterogeneous converter structure based on template prompt optimization to obtain a set of cross-domain optimized query encoding feature vectors of the forage object to be detected; The position mean vector of the set of cross-domain optimized query coding feature vectors of the forage object to be detected is calculated to obtain the optimized coding vector of the query response of the forage object to be detected.

7. The method for identifying and collecting forage information based on machine vision according to claim 6, characterized in that: The query vector, the value vector, each key vector in the key matrix and the prompt template are respectively input into a heterogeneous converter structure based on template prompt optimization to obtain a set of cross-domain optimized query encoding feature vectors of the forage object to be detected, including: Multiplying the query vector by the transposed vector of the key vector and dividing by the two-norm of the prompt template to obtain an attention score matrix; The attention score matrix is ​​multiplied by the prompt template after passing it through a softmax function to obtain a template prompt optimized attention weight vector; The template prompt optimized attention weight vector and the value vector are calculated by multiplying the positions by points to obtain the cross-domain optimized query encoding feature vector of the forage object to be detected.

8. The method for identifying and collecting forage information based on machine vision according to claim 7, characterized in that: Determining the type label of the forage object based on the query response optimization encoding vector of the forage to be detected includes: The optimized encoding vector of the query response to the grass to be detected is input into a grass recognition module based on a classifier to obtain a recognition result, and the recognition result is used to represent the type label of the grass object.

9. A forage information recognition and collection system based on machine vision, characterized in that: include: A target detection module is used to perform target detection on the grazing image to obtain a local focused image of the forage object; A database query module, used for extracting a set of grassland type images labeled with grassland types from a database; A feature extraction module, used for extracting grass features from each pasture type image in the pasture type image set and the pasture object local focus image to obtain a pasture type image feature vector set and a pasture object image feature vector to be detected; A feature encoding module, used for performing query encoding on a set of the forage object image feature vector to be detected and the forage type image feature vector to obtain a query response optimized encoding vector of the forage to be detected; The label generation module is used to determine the type label of the forage object based on the query response optimization encoding vector of the forage to be detected.

Images