Image matching method and device, equipment and storage medium

Abstract

The invention discloses an image matching method and device, equipment and a storage medium, which are applied to the field of plant disease and insect pest analysis, and the method comprises the steps: segmenting a visible light image to obtain a mean segmented image; performing hierarchical division on the plant temperature based on the infrared image, and constructing a temperature hierarchical list; constructing a temperature hierarchy operator, and inputting the upper and lower limits of the temperature threshold of each temperature hierarchy in the temperature hierarchy list into the temperature hierarchy operator in sequence to obtain a homography matrix of matching points between the infrared image and the mean segmentation image under each temperature hierarchy; performing perspective transformation on the infrared image based on the homography matrix according to the sequence of the temperature levels to obtain a transformed image; and sequentially updating the transformation images under all the temperature levels to the infrared image to obtain an image matching result. The infrared image is divided into a plurality of levels from the temperature angle by using the operator, and the infrared image is gradually stretched and adjusted for matching, so that accurate matching of the image can be realized without training samples, and the matching efficiency and accuracy of the image are improved.

Description

Technical Field

[0001] The present invention relates to the field of plant disease and insect pest analysis, and in particular to an image matching method, an image matching device, an electronic device and a computer-readable storage medium. Background Art

[0002] Infrared images can be used to obtain the temperature of specific areas on visible light images of plants. These temperature data can better reflect the growth status of plants, pests and diseases, and the growth trends of key tissues, which is very important for controlling the growth status of rice grains, accurately monitoring pests and diseases, and predicting yields. Since the resolution, angle, and distortion of infrared sensors and visible light sensors are different, the position, direction, and size are greatly misaligned after shooting, and they need to be matched for effective processing. Therefore, it is very necessary to carry out infrared and visible light image matching to improve the level of modern monitoring and management of the planting industry.

[0003] At present, deep learning methods are generally used in the matching of infrared and visible light images, and a neural network architecture is introduced to match images. Deep learning methods require a large amount of sample support. The texture of rice is complex, and it is difficult to correctly detect the significant feature points of the plant. When the sample feature points are complex, the deep learning algorithm cannot work. In addition, the deep learning model may perform well on a specific data set, but its generalization ability may be limited when faced with new or different seasons, varieties, and lighting conditions of rice vegetation, resulting in a significant decrease in matching performance. Summary of the invention

[0004] The purpose of the present invention is to provide an image matching method, device, equipment and storage medium, which are applied to the field of plant disease and pest analysis. The method establishes a temperature level operator, uses the operator to divide the infrared image into multiple levels from the temperature perspective, and gradually stretches and adjusts the infrared image for matching. Accurate image matching can be achieved without training samples, thereby improving the image matching efficiency and accuracy.

[0005] In order to solve the above technical problems, the present invention provides an image matching method, comprising:

[0006] Obtaining a visible light image and an infrared image of a target plant planting area, and performing image segmentation on the visible light image based on a clustering algorithm to obtain a mean segmentation image;

[0007] Based on the infrared image, the plant temperature is divided into levels to construct a temperature level list;

[0008] Constructing a temperature level operator, inputting the upper and lower limits of the temperature thresholds of each temperature level in the temperature level list into the temperature level operator in turn, and obtaining a homography matrix of matching points between the infrared image and the mean segmentation image at each temperature level;

[0009] Performing perspective transformation on the infrared image based on the homography matrix in the order of the temperature levels to obtain a transformed image;

[0010] The transformed images at all the temperature levels are sequentially updated to the infrared image to obtain an image matching result.

[0011] Optionally, the temperature level operator is constructed, and the upper and lower limit thresholds of each level in the temperature level list are sequentially input into the temperature level operator to obtain the homography matrix of the matching points between the infrared image and the mean segmentation image at the current temperature level, including:

[0012] Constructing the temperature level operator; the input of the temperature level operator is the lower limit of the temperature threshold and the upper limit of the temperature threshold, and the output is the homography matrix;

[0013] Inputting the lower limit of the temperature threshold and the upper limit of the temperature threshold of each level in the temperature level list into the temperature level operator in sequence;

[0014] Based on the temperature level operator, the values ​​of the pixels in the infrared image that are less than the lower limit of the temperature threshold and greater than the upper limit of the temperature threshold are set as the minimum infrared pixel value to obtain the infrared image to be calculated;

[0015] Based on the temperature level operator, the SIFT algorithm is called to match the mean segmented image and the infrared image to be calculated, so as to obtain a first key point list of the visible light image and a second key point list of the infrared image to be calculated;

[0016] Based on the temperature level operator, a brute force matching algorithm is called to determine matching point pairs in the first key point list and the second key point list, and the matching point pairs are stored in the temperature level operator matching point list;

[0017] The RANSAC algorithm is called based on the temperature level operator to determine the homography matrix between the matching point pairs in the matching point list of the temperature level operator.

[0018] Optionally, the dividing the plant temperature into levels based on the infrared image to construct a temperature level list includes:

[0019] Constructing a temperature level list; each element of the temperature level list includes two fields, namely, an upper temperature threshold and a lower temperature threshold;

[0020] Assigning the current high temperature value to be processed as the maximum infrared pixel value of the infrared image;

[0021] Determine the standard deviation of all pixel values ​​in the infrared image, and divide the standard deviation by 4 as the temperature span value;

[0022] The result of subtracting the temperature span value from the current high value of the temperature to be processed is used as the lower limit value of the temporary temperature, and the current high value of the temperature to be processed is used as the upper limit value of the temporary temperature;

[0023] Acquire all pixels in the infrared image whose pixel values ​​are less than or equal to the lower limit of the temporary storage temperature and greater than or equal to the upper limit of the temporary storage temperature as filtered pixels;

[0024] If the number of pixels after the screening is greater than the number of pixels in the infrared image divided by 4, the result of subtracting the temperature span value from the current high temperature value to be processed divided by 2 is used as the lower limit of the temporary temperature;

[0025] storing the temporary storage temperature upper limit value and the temporary storage temperature lower limit value as an element in the temperature level list;

[0026] Assigning the current high value of the temperature to be processed to the lower limit value of the temporary temperature;

[0027] If the current high temperature value to be processed is greater than the minimum infrared pixel value, the step of subtracting the temperature span value from the current high temperature value to be processed as the lower limit value of the temporary temperature is started again.

[0028] Optionally, performing image segmentation on the visible light image based on a clustering algorithm to obtain a mean segmented image includes:

[0029] Obtaining a maximum infrared pixel value, a minimum infrared pixel value, and a standard deviation of all pixel values ​​in the infrared image;

[0030] Subtracting the minimum infrared pixel value from the maximum infrared pixel value to obtain an infrared pixel value difference;

[0031] The infrared pixel value difference is divided by the standard deviation and multiplied by 4, and the result is rounded to an integer to obtain the number of segmentations;

[0032] The visible light image is segmented using a K-means clustering algorithm based on the number of segmentations to obtain the mean segmented image.

[0033] In order to solve the above technical problems, the present invention provides an image matching device, comprising:

[0034] The first module is used to obtain visible light images and infrared images of the target plant planting area, and perform image segmentation on the visible light images based on a clustering algorithm to obtain mean segmentation images;

[0035] The second module is used to classify the plant temperature based on the infrared image and construct a temperature level list;

[0036] The third module is used to construct a temperature level operator, and input the upper and lower limits of the temperature thresholds of each level in the temperature level list into the temperature level operator in turn to obtain the homography matrix of the matching points between the infrared image and the mean segmentation image at the current temperature level;

[0037] A fourth module is used to perform perspective transformation on the infrared image based on the homography matrix according to the order of the temperature levels to obtain a transformed image;

[0038] The fifth module is used to update the transformed images at all the temperature levels into the infrared image in sequence to obtain an image matching result.

[0039] Optionally, the third module includes:

[0040] The first unit is used to construct the temperature level operator; the input of the temperature level operator is the lower temperature threshold and the upper temperature threshold, and the output is the homography matrix;

[0041] A second unit is used to sequentially input the lower limit of the temperature threshold and the upper limit of the temperature threshold of each temperature level in the temperature level list into the temperature level operator;

[0042] The third unit is used to set the values ​​of the pixels in the infrared image that are less than the lower limit of the temperature threshold and greater than the upper limit of the temperature threshold as the minimum infrared pixel value based on the temperature level operator to obtain the infrared image to be calculated;

[0043] The fourth unit is used to call the SIFT algorithm based on the temperature level operator to match the mean segmentation image and the infrared image to be calculated, so as to obtain a first key point list of the visible light image and a second key point list of the infrared image to be calculated;

[0044] A fifth unit is configured to determine matching point pairs in the first key point list and the second key point list by calling a brute force matching algorithm based on the temperature level operator, and store the matching point pairs in the temperature level operator matching point list;

[0045] The sixth unit is used to call the RANSAC algorithm based on the temperature level operator to determine the homography matrix between the matching point pairs in the matching point list of the temperature level operator.

[0046] Optionally, the second module includes:

[0047] The seventh unit is used to construct a temperature level list; each element of the temperature level list includes two fields, namely, an upper temperature threshold and a lower temperature threshold;

[0048] An eighth unit is used to assign the current high-bit value of the temperature to be processed as the maximum infrared pixel value of the infrared image;

[0049] A ninth unit is used to determine a standard deviation of all pixel values ​​in the infrared image, and divide the standard deviation by 4 to obtain a temperature span value;

[0050] A tenth unit is used for subtracting the temperature span value from the current high value of the temperature to be processed as the lower limit value of the temporary temperature, and taking the current high value of the temperature to be processed as the upper limit value of the temporary temperature;

[0051] The eleventh unit is used to obtain all pixels in the infrared image whose pixel values ​​are less than or equal to the lower limit value of the temporary storage temperature and greater than or equal to the upper limit value of the temporary storage temperature as the filtered pixels;

[0052] The twelfth unit is used for, if the number of the screened pixels is greater than the number of pixels of the infrared image divided by 4, taking the result of subtracting the temperature span value divided by 2 from the current high temperature value to be processed as the lower limit value of the temporary temperature;

[0053] A thirteenth unit, used for storing the temporary storage temperature upper limit value and the temporary storage temperature lower limit value as an element in the temperature level list;

[0054] A fourteenth unit is used to assign the current high value of the temperature to be processed to the lower limit value of the temporary temperature;

[0055] The fifteenth unit is used to restart execution from the tenth unit if the current high temperature value to be processed is greater than the minimum infrared pixel value.

[0056] Optionally, the first module includes:

[0057] A sixteenth unit is used to obtain a maximum infrared pixel value, a minimum infrared pixel value and a standard deviation of all pixel values ​​in the infrared image;

[0058] A seventeenth unit is used to obtain an infrared pixel value difference by subtracting the minimum infrared pixel value from the maximum infrared pixel value;

[0059] An eighteenth unit is used for rounding off the result of dividing the infrared pixel value difference by the standard deviation and multiplying by 4 to obtain the number of divisions;

[0060] The nineteenth unit is used to perform K-means clustering algorithm segmentation on the visible light image based on the number of segmentations to obtain the mean segmented image.

[0061] In order to solve the above technical problems, the present invention provides an electronic device, comprising:

[0062] Memory, for storing computer programs;

[0063] A processor is used to implement the above-mentioned image matching method when executing the computer program.

[0064] In order to solve the above technical problem, the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the above-mentioned image matching method is implemented.

[0065] It can be seen that the method of the present invention obtains visible light images and infrared images of the target plant planting area, performs image segmentation on the visible light image based on the clustering algorithm to obtain a mean segmentation image; divides the plant temperature into levels based on the infrared image, and constructs a temperature level list; constructs a temperature level operator, and inputs the upper and lower limits of the temperature threshold of each level in the temperature level list into the temperature level operator in turn, and obtains the homography matrix of the matching points between the infrared image and the mean segmentation image at the current temperature level; performs perspective transformation on the infrared image based on the homography matrix in the order of the temperature levels to obtain a transformed image; and updates the transformed images at all temperature levels to the infrared image in turn to obtain an image matching result.

[0066] The present invention provides an image matching method, which establishes a temperature level operator, utilizes the operator to divide the infrared image into multiple levels from the temperature perspective, and gradually stretches and adjusts the infrared image for matching. Accurate image matching can be achieved without the need for training samples, thereby improving the image matching efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0068] Figure 1 A flowchart of an image matching method provided by an embodiment of the present invention;

[0069] Figure 2 A structural block diagram of an image matching device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Infrared images can be used to obtain the temperature of specific areas on visible light images of plants. These temperature data can better reflect vegetation growth conditions, pests and diseases, and growth trends of key tissues. This is very important for controlling food growth conditions, accurately monitoring crop pests and diseases, and predicting crop yields.

[0072] The resolution, angle and distortion of infrared sensors and visible light sensors are different. After shooting, the position, direction and size are greatly misaligned, and they need to be matched for effective processing. Therefore, it is very necessary to carry out infrared and visible light image matching to improve the level of agricultural modernization monitoring and management.

[0073] The image matching method provided by the present invention can be used to match visible light images and infrared images of plants, wherein the plants can specifically refer to crops, such as rice.

[0074] There are two main methods for automatic infrared and visible light image matching. The first is the traditional feature-based matching method. First, a specific algorithm (such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF, an algorithm that integrates FAST key point detection and BRIEF descriptors)) is used in the source image and the target image to detect and describe the significant feature points. Then, a similarity metric (such as Euclidean distance, Hamming distance, etc.) is used to find the best match between the feature vectors of the source image and the target image to achieve image alignment. This method is easy to implement, but a key problem of this method is that the matching effect depends largely on the "matching points" of the matched scene. Since rice leaves and rice ears have many very similar "matching points" (such as the tip of the leaf), these matching points may have serious problems such as uneven matching density and matching errors. Therefore, this method is difficult to handle the infrared and visible light image matching of rice. The second category is deep learning methods, which introduce the convolutional neural network (CNN) architecture to match images. This type of method utilizes the powerful capabilities of CNN in image recognition and feature extraction, and can obtain better matching results. However, on the one hand, deep learning methods all require a large amount of sample support. The texture of plants is complex, and it is difficult to correctly detect the significant feature points of plants. Deep learning algorithms cannot function when the sample feature points are complex. On the other hand, the CNN model may perform well on specific data sets, but its generalization ability may be limited when faced with new or different seasons, varieties, and lighting conditions. Plants, resulting in a significant decrease in matching performance.

[0075] Faced with the problem of matching infrared and visible light images of plants with complex textures and key points, even experienced experts cannot find the right matching points in one go; finding the right matching points is a gradual and incremental process, so it is necessary to propose a method that can gradually improve the matching accuracy from easy-to-find matching points to fine-grained matching points step by step.

[0076] In view of the problems existing in the prior art, the present invention provides an image matching method, which constructs a temperature level operator. The operator can be used to divide the plant infrared image into multiple levels from the temperature perspective, and the infrared image is gradually stretched and adjusted for matching, forming a method specifically for matching plant infrared and visible light images.

[0077] By using the method of the present invention, the complex content matching of plant images of two different sensors can be transformed into a layered step-by-step matching process; the method can first match the rice ear part of the plant species with higher temperature and obvious directionality, so that the two images are initially matched; then gradually transition to the relatively complex leaves with relatively lower temperature, and finally achieve accurate matching; using the patent of the present invention helps to achieve accurate matching of plant infrared and visible light images, so that the farmland monitoring program can more accurately find the relative temperature of different components in the visible light image of the plant, and then obtain the plant grain growth status and pest and disease status, thereby improving agricultural production efficiency and the scientific nature of decision-making.

[0078] The following combination Figure 1 , Figure 1 A flowchart of an image matching method provided by an embodiment of the present invention, the method may include:

[0079] S101: Obtain visible light images and infrared images of the target plant planting area, and perform image segmentation on the visible light image based on a clustering algorithm to obtain a mean segmentation image.

[0080] This embodiment can obtain visible light images and infrared images of the target plant planting area. This embodiment is not limited to the specific type of plants, and generally can be rice, etc.

[0081] Furthermore, the present embodiment can perform image segmentation on the visible light image based on the clustering algorithm to obtain a mean-segmented image. The present embodiment does not limit the specific method of obtaining the mean-segmented image. Generally, the maximum infrared pixel value, the minimum infrared pixel value and the standard deviation of all pixel values ​​in the infrared image can be obtained; the maximum infrared pixel value is subtracted from the minimum infrared pixel value to obtain the infrared pixel value difference; the infrared pixel value difference is divided by the standard deviation and multiplied by 4, and the result is rounded to the integer to obtain the number of segmentations; based on the number of segmentations, the visible light image is segmented using the K-means clustering algorithm to obtain the mean-segmented image.

[0082] Specifically, this embodiment can obtain a visible light image KJImage and an infrared image HWImage of a target plant planting area (such as a regional rice field); KJImage comes from a visible light sensor and is a color image composed of three bands of red, green and blue; HWImage comes from an infrared sensor and is a black and white image composed of one band. Each pixel in the infrared image represents a relative temperature value, and the higher the value, the higher the temperature.

[0083] Furthermore, the maximum pixel value in the infrared image, that is, the maximum infrared pixel value HWImageMax, can be obtained;

[0084] Get the minimum pixel value in the infrared image, that is, the minimum infrared pixel value HWImageMin;

[0085] Get the pixel number HWNumber of the infrared image;

[0086] Get the standard deviation HWStd of all pixel values ​​in the infrared image;

[0087] The maximum infrared pixel value is subtracted from the minimum infrared pixel value to obtain the infrared pixel value difference; the infrared pixel value difference is divided by the standard deviation and multiplied by 4, and the result is rounded to the integer to obtain the number of segmentations KCluster. That is, KCluster = Round ((HWImageMax-HWImageMin) / HWStd×4), where Round is a function for rounding the decimal part.

[0088] KJImage is segmented based on the K-means clustering algorithm (K-Means). K-Means is a classic unsupervised learning algorithm that is widely used in data clustering, image segmentation, pattern recognition, etc. The number of clusters is KCluster, and the mean segmentation image MeansImage is obtained.

[0089] S102: Divide the plant temperature into levels based on the infrared image, and construct a temperature level list.

[0090] In this embodiment, plant temperatures can be classified into different levels based on infrared images, and a temperature level list can be constructed.

[0091] This embodiment does not limit the specific method of constructing the temperature level list, and generally a temperature level list can be constructed; each element of the temperature level list includes two fields, namely, an upper temperature threshold and a lower temperature threshold;

[0092] Assign the current high temperature value to be processed to the maximum infrared pixel value of the infrared image;

[0093] Determine the standard deviation of all pixel values ​​in the infrared image, and divide the standard deviation by 4 as the temperature span value;

[0094] The result of subtracting the temperature span value from the current high value of the temperature to be processed is used as the lower limit of the temporary temperature, and the current high value of the temperature to be processed is used as the upper limit of the temporary temperature;

[0095] Acquire all pixels in the infrared image whose pixel values ​​are less than or equal to the lower limit of the temporary temperature and greater than or equal to the upper limit of the temporary temperature as the filtered pixels;

[0096] If the number of pixels after screening is greater than the number of pixels in the infrared image divided by 4, the result of subtracting the temperature span value divided by 2 from the current high temperature value to be processed is used as the lower limit of the temporary temperature;

[0097] The temporary temperature upper limit value and the temporary temperature lower limit value are stored as an element in the temperature level list;

[0098] Assign the current high value of the temperature to be processed to the lower limit of the temporary temperature;

[0099] If the current high temperature value to be processed is greater than the minimum infrared pixel value, the process starts again from the step of subtracting the temperature span value from the current high temperature value to be processed as the lower limit of the temporary temperature.

[0100] Specifically, in this embodiment, an empty temperature level list CJPPList may be firstly established, and each element of CJPPList includes two fields, namely, a lower limit of a temperature threshold and an upper limit of a temperature threshold;

[0101] Set the current high value of the temperature to be processed WDGW, and assign the current high value of the temperature to be processed to the maximum infrared pixel value, that is, WDGW=HWImageMax;

[0102] Set the temperature span value WDKD, and assign the temperature span value to the standard deviation of all pixel values ​​in the infrared image divided by 4, that is, WDKD=HWStd / 4;

[0103] Set the lower limit value TempXX and the upper limit value TempSX of the temporary temperature; subtract the temperature span value from the current high value of the temperature to be processed as the lower limit value of the temporary temperature, that is, TempXX=WDGW-WDKD; and use the current high value of the temperature to be processed as the upper limit value of the temporary temperature, that is, TempSX=WDGW.

[0104] Further, all pixels in the infrared image whose pixel values ​​are less than or equal to the lower limit of the temporary temperature and greater than or equal to the upper limit of the temporary temperature are obtained as the filtered pixels;

[0105] Determine the number of pixels after screening. If the number of pixels after screening is greater than the number of pixels in the infrared image divided by 4, that is, greater than HWNumber / 4, then the result of subtracting the temperature span value divided by 2 from the current high value of the temperature to be processed is taken as the lower limit of the temporary temperature, that is, TempXX=WDGW-WDKD / 2; further, store the upper limit of the temporary temperature and the lower limit of the temporary temperature as an element in the temperature level list, the upper limit of the temperature threshold of the element is equal to TempXX, and the lower limit of the temperature threshold is equal to TempSX;

[0106] If the number of pixels after screening is less than or equal to the result of dividing the number of pixels in the infrared image by 4, that is, less than or equal to HWNumber / 4, the temporary temperature upper limit value and the temporary temperature lower limit value can be directly stored as an element in the temperature level list;

[0107] After the storage is completed, the current high value of the temperature to be processed is assigned to the lower limit of the temporary temperature, that is, WDGW = TempSX;

[0108] Further, determine whether WDGW is greater than HWImageMin;

[0109] If yes, it can be determined that the temperature level of the plant has not been divided. At this time, it is necessary to jump to the step of taking the result of subtracting the temperature span value from the current high temperature value to be processed as the lower limit of the temporary temperature and start execution, and execute in a loop according to the above process sequence;

[0110] If not, it can be determined that the temperature levels of the plant have been divided, and a complete temperature level list can be obtained.

[0111] S103: construct a temperature level operator, input the upper and lower limits of the temperature threshold of each temperature level in the temperature level list into the temperature level operator in turn, and obtain the homography matrix of the matching points between the infrared image and the mean segmentation image at each temperature level.

[0112] This embodiment can construct a temperature level operator, input the upper and lower limits of the temperature threshold of each temperature level in the temperature level list into the temperature level operator in turn, and obtain the homography matrix of the matching points between the infrared image and the mean segmentation image at each temperature level.

[0113] This embodiment does not limit the specific method of obtaining the homography matrix, and generally a temperature level operator can be constructed; the input of the temperature level operator is the lower limit of the temperature threshold and the upper limit of the temperature threshold, and the output is the homography matrix;

[0114] Input the lower limit of the temperature threshold and the upper limit of the temperature threshold of each level in the temperature level list into the temperature level operator in turn;

[0115] Based on the temperature level operator, the values ​​of the pixels in the infrared image that are less than the lower limit of the temperature threshold and greater than the upper limit of the temperature threshold are set as the minimum infrared pixel value to obtain the infrared image to be calculated;

[0116] Based on the temperature level operator, the SIFT algorithm is called to match the mean segmentation image and the infrared image to be calculated, and the first key point list of the visible light image and the second key point list of the infrared image to be calculated are obtained;

[0117] Based on the temperature level operator, a brute force matching algorithm is called to determine matching point pairs in the first key point list and the second key point list, and the matching point pairs are stored in the temperature level operator matching point list;

[0118] Based on the temperature level operator, the RANSAC (Random Sample Consensus) algorithm is called to determine the homography matrix between the matching point pairs in the matching point list of the temperature level operator.

[0119] Specifically, a temperature level operator CJOpt is established, and the input of CJOpt is the lower limit of the temperature threshold TempXX and the upper limit of the temperature threshold TempSX.

[0120] The built-in operation flow of the temperature level operator for TempXX and TempSX is:

[0121] All pixels in HWImage are processed, the pixel values ​​of pixels greater than or equal to TempXX and less than or equal to TempSX are kept unchanged, the pixel values ​​of other pixels are set to HWImageMin, and the processed results are stored in the infrared image to be calculated CJOptTempImage.

[0122] Use SIFT algorithm to match MeansImage and CJOptTempImage, and obtain the first key point list srcpts on KJImage and the second key point list dstpts on CJOptTempImage;

[0123] Use the Brute-Force Matching algorithm to obtain the most similar key point pairs in srcpts and dstpts as matching point pairs, and store the matching point pairs in the temperature level operator matching point list CJOptMatches;

[0124] Use the RANSAC algorithm to calculate the homography matrix CJOptMatrix between the matching point pairs corresponding to CJOptMatches;

[0125] Output CJOptMatrix as the result of CJOpt.

[0126] In this embodiment, the upper limit of the temperature threshold and the lower limit of the temperature threshold of each level in each temperature level list can be input into the temperature level operator in order to obtain the homography matrix corresponding to each level.

[0127] Specifically, in this embodiment, the matching counter PPCounter=1 may be set;

[0128] Input the lower temperature threshold and the upper temperature threshold of the PPCounterth element in the temperature level list into CJOpt, and obtain the output homography matrix CJOptMatrix of the PPCounterth temperature level;

[0129] Set PPCounter=PPCounter+1;

[0130] Determine whether PPCounter is less than or equal to the number of elements in CJPPList;

[0131] If yes, jump to the step of inputting the lower temperature threshold and the upper temperature threshold of the PPCounterth element in the temperature level list into CJOpt to start execution;

[0132] If not, it is determined that the upper and lower thresholds of all temperature levels have been input into CJOpt, and the calculation of the homography matrix corresponding to all levels is completed.

[0133] In this embodiment, when constructing the temperature level list, elements are stored in order from high to low temperature. Therefore, in this embodiment, the upper limit of the temperature threshold and the lower limit of the temperature threshold of each temperature level in each temperature level list can be input into the temperature level operator in order from high to low temperature.

[0134] S104: Performing perspective transformation on the infrared image based on the homography matrix in the order of the temperature levels to obtain a transformed image.

[0135] S105: Update the transformed images at all temperature levels to the infrared image in sequence to obtain image matching results.

[0136] In this embodiment, the infrared image can be perspective transformed based on the homography matrix in the order of temperature levels to obtain a transformed image, and the transformed images at all temperature levels can be updated to the infrared image in sequence to obtain an image matching result.

[0137] Specifically, when calculating the homography matrix at each temperature level, this embodiment can perform perspective transformation on all pixels of HWImage through the homography matrix to obtain a transformed image; and update the transformed images at all temperature levels to the infrared image in turn to obtain an image matching result.

[0138] An example of the overall process could be:

[0139] Set the match counter PPCounter=1;

[0140] Input the lower temperature threshold and the upper temperature threshold of the PPCounterth element in the temperature level list into CJOpt, and obtain the output homography matrix CJOptMatrix of the PPCounterth temperature level;

[0141] Set the mapping matrix PPMatrix, and assign the mapping matrix to the homography matrix CJOptMatrix of the PPCounterth temperature level, that is, PPMatrix=CJOptMatrix;

[0142] Perform perspective transformation on all pixels of HWImage through PPMatrix to obtain the transformed image BHResult;

[0143] Update the transformed image to the infrared image, that is, HWImage=BHResult;

[0144] Set PPCounter=PPCounter+1;

[0145] Determine whether PPCounter is less than or equal to the number of elements in CJPPList;

[0146] If yes, jump to the step of inputting the lower temperature threshold and the upper temperature threshold of the PPCounterth element in the temperature level list into CJOpt to start execution;

[0147] If not, it is determined that the upper and lower limits of the thresholds of all temperature levels have been input into CJOpt, and the calculation of the homography matrix corresponding to all levels is completed, and the final updated HWImage is obtained, which is the result of matching the infrared image with the visible light image.

[0148] This embodiment does not limit the specific application of the image matching result. In this embodiment, it can be applied to the growth status of rice grains and the status of pests and diseases, scientific farming, increased yields, and reduced economic losses.

[0149] Based on the above embodiments, the method of the present invention provides an image matching method, which establishes a temperature level operator, uses the operator to divide the infrared image into multiple levels from the temperature perspective, and gradually stretches and adjusts the infrared image for matching. Accurate image matching can be achieved without the need for training samples, thereby improving the image matching efficiency and accuracy.

[0150] The following combination Figure 2 , Figure 2 A structural block diagram of an image matching device provided by an embodiment of the present invention, the device may include:

[0151] The first module 100 is used to obtain a visible light image and an infrared image of a target plant planting area, and perform image segmentation on the visible light image based on a clustering algorithm to obtain a mean segmentation image;

[0152] The second module 200 is used to classify the plant temperature based on the infrared image and construct a temperature level list;

[0153] The third module 300 is used to construct a temperature level operator, and input the upper and lower limits of the temperature threshold of each temperature level in the temperature level list into the temperature level operator in turn to obtain the homography matrix of the matching points between the infrared image and the mean segmentation image at each temperature level;

[0154] A fourth module 400 is used to perform perspective transformation on the infrared image based on the homography matrix according to the order of the temperature levels to obtain a transformed image;

[0155] The fifth module 500 is used to update the transformed images at all the temperature levels into the infrared image in sequence to obtain an image matching result.

[0156] Based on the above embodiments, the present invention provides an image matching method, which establishes a temperature level operator, uses the operator to divide the infrared image into multiple levels from the temperature perspective, and gradually stretches and adjusts the infrared image for matching. Accurate image matching can be achieved without the need for training samples, thereby improving the image matching efficiency and accuracy.

[0157] Based on the above embodiment, the third module 300 may include:

[0158] The first unit is used to construct the temperature level operator; the input of the temperature level operator is the lower temperature threshold and the upper temperature threshold, and the output is the homography matrix;

[0159] A second unit is used to sequentially input the lower limit of the temperature threshold and the upper limit of the temperature threshold of each temperature level in the temperature level list into the temperature level operator;

[0160] The third unit is used to set the values ​​of the pixels in the infrared image that are less than the lower limit of the temperature threshold and greater than the upper limit of the temperature threshold as the minimum infrared pixel value based on the temperature level operator to obtain the infrared image to be calculated;

[0161] The fourth unit is used to call the SIFT algorithm based on the temperature level operator to match the mean segmentation image and the infrared image to be calculated, so as to obtain a first key point list of the mean segmentation image and a second key point list of the infrared image to be calculated;

[0162] A fifth unit is configured to determine matching point pairs in the first key point list and the second key point list by calling a brute force matching algorithm based on the temperature level operator, and store the matching point pairs in the temperature level operator matching point list;

[0163] The sixth unit is used to call the RANSAC algorithm based on the temperature level operator to determine the homography matrix between the matching point pairs in the matching point list of the temperature level operator.

[0164] Based on the above embodiments, the second module 200 may include:

[0165] The seventh unit is used to construct a temperature level list; each element of the temperature level list includes two fields, namely, an upper temperature threshold and a lower temperature threshold;

[0166] An eighth unit is used to assign the current high-bit value of the temperature to be processed as the maximum infrared pixel value of the infrared image;

[0167] A ninth unit is used to determine a standard deviation of all pixel values ​​in the infrared image, and divide the standard deviation by 4 to obtain a temperature span value;

[0168] A tenth unit is used for subtracting the temperature span value from the current high value of the temperature to be processed as the lower limit value of the temporary temperature, and taking the current high value of the temperature to be processed as the upper limit value of the temporary temperature;

[0169] The eleventh unit is used to obtain all pixels in the infrared image whose pixel values ​​are less than or equal to the lower limit value of the temporary storage temperature and greater than or equal to the upper limit value of the temporary storage temperature as screening pixels;

[0170] A twelfth unit is used for, if the number of the screened pixels is greater than the number of pixels of the infrared image divided by 4, taking the result of subtracting the temperature span value divided by 2 from the current high temperature value to be processed as the lower limit value of the temporary temperature;

[0171] A thirteenth unit, used for storing the temporary storage temperature upper limit value and the temporary storage temperature lower limit value as an element in the temperature level list;

[0172] A fourteenth unit is used to assign the current high value of the temperature to be processed to the lower limit value of the temporary temperature;

[0173] The fifteenth unit is used to restart execution from the tenth unit if the current high temperature value to be processed is greater than the minimum infrared pixel value.

[0174] Based on the above embodiments, the first module 100 may include:

[0175] A sixteenth unit is used to obtain a maximum infrared pixel value, a minimum infrared pixel value and a standard deviation of all pixel values ​​in the infrared image;

[0176] A seventeenth unit is used to obtain an infrared pixel value difference by subtracting the minimum infrared pixel value from the maximum infrared pixel value;

[0177] An eighteenth unit is used for rounding off the result of dividing the infrared pixel value difference by the standard deviation and multiplying by 4 to obtain the number of divisions;

[0178] The nineteenth unit is used to perform K-means clustering algorithm segmentation on the visible light image based on the number of segmentations to obtain the mean segmented image.

[0179] Based on the above embodiments, the present invention further provides an electronic device, which may include a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the steps provided in the above embodiments may be implemented. Of course, the device may also include various necessary network interfaces, power supplies, and other components.

[0180] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by an execution terminal or a processor, the method provided in the embodiment of the present invention can be implemented; the storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0181] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. An image matching method, characterized in that: include: Obtaining a visible light image and an infrared image of a target plant planting area, and performing image segmentation on the visible light image based on a clustering algorithm to obtain a mean segmentation image; Based on the infrared image, the plant temperature is divided into levels to construct a temperature level list; Constructing a temperature level operator, inputting the upper and lower limits of the temperature thresholds of each temperature level in the temperature level list into the temperature level operator in turn, and obtaining a homography matrix of matching points between the infrared image and the mean segmentation image at each temperature level; Performing perspective transformation on the infrared image based on the homography matrix in the order of the temperature levels to obtain a transformed image; The transformed images at all the temperature levels are sequentially updated to the infrared image to obtain an image matching result.

2. The image matching method according to claim 1, characterized in that: The temperature level operator is constructed, and the upper and lower limits of the temperature thresholds of each temperature level in the temperature level list are input into the temperature level operator in sequence to obtain the homography matrix of the matching points between the infrared image and the mean segmentation image at each temperature level, including: Constructing the temperature level operator; the input of the temperature level operator is the lower limit of the temperature threshold and the upper limit of the temperature threshold, and the output is the homography matrix; Inputting the lower temperature threshold and the upper temperature threshold of each temperature level in the temperature level list into the temperature level operator in sequence; Based on the temperature level operator, the values ​​of the pixels in the infrared image that are less than the lower limit of the temperature threshold and greater than the upper limit of the temperature threshold are set as the minimum infrared pixel value to obtain the infrared image to be calculated; Based on the temperature level operator, the SIFT algorithm is called to match the mean segmented image and the infrared image to be calculated, so as to obtain a first key point list of the visible light image and a second key point list of the infrared image to be calculated; Based on the temperature level operator, a brute force matching algorithm is called to determine matching point pairs in the first key point list and the second key point list, and the matching point pairs are stored in the temperature level operator matching point list; The RANSAC algorithm is called based on the temperature level operator to determine the homography matrix between the matching point pairs in the matching point list of the temperature level operator.

3. The image matching method according to claim 1, characterized in that: The step of dividing the plant temperature into different levels based on the infrared image and constructing a temperature level list includes: Constructing a temperature level list; each element of the temperature level list includes two fields, namely, an upper temperature threshold and a lower temperature threshold; Assigning the current high temperature value to be processed as the maximum infrared pixel value of the infrared image; Determine the standard deviation of all pixel values ​​in the infrared image, and divide the standard deviation by 4 as the temperature span value; The result of subtracting the temperature span value from the current high value of the temperature to be processed is used as the lower limit value of the temporary temperature, and the current high value of the temperature to be processed is used as the upper limit value of the temporary temperature; Acquire all pixels in the infrared image whose pixel values ​​are less than or equal to the lower limit of the temporary storage temperature and greater than or equal to the upper limit of the temporary storage temperature as filtered pixels; If the number of pixels after the screening is greater than the number of pixels in the infrared image divided by 4, the result of subtracting the temperature span value from the current high temperature value to be processed divided by 2 is used as the lower limit of the temporary temperature; storing the temporary temperature upper limit value and the temporary temperature lower limit value as an element in the temperature level list; Assigning the current high value of the temperature to be processed to the lower limit value of the temporary temperature; If the current high temperature value to be processed is greater than the minimum infrared pixel value, the step of subtracting the temperature span value from the current high temperature value to be processed as the lower limit value of the temporary temperature is started again.

4. The image matching method according to claim 1, characterized in that: The step of performing image segmentation on the visible light image based on a clustering algorithm to obtain a mean segmented image includes: Obtaining a maximum infrared pixel value, a minimum infrared pixel value, and a standard deviation of all pixel values ​​in the infrared image; Subtracting the minimum infrared pixel value from the maximum infrared pixel value to obtain an infrared pixel value difference; The infrared pixel value difference is divided by the standard deviation and multiplied by 4, and the result is rounded to an integer to obtain the number of segmentations; The visible light image is segmented using a K-means clustering algorithm based on the number of segmentations to obtain the mean segmented image.

5. An image matching device, characterized in that: include: The first module is used to obtain visible light images and infrared images of the target plant planting area, and perform image segmentation on the visible light images based on a clustering algorithm to obtain mean segmentation images; The second module is used to classify the plant temperature based on the infrared image and construct a temperature level list; The third module is used to construct a temperature level operator, and input the upper and lower limits of the temperature threshold of each temperature level in the temperature level list into the temperature level operator in turn to obtain the homography matrix of the matching points between the infrared image and the mean segmentation image at each temperature level; A fourth module is used to perform perspective transformation on the infrared image based on the homography matrix according to the order of the temperature levels to obtain a transformed image; The fifth module is used to update the transformed images at all the temperature levels into the infrared image in sequence to obtain an image matching result.

6. The image matching device according to claim 5, characterized in that: The third module comprises: The first unit is used to construct the temperature level operator; the input of the temperature level operator is the lower temperature threshold and the upper temperature threshold, and the output is the homography matrix; A second unit is used to sequentially input the lower limit of the temperature threshold and the upper limit of the temperature threshold of each temperature level in the temperature level list into the temperature level operator; The third unit is used to set the values ​​of the pixels in the infrared image that are less than the lower limit of the temperature threshold and greater than the upper limit of the temperature threshold as the minimum infrared pixel value based on the temperature level operator to obtain the infrared image to be calculated; The fourth unit is used to call the SIFT algorithm based on the temperature level operator to match the mean segmentation image and the infrared image to be calculated, so as to obtain a first key point list of the visible light image and a second key point list of the infrared image to be calculated; A fifth unit is configured to determine matching point pairs in the first key point list and the second key point list by calling a brute force matching algorithm based on the temperature level operator, and store the matching point pairs in the temperature level operator matching point list; The sixth unit is used to call the RANSAC algorithm based on the temperature level operator to determine the homography matrix between the matching point pairs in the matching point list of the temperature level operator.

7. The image matching device according to claim 5, characterized in that: The second module comprises: The seventh unit is used to construct a temperature level list; each element of the temperature level list includes two fields, namely, an upper temperature threshold and a lower temperature threshold; An eighth unit is used to assign the current high-bit value of the temperature to be processed as the maximum infrared pixel value of the infrared image; A ninth unit is used to determine a standard deviation of all pixel values ​​in the infrared image, and divide the standard deviation by 4 to obtain a temperature span value; A tenth unit is used for subtracting the temperature span value from the current high value of the temperature to be processed as the lower limit value of the temporary temperature, and taking the current high value of the temperature to be processed as the upper limit value of the temporary temperature; The eleventh unit is used to obtain all pixels in the infrared image whose pixel values ​​are less than or equal to the lower limit value of the temporary storage temperature and greater than or equal to the upper limit value of the temporary storage temperature as the filtered pixels; The twelfth unit is used for, if the number of the screened pixels is greater than the number of pixels of the infrared image divided by 4, taking the result of subtracting the temperature span value divided by 2 from the current high temperature value to be processed as the lower limit value of the temporary temperature; A thirteenth unit, used for storing the temporary storage temperature upper limit value and the temporary storage temperature lower limit value as an element in the temperature level list; A fourteenth unit is used to assign the current high value of the temperature to be processed to the lower limit value of the temporary temperature; The fifteenth unit is used to restart execution from the tenth unit if the current high temperature value to be processed is greater than the minimum infrared pixel value.

8. The image matching device according to claim 5, characterized in that: The first module comprises: A sixteenth unit is used to obtain a maximum infrared pixel value, a minimum infrared pixel value and a standard deviation of all pixel values ​​in the infrared image; A seventeenth unit is used to obtain an infrared pixel value difference by subtracting the minimum infrared pixel value from the maximum infrared pixel value; An eighteenth unit is used for rounding off the result of dividing the infrared pixel value difference by the standard deviation and multiplying by 4 to obtain the number of divisions; The nineteenth unit is used to perform K-means clustering algorithm segmentation on the visible light image based on the number of segmentations to obtain the mean segmented image.

9. An electronic device, characterized in that: include: Memory, for storing computer programs; A processor, configured to implement the image matching method as claimed in any one of claims 1 to 4 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the image matching method according to any one of claims 1 to 4 is implemented.

Images