A crop three-dimensional reconstruction method based on weak light image enhancement

Abstract

The application discloses a kind of weak light image enhancement-based crop three-dimensional reconstruction method, its method includes weak light crop image enhancement and crop three-dimensional reconstruction: the weak light crop image enhancement includes acquisition weak light crop image, weak light crop image enhancement, main function is to enhance the brightness of crop image under the weak light environment collected, to improve image quality;The crop three-dimensional reconstruction includes scale invariant feature transform algorithm (SIFT) feature detection and matching based on, based on motion in structure (SFM) spatial point cloud reconstruction, main function is to obtain key point as feature point and obtain feature point description vector, application Euclidean distance determination obtains matching point pair set, combined with camera internal and external parameter obtains spatial point cloud and camera pose and carries out crop three-dimensional reconstruction;Realize the accurate perception of crop of agricultural machinery equipment in weak light operation environment.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method for three-dimensional reconstruction of crops based on low-light image enhancement. Background Technology

[0002] Real-scene 3D reconstruction, applied to agricultural machinery operations, effectively addresses the challenges of complex production environments and diverse crop characteristics. It not only meets the crucial needs of smart agriculture for precision and efficiency but also provides accurate information for precise crop identification by agricultural machinery. For precise 3D reconstruction in farmland operating environments with less than ideal lighting conditions, the process begins by enhancing crop images captured under low light. Then, feature detection and matching are performed on the enhanced images to extract key points as feature points and obtain feature point descriptors. Finally, combining camera intrinsic and extrinsic parameters to obtain a spatial point cloud and camera pose for 3D crop reconstruction.

[0003] Currently, the SFM method is mainly used for 3D reconstruction of crops. However, it is difficult to complete the reconstruction task under low light conditions. When the input crop image is constrained by illumination, it will produce incorrect spatial point cloud shape estimation, inaccurate camera trajectory, and in some scenarios, there are problems such as failure to detect real features or false features, and inability to match features in other images. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a method for three-dimensional reconstruction of crops based on low-light image enhancement. This method obtains the correct spatial point cloud shape and accurate camera pose, reconstructs a more accurate crop point cloud model, and improves the accuracy of agricultural machinery in perceiving crops under low-light operating conditions.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for 3D reconstruction of crops based on low-light image enhancement is proposed. The method includes low-light crop image enhancement and 3D reconstruction: the low-light crop image enhancement includes acquiring low-light crop images and enhancing the low-light crop image, the main function of which is to enhance the brightness of the acquired crop images in low-light environments to improve image quality; the 3D reconstruction of crops includes feature detection and matching based on the Scale Invariant Feature Transform (SIFT) algorithm and spatial point cloud reconstruction based on Structure in Motion (SFM), the main function of which is to obtain key points as feature points and obtain feature point description vectors, apply Euclidean distance to determine the matching point pair set, and combine camera intrinsic and extrinsic parameters to obtain the spatial point cloud and camera pose for 3D reconstruction of crops.

[0007] Furthermore, the process of acquiring low-light crop images is as follows: During the outdoor low-light image acquisition process, a camera is used to take pictures centered on the crop to be reconstructed, which is conducive to completing the panoramic stitching of the shooting scene; the camera is taken around the scene of the crop to be reconstructed, and an image is acquired every 10 degrees. A large number of low-light crop images containing different height information are collected in an orderly manner around the crop, and more identical objects in the scene are retained in adjacent images, so that the reconstruction details are rich while reducing the time spent on image matching.

[0008] Furthermore, the low-light crop image enhancement method utilizes a low-light crop image enhancement network comprised of three sub-networks: layer decomposition, reflectance recovery, and illumination adjustment. This network enhances low-light crop images by using two crop images under different exposure conditions as input. First, the layer decomposition sub-network decomposes the input low-light crop image into two parts: an illumination component and a reflectance component. The illumination component is responsible for brightness adjustment, while the reflectance component is used to remove degradation, and the two parts share weights. The decomposed illumination component is then passed through an illumination adjustment sub-network composed of multiple convolutional layers to flexibly adjust the light intensity. The decomposed reflectance component is used as input to the reflectance recovery sub-network, which performs denoising through an encoder-decoder network with residual connections. Finally, the reflectance image generated by the reflectance recovery sub-network and the illumination image generated by the illumination adjustment sub-network are combined to form the final low-light crop enhanced image. This effectively removes noise, color distortion, and other degradation problems hidden in dark areas and allows for flexible adjustment of suitable lighting conditions.

[0009] Furthermore, the SIFT feature detection and matching process comprises the following steps:

[0010] (1) By constructing a Gaussian difference pyramid on the input enhanced low-light crop image, the spatial representation of the image at multiple scales is obtained, the scale space is constructed, and candidate points are found. The process is as follows:

[0011] The Gaussian pyramid is composed of multiple image sequences. Each image sequence is formed by convolving a base image I(x,y) with a Gaussian function G(x,y,σ) that has been modified multiple times to change the scale factor σ, resulting in images L(x,y,σ) at different scales. This constructs a multi-scale space. The convolution calculation formula is as follows:

[0012]

[0013] The formula for calculating the Gaussian function containing the scale factor is:

[0014]

[0015] The number of Gaussian pyramid groups O in an image is determined by the row height M and column width N of the image I(x,y), and the calculation formula is:

[0016] O = [log2min(M,N)] - 3

[0017] The number of layers S in each group of the Gaussian pyramid is related to the number of images n from which image features are to be extracted, and the calculation formula is:

[0018] S = n + 3

[0019] The formula for calculating the Gaussian blur coefficient σ(o,r) of the corresponding image is:

[0020]

[0021] o is the group index number of each image sequence in the Gaussian pyramid; r is the index number of the scale image layer in a group of image sequences; σ0 is the initial value of Gaussian blur, which is set to 1.6 by default in the SIFT algorithm. Considering that the camera has actually performed blurring processing on the image with σ = 0.5, the actual initial Gaussian blur coefficient σ0 is:

[0022]

[0023] The Difference of Gaussian (DG) pyramid is a prerequisite for subsequent detection of image extrema. It is obtained by subtracting adjacent layers within each image sequence in the created DG pyramid. The extrema of each layer in the DG pyramid are the feature points to be extracted. The formula for calculating the DG pyramid is:

[0024]

[0025] Where kσ represents the scale factor for different layers than σ;

[0026] (2) In scale space extreme point detection, key points are composed of local extreme points in the Gaussian difference pyramid space. The process is as follows: the initial exploration of key points is completed by comparing the two adjacent layers of images in the same group in the Gaussian difference pyramid. Each intermediate detection point is compared with all 26 of its 26 neighboring points, that is, the 8 neighboring points at the same scale and the 18 points corresponding to the adjacent scales above and below, to ensure that extreme points are detected in both scale space and two-dimensional image space. If the value of the detection point D(x,y,kσ) is the maximum or minimum value among the values ​​of its 26 neighboring points D(x,y,kσ), then the detection point is determined to be a key point of the image at that scale. In order to obtain more accurate results, extreme points are found by curve fitting of the scale space D(x,y,kσ) function to reduce this error. Considering that the edge of the object may change in different images or in the same image, the existing edge effect is deleted to obtain stable key points. At this time, the key points have scaling invariance. The coordinates (x,y) and scale L(x,y) of the point are recorded as the feature information of the point.

[0027] (3) The SIFT feature region is determined by three values ​​representing the location, scale, and orientation information: the center represents the location of the feature point, the radius represents the scale of the key point, and the arrow represents the main direction. The stable orientation of the local structure is obtained using the image gradient method. The formulas for calculating the gradient magnitude m and gradient direction θ are as follows:

[0028]

[0029]

[0030] Histograms are used to count the gradient direction and magnitude of pixels in the neighborhood of keypoints. Eight basic directions spaced 45 degrees apart are selected as the horizontal axis to represent the angle of the gradient direction, and the vertical axis is the sum of the gradient magnitudes corresponding to the gradient directions. The peak value in the histogram is the main direction of the keypoint. If the peak value of a certain keypoint direction is not less than 80% of the set main direction, it is set as the auxiliary direction of the keypoint to increase the matching stability. At this time, the keypoint is determined as a SIFT feature point and has rotation invariance.

[0031] (4) Based on the position, scale, and orientation information of each feature point, a corresponding feature point descriptor is generated to represent the gradient statistics of the Gaussian image in the neighborhood of the feature point. This ensures that the descriptor does not change with various variations and has high uniqueness, thereby improving the probability of correct feature point matching. The descriptor generation process is as follows:

[0032] (4-1) Correcting the principal direction of rotation: To ensure the rotation invariance of the feature vector, the coordinate axes should be rotated by θ in the vicinity of the feature point, centered on the feature point. 主 Angle, which is the rotation of the coordinate axes to the principal direction of the feature point:

[0033] (4-2) Generate a descriptor to obtain a 128-dimensional feature vector. After rotation, the neighborhood pixel region is divided into 4×4 sub-regions centered on the main direction of the feature point. These 16 sub-regions are used to generate 16 seed points describing the feature point. By dividing the pixels around the feature point into blocks, each gradient is decomposed in 8 directions at 45-degree intervals. The gradient magnitude and gradient direction of each pixel in the scale space of the feature point's neighborhood are obtained. Then, a Gaussian window is used to perform weighted operations to calculate the gradient histogram in the 8 directions within the block. The cumulative value of each gradient direction is calculated to generate a seed point. Since each seed point has 8 directional vector information, a 128-dimensional SIFT feature vector can be generated from such a feature point. This joint enhancement of neighborhood directional information enhances the algorithm's noise resistance and provides a more rational fault tolerance for feature matching with localization errors.

[0034] (4-3) Normalize the length of the 128-dimensional feature vector to reduce illumination interference and make the descriptor have illumination-invariant properties. Assume the feature vector is H = (h1, h2, ..., h...). 128 The normalized eigenvectors are L = (l1, l2, ..., l...). 128 The normalized calculation formula is:

[0035]

[0036] After normalization, a threshold of 0.2 needs to be set, and after filtering, the final SIFT feature description vector is obtained.

[0037] (5) Feature point description vector matching is performed. The matching point pair set is obtained by calculating the Euclidean distance to solve the problem of no matching caused by object occlusion and foreground / background blur. After the SIFT feature description vectors of two images are generated, the Euclidean distance of the feature description vectors is used as the similarity metric for feature points in the two images. Take a feature description vector v in one image and find the two feature description vectors v1 and v2 in the other image that are closest in Euclidean distance, and v1 < v2. If the ratio of the closest distance to the second closest distance between these two feature points is less than a set ratio threshold M, the calculation formula is as follows:

[0038]

[0039] Accepting the pair of matching points, the feature description vector v1 is the match of the feature description vector v; reducing the scale threshold will reduce the number of SIFT matching points, but make it more stable; matching a large number of two images with arbitrary scale, rotation and brightness changes shows that the scale threshold is best set between 0.4 and 0.6, and is generally set to 0.5; for matching with high accuracy requirements, the scale threshold is 0.4; when matching requires a large number of matching points, the scale threshold is 0.6.

[0040] Furthermore, the SFM spatial point cloud reconstruction utilizes the matched feature point pairs and camera parameters to solve for the 3D information of the feature points, restoring the matched feature points to 3D space. The point cloud reconstruction process is as follows:

[0041] (1) Extract the EXIF ​​information inside the input low-light crop image. The EXIF ​​information of each image contains the attribute information and shooting data of the image, so as to obtain the camera's focal length and principal point, and calculate the camera intrinsic parameter matrix.

[0042] (2) Using the precise point correspondence between the two crop images obtained in claim 4, the fundamental matrix between the two images is calculated using the matched feature point pairs and the epipolar constraint between the two images, and the essential matrix is ​​calculated using the camera intrinsic parameters obtained in (1); given the source point cloud P and the target point cloud Q, the expression is:

[0043] P = {p1, p2, ..., p} n}, Q={q1,q2,…,q n}

[0044] By finding a spatially optimal transformation matrix consisting of rotation matrix R and translation matrix t that minimizes the distance between point sets P and Q, R and t are solved by minimizing the following equation:

[0045]

[0046] (3) By performing singular value decomposition on the essential matrix, the extrinsic parameter matrix between cameras is obtained. The process is as follows:

[0047] (3-1) Subtract the centroid of each point in point sets P and Q:

[0048] The centroids of the two point sets are as follows:

[0049]

[0050] The coordinates of the two sets of points after removing their centroids are:

[0051]

[0052] (3-2) Find the essential matrix of the two point sets after removing the centroid:

[0053]

[0054] (3-3) Calculate the rotation matrix R and the translation matrix t:

[0055] Singular value decomposition of the covariance matrix W in (3-2) yields a unique solution when W is of full rank:

[0056]

[0057] t = μ q -Rμ p

[0058] The rotation matrix R and translation matrix t obtained by solving are the camera pose. Then, the three-dimensional coordinates M of the crop feature points in the world coordinate system are calculated by triangulation. The positional relationship of the point is determined by the angle between two observations of the same point.

[0059] (4) Perform bundle adjustment optimization on the camera pose and crop point cloud in sequence to reduce the difference between the projection and reprojection of the real 3D space points on the image plane. The crop points observed from each camera pose can be regarded as a bundle. By optimizing the (R,t,M) to minimize the overall bundle cost, the objective function calculation formula for each frame containing K feature points in n frames is as follows:

[0060]

[0061] m ij This represents the coordinates of the feature point corresponding to the i-th 3D point in the j-th image; v ij Indicates whether the i-th 3D point has a projection on the j-th image; if so, v ij =1, otherwise v ij =0; Q is the reprojection function that maps 3D points to the image; d is the Euclidean distance metric function; after bundle adjustment optimization, the accurate camera pose and 3D coordinates of the crop point cloud are obtained, and finally the crop point cloud model is generated.

[0062] The beneficial effects of this invention are reflected in:

[0063] This invention describes a method for 3D reconstruction of crops based on low-light image enhancement. This method improves the accuracy of agricultural machinery's perception of crops in low-light operating environments and reconstructs a more accurate crop point cloud model. Using a camera, images of low-light crops are systematically acquired, centered on the crops. A low-light crop image enhancement network is then used to adjust the lighting conditions of the crop images while removing image noise and color distortion, generating the final enhanced low-light crop image. This enhanced image improves the accuracy of SIFT feature point detection in low-light environments, reducing mismatches and resulting in more accurate feature points and feature point description vectors. Euclidean distance is used to determine the set of matching point pairs. Finally, the spatial point cloud and camera pose are obtained by combining camera intrinsic and extrinsic parameters. This solves the problems of incorrect spatial point cloud shape estimation and inaccurate camera pose caused by lighting constraints. The SFM spatial point cloud reconstruction algorithm is then used for 3D reconstruction of crops, enabling agricultural machinery to accurately perceive crops in low-light operating environments. The crop 3D reconstruction method based on low-light image enhancement effectively improves the accuracy of agricultural machinery in perceiving crops in low-light working environments, providing more accurate environmental information for the later identification of crops. Attached Figure Description

[0064] Figure 1 This is an overall block diagram of the method of the present invention.

[0065] Figure 2 This is a flowchart of the low-light crop image enhancement method of the present invention.

[0066] Figure 3 This is a flowchart of the overall SIFT feature detection process of the present invention.

[0067] Figure 4 This is a schematic diagram of the Gaussian difference pyramid structure constructed by the method of the present invention.

[0068] Figure 5 This is a schematic diagram of the extreme point detection method of the present invention.

[0069] Figure 6 This is a schematic diagram illustrating the generation of feature descriptors in the method of the present invention.

[0070] Figure 7 This is a flowchart of the SFM spatial point cloud reconstruction process of the method of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] like Figure 1 As shown, the overall process of a crop 3D reconstruction method based on low-light image enhancement is as follows:

[0073] The system includes low-light crop image enhancement and 3D crop reconstruction: the low-light crop image enhancement includes acquiring low-light crop images and enhancing them, with the main function being to increase the brightness of the acquired crop images in low-light environments to improve image quality; the feature detection and matching based on the Scale Invariant Feature Transform (SIFT) algorithm and the spatial point cloud reconstruction based on Structure in Motion (SFM) mainly involve finding key points as feature points and obtaining feature point description vectors, applying Euclidean distance to determine matching point pairs, and combining camera intrinsic and extrinsic parameters to obtain the spatial point cloud and camera pose for 3D crop reconstruction.

[0074] The process of acquiring low-light crop images is as follows: During the outdoor low-light image acquisition process, the camera is used to take pictures centered on the crop to be reconstructed, which is conducive to completing the panoramic stitching of the shooting scene; the camera is taken around the scene of the crop to be reconstructed, and an image is acquired every 10 degrees. A large number of low-light crop images containing different height information are collected in an orderly manner around the crop, and more identical objects in the scene are retained in adjacent images, which makes the reconstruction details rich while reducing the time spent on image matching.

[0075] like Figure 2As shown, the low-light crop image enhancement process is as follows: This invention constructs a low-light crop image enhancement network using three sub-networks: layer decomposition, reflectivity recovery, and illumination adjustment, to enhance low-light crop images. Two crop images under different exposure conditions are used as input images to the network. First, the layer decomposition sub-network decomposes the input low-light crop image into two parts: an illumination component and a reflectivity component. The illumination component is responsible for brightness adjustment, and the reflectivity component is used to remove degradation, with the two parts sharing weights. The decomposed illumination component is then passed through an illumination adjustment sub-network composed of multiple convolutional layers to flexibly adjust the light intensity. The decomposed reflectivity component is used as input to the reflectivity recovery sub-network, and after passing through an encoder-decoder network with residual connections, a denoising operation is performed. Finally, the reflectivity image generated by the reflectivity recovery sub-network and the illumination image generated by the illumination adjustment sub-network are combined to form the final low-light crop enhanced image, effectively removing noise, color distortion, and other degradation problems hidden in dark areas, and freely adjusting suitable lighting conditions.

[0076] like Figure 3 As shown, the SIFT feature detection and matching process is as follows:

[0077] (1) By constructing a Gaussian difference pyramid on the input enhanced low-light crop image, the spatial representation of the image at multiple scales is obtained, the scale space is constructed, and candidate points are found.

[0078] like Figure 4 As shown, the Gaussian difference pyramid construction process is further as follows:

[0079] The Gaussian pyramid is composed of multiple image sequences. Each image sequence is formed by convolving a base image I(x,y) with a Gaussian function G(x,y,σ) that has been modified multiple times to change the scale factor σ, resulting in images L(x,y,σ) at different scales. This constructs a multi-scale space. The convolution calculation formula is as follows:

[0080]

[0081] The formula for calculating the Gaussian function containing the scale factor is:

[0082]

[0083] The number of Gaussian pyramid groups O in an image is determined by the row height M and column width N of the image I(x,y), and the calculation formula is:

[0084] O = [log2min(M,N)] - 3

[0085] The number of layers S in each group of the Gaussian pyramid is related to the number of images n from which image features are to be extracted, and the calculation formula is:

[0086] S = n + 3

[0087] The formula for calculating the Gaussian blur coefficient σ(o,r) of the corresponding image is:

[0088]

[0089] o is the group index number of each image sequence in the Gaussian pyramid; r is the index number of the scale image layer in a group of image sequences; σ0 is the initial value of Gaussian blur, which is set to 1.6 by default in the SIFT algorithm. Considering that the camera has actually performed blurring processing on the image with σ = 0.5, the actual initial Gaussian blur coefficient σ0 is:

[0090]

[0091] The Difference of Gaussian (DG) pyramid is a prerequisite for subsequent detection of image extrema. It is obtained by subtracting adjacent layers within each image sequence in the created DG pyramid. The extrema of each layer in the DG pyramid are the feature points to be extracted. The formula for calculating the DG pyramid is:

[0092]

[0093] Where kσ represents the scale factor for different layers than σ;

[0094] (2) In the detection of extreme points in scale space, key points are composed of local extreme points in the Gaussian difference pyramid space.

[0095] like Figure 5 As shown, further, the Gaussian difference pyramid extreme point detection is as follows: the initial exploration of key points is completed by comparing each adjacent pair of images within the same group of the Gaussian difference pyramid. Each intermediate detection point is compared with all 26 of its 26 neighboring points, that is, the 8 neighboring points at the same scale and the 18 points corresponding to the adjacent scales above and below, to ensure that extreme points are detected in both the scale space and the two-dimensional image space. If the detection point D(x,y,kσ) value is the maximum or minimum value among its 26 neighboring points D(x,y,kσ), then the detection point is determined to be a key point of the image at that scale. To obtain more accurate results, extreme points are found by curve fitting of the scale space D(x,y,kσ) function to reduce this error. Considering that the edges of objects may change in different images or in the same image, the existing edge effects are deleted to obtain stable key points. At this time, the key points have scaling invariance, and the coordinates (x,y) and scale L(x,y) of the point are recorded as the feature information of the point.

[0096] (3) The SIFT feature region is determined by three values ​​representing the location, scale, and orientation information: the center represents the location of the feature point, the radius represents the scale of the key point, and the arrow represents the main direction. The stable orientation of the local structure is obtained using the image gradient method. The formulas for calculating the gradient magnitude m and gradient direction θ are as follows:

[0097]

[0098]

[0099] Histograms are used to count the gradient direction and magnitude of pixels in the neighborhood of keypoints. Eight basic directions spaced 45 degrees apart are selected as the horizontal axis to represent the angle of the gradient direction, and the vertical axis is the sum of the gradient magnitudes corresponding to the gradient directions. The peak value in the histogram is the main direction of the keypoint. If the peak value of a certain keypoint direction is not less than 80% of the set main direction, it is set as the auxiliary direction of the keypoint to increase the matching stability. At this time, the keypoint is determined as a SIFT feature point and has rotation invariance.

[0100] (4) Generate corresponding feature point descriptors by using the position, scale and orientation information of each feature point to represent the gradient statistics of the Gaussian image in the neighborhood of the feature point, so that it does not change with various changes, and the descriptors have high uniqueness, so as to improve the probability of correct matching of feature points.

[0101] like Figure 6 As shown, the feature descriptor generation process is further as follows:

[0102] (4-1) Correcting the principal direction of rotation: To ensure the rotation invariance of the feature vector, the coordinate axes should be rotated by θ in the vicinity of the feature point, centered on the feature point. 主 Angle, which is the rotation of the coordinate axes to the principal direction of the feature point:

[0103] (4-2) Generate a descriptor to obtain a 128-dimensional feature vector. After rotation, the neighborhood pixel region is divided into 4×4 sub-regions centered on the main direction of the feature point. These 16 sub-regions are used to generate 16 seed points describing the feature point. By dividing the pixels around the feature point into blocks, each gradient is decomposed in 8 directions at 45-degree intervals. The gradient magnitude and gradient direction of each pixel in the scale space of the feature point's neighborhood are obtained. Then, a Gaussian window is used to perform weighted operations to calculate the gradient histogram in the 8 directions within the block. The cumulative value of each gradient direction is calculated to generate a seed point. Since each seed point has 8 directional vector information, a 128-dimensional SIFT feature vector can be generated from such a feature point. This joint enhancement of neighborhood directional information enhances the algorithm's noise resistance and provides a more rational fault tolerance for feature matching with localization errors.

[0104] (4-3) Normalize the length of the 128-dimensional feature vector to reduce illumination interference and make the descriptor have illumination-invariant properties. Assume the feature vector is H = (h1, h2, ..., h...). 128 The normalized eigenvectors are L = (l1, l2, ..., l...). 128 The normalized calculation formula is:

[0105]

[0106] After normalization, a threshold of 0.2 needs to be set, and after filtering, the final SIFT feature description vector is obtained.

[0107] (5) Feature point description vector matching is performed. The matching point pair set is obtained by calculating the Euclidean distance to solve the problem of no matching caused by object occlusion and foreground / background blur. After the SIFT feature description vectors of two images are generated, the Euclidean distance of the feature description vectors is used as the similarity metric for feature points in the two images. Take a feature description vector v in one image and find the two feature description vectors v1 and v2 in the other image that are closest in Euclidean distance, and v1 < v2. If the ratio of the closest distance to the second closest distance between these two feature points is less than a set ratio threshold M, the calculation formula is as follows:

[0108]

[0109] Accepting the pair of matching points, feature description vector v1 is the match of feature description vector v; reducing the scale threshold will reduce the number of SIFT matching points, but make it more stable; matching a large number of two images with arbitrary scale, rotation and brightness changes shows that the scale threshold is best set between 0.4 and 0.6, and is set to 0.5; for matching with high accuracy requirements, the scale threshold is 0.4; when matching requires a large number of matching points, the scale threshold is 0.6.

[0110] like Figure 7 As shown, SFM spatial point cloud reconstruction utilizes the matched feature point pairs and camera parameters to solve for the 3D information of the feature points, restoring the matched feature points to 3D space. The SFM spatial point cloud reconstruction process is as follows:

[0111] (1) Extract the EXIF ​​information inside the input low-light crop image. The EXIF ​​information of each image contains the attribute information and shooting data of the image, so as to obtain the camera's focal length and principal point, and calculate the camera intrinsic parameter matrix.

[0112] (2) The precise point correspondence between two crop images is obtained through SIFT feature detection and matching. The basic matrix between the two images is calculated using the matched feature point pairs and the epipolar constraint between the two images, and the essential matrix is ​​calculated with the camera intrinsic parameters obtained in (1). Given the source point cloud P and the target point cloud Q, the expression is:

[0113] P = {p1, p2, ..., p} n}, Q={q1,q2,…,q n}

[0114] By finding a spatially optimal transformation matrix consisting of rotation matrix R and translation matrix t that minimizes the distance between point sets P and Q, R and t are solved by minimizing the following equation:

[0115]

[0116] (3) By performing singular value decomposition on the essential matrix, the extrinsic parameter matrix between cameras is obtained. The process is as follows:

[0117] (3-1) Subtract the centroid of each point in point sets P and Q:

[0118] The centroids of the two point sets are as follows:

[0119]

[0120] The coordinates of the two sets of points after removing their centroids are:

[0121]

[0122] (3-2) Find the essential matrix of the two point sets after removing the centroid:

[0123]

[0124] (3-3) Calculate the rotation matrix R and the translation matrix t:

[0125] Singular value decomposition of the covariance matrix W in (3-2) yields a unique solution when W is of full rank:

[0126]

[0127] t = μ q -Rμ p

[0128] The rotation matrix R and translation matrix t obtained by solving are the camera pose. Then, the three-dimensional coordinates M of the crop feature points in the world coordinate system are calculated by triangulation. The positional relationship of the point is determined by the angle between two observations of the same point.

[0129] (4) Perform bundle adjustment optimization on the camera pose and crop point cloud in sequence to reduce the difference between the projection and reprojection of the real 3D space points on the image plane. The crop points observed from each camera pose can be regarded as a bundle. By optimizing the (R,t,M) to minimize the overall bundle cost, the objective function calculation formula for each frame containing K feature points in n frames is as follows:

[0130]

[0131] m ij This represents the coordinates of the feature point corresponding to the i-th 3D point in the j-th image; v ij Indicates whether the i-th 3D point has a projection on the j-th image; if so, v ij =1, otherwise v ij =0; Q is the reprojection function that maps 3D points to the image; d is the Euclidean distance metric function; after BA optimization, the accurate camera pose and 3D coordinates of the crop point cloud are obtained, and finally the crop point cloud model is generated.

[0132] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0133] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for three-dimensional reconstruction of crops based on low-light image enhancement, characterized in that, The system includes low-light crop image enhancement and crop 3D reconstruction: the low-light crop image enhancement includes acquiring low-light crop images and enhancing these images, primarily to increase the brightness of the acquired crop images in low-light environments to improve image quality; the crop 3D reconstruction includes feature detection and matching based on scale-invariant feature transform algorithm and spatial point cloud reconstruction based on in-motion structure recovery, primarily to obtain key points as feature points and obtain feature point description vectors, apply Euclidean distance to determine matching point pairs, and combine camera intrinsic and extrinsic parameters to obtain spatial point cloud and camera pose for crop 3D reconstruction. The low-light crop image enhancement includes the following steps: A low-light crop image enhancement network is constructed by using three subnetworks: layer decomposition, reflectivity restoration, and illumination adjustment, to enhance low-light crop images. Using two crop images under different exposure conditions as input images for the network, the layer decomposition sub-network is first used to decompose the input low-light crop image into two parts: illumination component and reflectance component. The illumination component is responsible for brightness adjustment, and the reflectance component is used to remove degradation. The two parts share weights. The decomposed illumination components are used to flexibly adjust the illumination intensity through an illumination adjustment sub-network consisting of multiple convolutional layers. The decomposed reflectance components are used as input to the reflectance recovery sub-network, and then pass through an encoder-decoder network with residual connections to perform denoising operations. The final low-light crop enhancement image is formed by combining the reflectance image generated by the reflectance restoration subnetwork and the illumination image generated by the illumination adjustment subnetwork, and the appropriate illumination conditions can be freely adjusted. The motion-based reconstruction of the structural point cloud utilizes matched feature point pairs and camera parameters to solve for the 3D information of the feature points, restoring the matched feature points to 3D space. The motion-based reconstruction of the structural point cloud includes the following steps: Extract exchangeable image file information from the input low-light crop image; The precise point correspondence between two crop images is obtained through feature detection and matching using the scale-invariant feature transform algorithm. By performing singular value decomposition on the essential matrix; Bundle adjustment optimization was performed sequentially on the camera pose and the crop point cloud.

2. The method for three-dimensional reconstruction of crops based on low-light image enhancement as described in claim 1, characterized in that, The process of acquiring low-light crop images is as follows: During the outdoor low-light image acquisition process, a camera is used to take pictures centered on the crop to be reconstructed; pictures are taken around the scene of the crop to be reconstructed, and an image is acquired every 10 degrees. A large number of low-light crop images containing different height information are systematically acquired around the crop, and more of the same scene objects are retained in adjacent images.

3. The method for three-dimensional reconstruction of crops based on low-light image enhancement as described in claim 1, characterized in that, The steps of feature detection and matching in the scale-invariant feature transformation algorithm are as follows: By constructing a Gaussian difference pyramid on the input enhanced low-light crop image, a multi-scale spatial representation of the image is obtained, realizing the construction of scale space and finding candidate points. The process is as follows: The Gaussian pyramid consists of multiple image sequences, each sequence being derived from a base image within that group. and multiple changes in scale factor Gaussian function Images of different scales obtained by convolution calculation Composed of these elements, a multi-scale space is constructed. The convolution calculation formula is as follows: The formula for calculating the Gaussian function containing the scale factor is: The number of Gaussian pyramid groups O in an image is determined by the image line height With column width The decision is made using the following formula: The number of layers S in each group of the Gaussian pyramid is related to the number of images n from which image features are to be extracted, and the calculation formula is: Gaussian blur coefficient of the corresponding image The calculation formula is: The group index number for each image sequence in the Gaussian pyramid; The image layer index number in a sequence of images at different scales; The initial value for Gaussian blur is 1.6, which is set to 1.6 by default in the SIFT algorithm, taking into account that the camera has already applied the image blurring. The fuzzy processing is performed, therefore the actual initial Gaussian fuzzy coefficients are... for: The Difference of Gaussian (DG) pyramid is a prerequisite for subsequent detection of image extrema. It is obtained by subtracting adjacent layers within each image sequence in the created DG pyramid. The extrema of each layer in the DG pyramid are the feature points to be extracted. The formula for calculating the DG pyramid is: in Indicates and Scale factors for different layers; In scale-space extremum detection, keypoints include local extrema in the Gaussian difference pyramid space. The detection process is as follows: Initial keypoint detection is achieved by comparing adjacent layers within the same group of images in the Gaussian difference pyramid. Each intermediate detection point is compared with all 26 of its 26 neighboring points (8 at the same scale and 18 at the adjacent scales above and below), ensuring extrema are detected in both scale space and 2D image space. When the D(x,y,kσ) value of the detection point is the maximum or minimum among its 26 neighboring points, it is determined to be a keypoint at that scale. To obtain more accurate results, curve fitting of the scale-space D(x,y,kσ) function is performed to find extrema and reduce this error. Considering that object edges may change in different images or even within the same image, edge effects are removed to obtain stable keypoints. At this point, the keypoints exhibit scaling invariance. and scale This information will be recorded as a feature of that point. The scale-invariant feature transform algorithm determines the feature region using three values ​​representing position, scale, and orientation information: the center represents the feature point's location, the radius represents the keypoint's scale, and the arrow indicates the main direction. The algorithm uses image gradients to determine the stable orientation of the local structure and calculates the gradient magnitude. and gradient direction The calculation formula is: Histograms are used to count the gradient direction and magnitude of pixels in the neighborhood of keypoints. Eight basic directions spaced 45 degrees apart are selected as the horizontal axis to represent the angle of the gradient direction, and the vertical axis is the sum of the gradient magnitudes corresponding to the gradient direction. The peak value in the histogram is the main direction of the keypoint. If the peak value of a certain keypoint direction is not less than 80% of the set main direction, it is set as the auxiliary direction of the keypoint to increase the matching stability. At this time, the keypoint is determined as the feature point of the scale-invariant feature transformation algorithm and has rotation invariance. By using the position, scale, and orientation information of each feature point, a corresponding feature point descriptor is generated, representing the statistical results of the Gaussian image gradient in the neighborhood of the feature point, so that it does not change with various variations.

4. The method for three-dimensional reconstruction of crops based on low-light image enhancement according to claim 3, characterized in that, The feature point descriptor generation process includes the following steps: To correct the principal direction of rotation and ensure the rotation invariance of the feature vector, the coordinate axes must be rotated by a principal angle θ around the feature point in the vicinity. In other words, the coordinate axes are rotated to align with the principal direction of the feature point. A descriptor is generated to obtain a 128-dimensional feature vector. After rotation, the neighborhood pixel region is divided into 4×4 sub-regions centered on the main direction of the feature point. Sixteen sub-regions are used to generate 16 seed points to describe the feature point. By dividing the pixels around the feature point into blocks, each gradient is decomposed in eight directions at 45-degree intervals. The gradient magnitude and gradient direction of each pixel in the scale space of the feature point's neighborhood are obtained. Then, a Gaussian window is used to perform weighted operations to calculate the gradient histogram in the eight directions within the block. The cumulative value of each gradient direction is calculated to generate a seed point. Since each seed point has eight direction vector information, a 128-dimensional SIFT feature vector can be generated from such a feature point. Normalizing the length of the 128-dimensional feature vector reduces illumination interference, making the descriptor light-invariant. Assume the feature vector is... The normalized feature vector is The normalization calculation formula is: After normalization, a threshold of 0.2 needs to be set. After filtering, the final SIFT feature description vector is obtained. Feature point description vectors are matched, and the matching point pair set is obtained by calculating the Euclidean distance to solve the problem of no matching caused by object occlusion and foreground / background blur. After the scale-invariant feature transform algorithm generates feature description vectors for two images, the Euclidean distance between the feature description vectors is used as the similarity metric for feature points in the two images. A feature description vector v in one image is taken, and the two feature description vectors with the closest Euclidean distance in the other image are found. and And in terms of distance If the ratio of the distance between the nearest and the second nearest of these two feature points is less than a set threshold... The calculation formula is: Accept the pair of matching points, feature description vector Feature description vector Matching; lowering the scale-invariant feature transform algorithm will reduce the number of matching points, but make it more stable; matching a large number of two images with arbitrary scale, rotation and brightness changes shows that the optimal scale threshold is set between 0.4 and 0.6, and is generally set to 0.5; for matching with high accuracy requirements, the scale threshold is 0.4; when matching with a large number of matching points is required, the scale threshold is 0.

6.

5. The method for three-dimensional reconstruction of crops based on low-light image enhancement as described in claim 1, characterized in that, Extract the exchangeable image file information from the input low-light crop images. The exchangeable image file information for each image contains the attribute information and shooting data of the image, thereby obtaining the camera's focal length and principal point, and calculating the camera intrinsic parameter matrix.

6. The method for three-dimensional reconstruction of crops based on low-light image enhancement according to claim 1, characterized in that, The precise point correspondence between two crop images is obtained by feature detection and matching using the scale-invariant feature transform algorithm. The basic matrix between the two images is calculated using the matched feature point pairs and the epipolar constraint between the two images. The essential matrix is ​​calculated using the obtained camera intrinsic parameters. Given a source point cloud P and a target point cloud Q, the expression is: The source point cloud P and the target point cloud Q are expressed as: By finding a spatially optimal transformation matrix consisting of rotation matrix R and translation matrix t that minimizes the distance between point sets P and Q, R and t are solved by minimizing the following equation: 。 7. The method for three-dimensional reconstruction of crops based on low-light image enhancement as described in claim 1, characterized in that, By performing singular value decomposition on the essential matrix, the extrinsic parameter matrix between cameras is obtained. The process is as follows: Subtract the centroid of each point in point sets P and Q: The centroids of the two point sets are as follows: The coordinates of the two sets of points after removing their centroids are: Find the essential matrix of the two point sets after removing the centroid: Calculate the rotation matrix R and the translation matrix t: Singular value decomposition of the covariance matrix W yields a unique solution when W is of full rank: The rotation matrix R and translation matrix t obtained by solving are the camera pose. Then, the three-dimensional coordinates M of the crop feature points in the world coordinate system are calculated by triangulation. The positional relationship of the point is determined by the angle between two observations of the same point.

8. The method for three-dimensional reconstruction of crops based on low-light image enhancement as described in claim 1, characterized in that, Bundle adjustment optimization is performed sequentially on the camera pose and crop point cloud to reduce the difference between the projection and reprojection of real 3D points on the image plane. Crop points observed from each camera pose can be considered as a bundle. By optimizing (R,t,M) to minimize the overall bundle cost, the objective function for each of the n frames containing K feature points is calculated as follows: This represents the feature point coordinates corresponding to the i-th 3D point in the j-th image; Indicate whether the i-th 3D point has a projection on the j-th image, if so. ,otherwise Q is the reprojection function that maps 3D points onto the image; d is the Euclidean distance metric function; after bundle adjustment optimization, the accurate camera pose and 3D coordinates of the crop point cloud are obtained, and finally the crop point cloud model is generated.

Images