Blueberry tree disease and pest detection method based on machine vision

Abstract

This invention relates to the field of agricultural intelligent detection technology, specifically a machine vision-based method for detecting diseases and pests in blueberry trees. The invention acquires ultraviolet grayscale and visible light color images of the same blueberry target, extracts local grayscale extreme points of the bloom layer in the ultraviolet image as anchor nodes, and constructs a cluster manifold topology describing the spatial distribution of the fruit. Using this topology as a deformation control framework, pixel-level non-rigid registration of the visible light image is performed using a thin-plate spline interpolation function. Subsequently, a weighted difference operation is performed on the registered image and the ultraviolet image to generate a spectral residual saliency map that suppresses the background of healthy bloom and highlights abnormal areas. Disease and pest targets are then extracted through threshold segmentation. This invention effectively solves the cross-modal registration problem of dense small fruit clusters under dynamic wind conditions, accurately removes bloom interference using spectral physical properties, and significantly improves detection robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural intelligent detection technology, in particular to a blueberry tree disease and pest detection method based on machine vision. BACKGROUND

[0002] In the process of blueberry cultivation and management, the prevention and control of diseases and pests is a crucial link. Existing agricultural technology introduces machine vision and image processing technology. For example, some methods propose a prevention and control method by collecting images of blueberry branches, fruits and the like, and double comparison with historical pest image data, trying to determine the disease and pest through image matching. With the development of deep learning, more advanced solutions are proposed, which use multispectral cameras to collect multi-dimensional information of fruit trees, and extract features such as color and shape of disease spots through convolutional neural networks, and conduct multi-modal fusion detection combined with environmental data.

[0003] However, the above methods still have universal defects that are difficult to overcome when directly applied to this specific crop of blueberries: first, blueberry fruits have significant biological characteristics, and their surfaces are naturally covered with a layer of white waxy fruit powder. In conventional visible light image processing, this layer of healthy white fruit powder is easily confused with the white mycelium of early white powdery mildew or gray mold, or the white marks left by the oviposition of the fruit weevil. Although existing methods introduce multispectral imaging, they mostly focus on detecting leaf thermal radiation or surface fluorescence using spectral differences, lacking targeted algorithms specifically for separating the visual features of the fruit powder layer and the disease layer. Under complex lighting conditions, the high light reflection of the fruit powder is often misjudged as noise and removed, or the fruit powder is misjudged as a disease spot due to physical shedding, resulting in distorted feature extraction. Second, blueberry fruits are densely clustered, small and attached to tender branches, and are easily affected by wind and other factors in the natural environment, resulting in high-frequency micro-movement. Existing multi-modal data fusion methods usually assume that the object being collected is relatively static or rigid, and use global image alignment or simple feature stitching. However, in outdoor dynamic scenes, small vibrations can cause pixel-level displacement (i.e., spectral mismatch) between visible light images and infrared / ultraviolet images on dense small targets. For a single blueberry with a diameter of only a few millimeters, this edge drift can incorrectly match the infrared features of healthy fruits to adjacent diseased fruits or leaves, making the fused comprehensive feature vector unreliable.

[0004] In summary, how to overcome the cross-modal registration error caused by the wind swinging of dense small fruit clusters in a dynamic and unstructured field environment, while effectively excluding the visual interference of the original fruit powder layer, so as to realize accurate feature extraction of blueberry diseases and pests is a technical problem to be solved in the field.

[0005] Therefore, a blueberry tree disease and pest detection method based on machine vision is proposed. SUMMARY

[0006] The application aims to provide a blueberry tree disease and pest detection method based on machine vision. By acquiring ultraviolet grayscale and visible color images of the same blueberry target, local grayscale extreme value points of the fruit powder layer in the ultraviolet image are extracted as anchor nodes, and a cluster system manifold topology network describing the spatial distribution of the fruit is constructed. The topology network is used as a deformation control skeleton, and a thin plate spline interpolation function is used for pixel-level non-rigid registration of the visible light image. Then, weighted difference operation is performed on the registered image and the ultraviolet image to generate a spectral residual saliency map that suppresses the healthy fruit powder background and highlights the abnormal area, and the disease and pest target is extracted by threshold segmentation. The application effectively solves the cross-modal registration problem of dense small fruit clusters in a dynamic wind blowing environment, accurately removes the fruit powder interference by using spectral physical characteristics, and significantly improves the detection robustness.

[0007] To achieve the above object, the application provides the following technical scheme:

[0008] The blueberry tree disease and pest detection method based on machine vision comprises:

[0009] Acquire ultraviolet grayscale images and visible color images synchronously collected for the same blueberry target area;

[0010] Perform multi-scale spot detection filtering processing on the ultraviolet grayscale image, and extract pixel coordinates of local grayscale minimum value points of the blueberry fruit powder layer as ultraviolet extreme value anchor nodes ； Based on the spatial adjacency relationship of the ultraviolet extreme value anchor node set, a cluster system manifold topology network describing the spatial distribution structure of the fruit in the image is constructed by using the Delaunay triangulation algorithm;

[0011] The cluster system manifold topology network is used as a deformation control skeleton, local displacement vectors of the visible color image relative to the ultraviolet grayscale image at each node are calculated, and a thin plate spline interpolation function is used for pixel-level geometric correction of the visible color image to generate a registered image spatially aligned with the ultraviolet grayscale image;

[0012] Perform weighted difference operation on the luminance component of the registered image and the grayscale component of the ultraviolet grayscale image to generate a spectral residual saliency map that suppresses the background and highlights the abnormal highlight area; the spectral residual saliency map is subjected to threshold segmentation processing to extract a binary mask and determine the abnormal target area in the image.

[0013] Preferably, a co-optical axis imaging optical path is constructed, and a light splitting prism is used to spatially decompose an incident light field into a first optical path and a second optical path, which are projected to a first image sensor and a second image sensor, respectively; a hardware trigger signal is used to synchronize the exposure start time and the integration time of the two sensors, and an original image pair with a fixed homographic mapping relationship is output; a distortion correction preprocessing is performed on the obtained image pair, radial distortion and tangential distortion are eliminated by using a pre-calibrated camera intrinsic parameter matrix, and multi-channel image data without geometric distortion is output.

[0014] Preferably, the step of performing multi-scale spot detection filtering processing on the ultraviolet gray-scale image and extracting local gray-scale minimum value points comprises: performing multi-scale morphological operation on the ultraviolet gray-scale image, suppressing salt and pepper noise in the image by using an isotropic structure element, and generating a preprocessed image with a smooth background; calculating a gray-scale gradient field of the preprocessed image, and locating a spot-shaped structure in the image by using a determinant response value of a Hessian matrix; searching for a gray-scale minimum value pixel in the spot-shaped structure region, and checking whether the gray-scale gradient direction in the pixel neighborhood is isotropically convergent or not; if the convergence condition is met and the gray-scale value is lower than a preset adaptive threshold, the pixel is marked as an effective ultraviolet extreme anchor node.

[0015] Preferably, the step of constructing a cluster system manifold topological network comprises: after an initial triangular mesh is generated by using a Delaunay triangulation algorithm, performing an edge filtering operation based on Euclidean distance, traversing all edges in the mesh, cutting off long edge connections with a length exceeding a preset neighborhood threshold, and eliminating false associations of non-connected regions in the image; calculating the connectivity of the remaining nodes, and removing isolated nodes and associated edges with a connectivity lower than a preset value; and finally, the generated topological network is defined as a vertex set and an edge set describing local deformation relationships of the image, which is used to constrain subsequent image space transformation.

[0016] Preferably, the calculation of the local displacement vector of the visible color image relative to the ultraviolet gray image at each node specifically comprises: projecting the ultraviolet extreme value anchor node into the coordinate system of the visible light band color image to form an initial search center; setting a search window with a preset radius with each initial search center as the center; performing edge detection and Hough circle transformation on the visible light image within the search window to extract the geometric center coordinates of the fruit outline, which are defined as visible light candidate matching points; if detection fails, an eccentricity compensation based on the light source direction vector is performed using the local gray maximum value point as an alternative matching point; a weighted objective function is constructed including a position matching cost term and a topological rigidity cost term; the position matching cost term calculates the Euclidean distance between the transformed coordinates of the current node and the coordinates of the visible light candidate matching points; the topological rigidity cost term is calculated based on the node connection relationship defined by the cluster system manifold topological net, and the specific calculation method is as follows: for each node in the net, the statistical difference degree between the displacement vector of the node and the displacement vector set of all directly connected neighbor nodes of the node in the cluster system manifold topological net is calculated; by minimizing the objective function, the optimal displacement vector that minimizes the sum of the position error and the deformation cost is solved.

[0017] Preferably, in the process of pixel-level geometric correction using thin-plate spline interpolation function, a smoothing regularization constraint for the transformation field is introduced: when solving the weighted coefficient matrix of the thin-plate spline function, a regularization term based on bending energy is added to the objective function, and the bending energy is defined as the square integral of the second-order partial derivative of the transformation function; the rigidity degree of the transformation field is controlled by adjusting the regularization parameter, and when the displacement vector of the local control point causes the curvature of the grid deformation to exceed the preset threshold, the regularization parameter is increased to suppress the deformation component whose curvature exceeds the preset threshold, thereby generating a continuous mapping field and preventing pixel folding and / or tearing in the local area of the registered image.

[0018] Preferably, the specific steps of performing weighted difference operation to generate a spectral residual saliency map are as follows: converting the visible light color image to a luminance channel and performing gray scale inversion transformation on the ultraviolet gray image; based on the gray scale distribution of the luminance channel, performing histogram specification processing on the inverted ultraviolet image to make the pixel gray scale values of the two channels in the same dynamic range; calculating the algebraic difference value of the corresponding pixels of the two channels, and at the same time, calculating the gradient amplitude of each pixel point of the luminance channel, and normalizing the gradient amplitude to a preset numerical interval; the weight coefficient is calculated by subtracting the normalized gradient amplitude from the preset unit constant; the algebraic difference value and the mapped weight coefficient are multiplied to output the spectral residual saliency map.

[0019] Preferably, the specific steps for threshold segmentation of the spectral residual saliency map include: calculating the gray-level histogram of the spectral residual saliency map; calculating a first threshold and a second threshold based on the statistical properties of the histogram, wherein the first threshold is greater than the second threshold; extracting all pixels with gray values ​​greater than the first threshold to form a high-confidence marker map; extracting all pixels with gray values ​​greater than the second threshold to form a generalized candidate mask map; using the high-confidence marker map as the marker image for morphological geodesic dilatation operation, and using the generalized candidate mask map as the mask image for morphological geodesic dilatation operation; performing iterative geodesic dilatation operation to divide the connected components in the marker image into... Spatial propagation is performed within the constraints of the mask image until the connected components no longer change, and the resulting image is defined as the initial segmentation mask. An Euler-based hole-filling operation is performed on the initial segmentation mask, flipping background pixels completely surrounded by foreground pixels in a four-connected manner to foreground pixels, generating a solid binary mask. The compactness factor and mean edge gradient of each connected component in the binary mask are calculated. Connected components with a compactness factor less than the preset shape factor threshold are removed using a preset shape factor threshold, and connected components with a mean edge gradient greater than the preset gradient threshold are removed using a preset gradient threshold. The remaining connected components are identified as abnormal target regions in the image.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] 1. This invention effectively solves the technical challenge of distinguishing between the bloom layer on the surface of blueberries and specific fungal diseases under visible light. By introducing ultraviolet (UV) grayscale images, and utilizing the differences in spectral response between healthy bloom layers and diseased areas in the UV band (e.g., bloom layers exhibit low reflectivity or strong scattering leading to a dark field at a specific 365nm wavelength, while specific fungal diseases exhibit high reflectivity or fluorescence), a spectral residual saliency map based on weighted difference operations is constructed. This method transforms biological characteristics into image grayscale logic, automatically filtering out visually white healthy bloom background interference during the image fusion stage. It can accurately highlight abnormal diseased areas lacking UV absorption characteristics without relying on high-performance deep learning semantic segmentation networks, significantly reducing false positive rates caused by bloom reflection, accumulation, or shedding.

[0022] 2. This invention achieves pixel-level precise registration of cross-modal images of dense clusters of small blueberry fruits under dynamic natural environments. Addressing the problem that blueberry bunches are susceptible to non-rigid deformation due to wind exposure outdoors, leading to misalignment of images across different channels, this invention innovatively extracts stable gray-level extrema points from ultraviolet images as anchor nodes and constructs a cluster manifold topology network describing the spatial distribution of the fruits. Using this topology network as the deformation control framework, a thin-plate spline interpolation algorithm is employed to perform nonlinear geometric correction on the visible light images. This method overcomes the limitations of traditional global rigid registration algorithms, adaptively compensating for local non-uniform displacements caused by branch swaying, fruit compression, or changes in viewing angle, ensuring spatial consistency of multispectral data on small targets, and significantly improving adaptability in complex field environments.

[0023] 3. This invention significantly improves the segmentation integrity and noise resistance of small lesions under complex lighting conditions. By employing a cascaded segmentation strategy combining dual-threshold hysteresis judgment and morphological geodesic dilatation, this invention establishes a target extraction mechanism based on connectivity reconstruction. High-confidence thresholds are used to lock the core lesions, and then low thresholds combined with spatial topological constraints are used to recall weak edge regions. This "anchoring-growth" strategy effectively solves the problem of single-threshold segmentation easily breaking or losing details when dealing with uneven lighting. Simultaneously, combined with geometric artifact filtering based on compactness and gradient, discrete high-light noise or branch reflection interference can be effectively shielded, ensuring the detection rate of early small lesions and the robustness of the detection results. Attached Figure Description

[0024] Figure 1 A flowchart of a machine vision-based blueberry tree pest and disease detection method provided in an embodiment of the present invention;

[0025] Figure 2 A non-rigid registration logic diagram based on clustered manifold topology network provided in this embodiment of the invention;

[0026] Figure 3 The principle of generating spectral residual saliency maps and signal flow graphs provided in the embodiments of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see Figures 1 to 3 This invention provides a machine vision-based method for detecting diseases and pests in blueberry trees, the technical solution of which is as follows:

[0029] Machine vision-based methods for detecting blueberry tree diseases and pests include:

[0030] Acquire ultraviolet grayscale images and visible light color images simultaneously collected for the same blueberry target area;

[0031] The ultraviolet grayscale image is subjected to multi-scale speckle detection filtering to extract the pixel coordinates of local grayscale minima in the blueberry bloom layer as ultraviolet extreme value anchoring nodes. ； Based on the spatial adjacency relationship of the ultraviolet extreme value anchor node set, the Delaunay triangulation algorithm is used to construct a clustered manifold topology network describing the spatial distribution structure of fruits in the image;

[0032] Using the cluster-system manifold topology network as the deformation control skeleton, the local displacement vector of the visible light color image relative to the ultraviolet grayscale image at each node is calculated, and the visible light color image is geometrically corrected at the pixel level using the thin plate spline interpolation function to generate a registered image spatially aligned with the ultraviolet grayscale image.

[0033] A weighted difference operation is performed on the luminance component of the registered image and the grayscale component of the ultraviolet grayscale image to generate a spectral residual saliency map that suppresses the background and highlights abnormally bright areas; the spectral residual saliency map is then subjected to threshold segmentation to extract a binarized mask and determine the abnormal target areas in the image.

[0034] Example 1:

[0035] This embodiment is applied to an intelligent inspection scenario in outdoor blueberry plantations, where mobile robots or drones equipped with detection devices operate continuously during the blueberry ripening period. At this time, the blueberries grow in dense clusters, covered with a thick layer of white bloom, and are subject to high-frequency, irregular micro-movements within the field of view due to natural wind. Traditional RGB vision solutions are prone to misidentifying normal bloom as powdery mildew, or misaligning multispectral images due to foliage obstruction and shaking.

[0036] As one embodiment of the present invention, refer to Figure 1 A flowchart of a machine vision-based blueberry tree pest and disease detection method, refer to... Figure 2 Based on the non-rigid registration logic diagram of the cluster-based manifold topology network, referencing Figure 3 The principle of generating spectral residual saliency maps and signal flow graphs.

[0037] Furthermore, a coaxial imaging optical path is constructed, and the incident light field is spatially decomposed into a first optical path and a second optical path using a beam splitter, which are then projected onto the first image sensor and the second image sensor, respectively. The exposure start time and integration time of the two sensors are synchronously locked by a hardware trigger signal, and the original image pair with a fixed homography mapping relationship is output. Distortion correction preprocessing is performed on the acquired image pair, and radial and tangential distortions are eliminated by using a pre-calibrated camera intrinsic parameter matrix (the calibration of this intrinsic parameter matrix should be based on calibration board images acquired from multiple field-of-view angles, with an average reprojection error of no more than 0.1 pixels and a maximum radial distortion rate of no more than 0.1% at the edge of the field of view), and multi-channel image data to be processed without geometric distortion is output.

[0038] The exposure start time and integration time of the two sensors are synchronously locked by a hardware trigger signal. The time difference between the exposure start times of the two sensors should not exceed 20 microseconds to ensure that the image captured on the fast-moving blueberry bunch is frozen at the same moment. The integration time (exposure time) should be set in the range of 5 milliseconds to 50 milliseconds, not less than 5 milliseconds under natural light conditions (to ensure sufficient signal-to-noise ratio), and not more than 50 milliseconds in overexposure (to avoid excessive highlight clipping in the image).

[0039] Specifically, this coaxial system can utilize a dichroic beam splitter with a cutoff wavelength of 380nm to precisely separate the incident light into a reflected ultraviolet band (center wavelength 365nm) and a transmitted visible light band (400nm-700nm). For the imaging unit, it is recommended to use two 2 / 3-inch CMOS industrial cameras with global shutter functionality (e.g., a resolution of 2448×2048), coupled with a 25mm focal length ultraviolet-specific quartz fixed-focus lens, to ensure that the transmittance exceeds 90% in the ultraviolet band and eliminate imaging blur caused by the absorption of ultraviolet light by ordinary optical glass. In terms of synchronization control, an FPGA is used to generate TTL level pulses, strictly controlling the trigger delay of the two cameras to within 10 microseconds, thereby effectively freezing the high-frequency micro-movements of the ears of fruit in natural environments with wind speeds less than 5m / s. In the distortion correction stage, image data of a 12×9 dot array high-precision ceramic calibration plate needs to be acquired in advance from multiple field-of-view angles. By optimizing the intrinsic parameters, the average reprojection error of the final image is reduced to less than 0.05 pixels, and the maximum radial distortion rate at the edge of the field of view is less than 0.1%. This high-precision hardware configuration and data preprocessing ensure that, within a depth of field range of 500mm to 1000mm, individual blueberries with a diameter of only 8-12mm maintain sub-pixel-level spatial alignment accuracy in dual-channel images, providing a reliable physical benchmark for calculating the spectral residuals of tiny lesions.

[0040] This invention employs a single-prism common-path design based on a beam splitter prism, ensuring that the ultraviolet and visible light sensors have completely consistent perspective and occlusion relationships at any depth of field. This allows subsequent software registration to only address non-rigid deformations in a two-dimensional plane, without having to deal with complex and difficult-to-estimate three-dimensional depth occlusion. This greatly reduces algorithm complexity and improves the upper limit accuracy of registration, making it particularly suitable for imaging blueberry clusters with complex structures and severe occlusion.

[0041] Furthermore, the steps of performing multi-scale speckle detection filtering on the ultraviolet grayscale image and extracting local grayscale minima specifically include: performing multi-scale morphological operations on the ultraviolet grayscale image, using circular isotropic structuring elements with a radius of 3 to 5 pixels, where "multi-scale" refers to using three structuring elements with radii of 3, 4, and 5 pixels respectively to perform morphological operations (erosion followed by dilation) on the input image in sequence, and taking the result with the smallest background variance among the three operation results as the preprocessing output. The purpose of this operation is to effectively suppress salt-and-pepper noise with a width of less than 2 pixels while maintaining the clarity of the fruit boundary. The salt-and-pepper noise in the image is suppressed by using isotropic structuring elements to generate a preprocessed image with a smooth background. The gray-level gradient field of the preprocessed image is calculated, and the speckled structure in the image is located using the determinant response value of the Hessian matrix. The pixels with the minimum gray-level value are searched within the speckled structure region, and it is verified whether the gray-level gradient direction in the neighborhood of the pixel is isotropically convergent. If the convergence condition is met and the gray-level value is lower than the preset adaptive threshold, the pixel is marked as a valid ultraviolet extreme value anchoring node.

[0042] The ultraviolet grayscale image is subjected to multi-scale speckle detection filtering processing. The specific steps are as follows: multi-scale morphological opening operation is performed on the ultraviolet grayscale image, and circular structuring elements with a radius of 3 to 5 pixels are used to eliminate salt-and-pepper noise with a width of less than 2 pixels to generate a pre-processed image with a smooth background; the grayscale gradient field of the pre-processed image is calculated, and the speckle-like structures in the image are located using the determinant response value of the Hessian matrix; in order to cover fruits of different sizes (from immature small fruits to mature large fruits), the standard deviation σ values ​​of the Gaussian smoothing kernel are successively used to calculate the Hessian matrix with values ​​of 2.0, 2.5, and 3. Calculate the response values ​​at each scale from 0 to 8.0 pixels (in 0.5-pixel increments); at the scale position with the strongest speckle response, search for the pixel with the minimum gray value. The criteria are: the gray value of the pixel is strictly less than all other pixels in its 7×7 neighborhood (i.e., a strict local minimum), and the gray value is lower than (the local background gray mean -15 to 25 gray levels); perform gradient direction convergence verification on all candidate minimum points, and count the gray gradient vectors in the 7×7 neighborhood centered on the point. Only when more than 85% of the gradient vectors point to the center point are the pixels marked as valid UV extreme anchor nodes.

[0043] Specifically, due to the dynamic changes in natural lighting and the thermal noise of the sensor itself, ultraviolet images often contain a large amount of non-target high-frequency interference. For input images with a resolution of 5 megapixels, morphological operations are first performed using circular structuring elements with a radius of 3 to 5 pixels. In the Hessian matrix spot detection stage, a multi-scale spatial strategy is adopted, setting the standard deviation of the Gaussian smoothing kernel to step between 2.0 and 8.0 to cover the size changes at different depths from young fruit to mature fruit. To further eliminate interference from leaf-like circular light spots, in the gradient convergence verification, the gradient vector angles within a 7×7 neighborhood centered on the extreme point are statistically analyzed. Only when more than 85% of the gradient vectors point to the central region are they determined as candidate anchor points. In addition, the adaptive threshold is set to be 15 to 25 gray levels lower than the local background mean. This value is determined based on the high absorption rate of approximately 90% of 365nm ultraviolet light by the blueberry bloom layer.

[0044] This method utilizes the geometric prior knowledge that blueberry fruits are approximately spherical and have a uniform powdery coating to effectively distinguish the real fruit center from pseudo-extreme points such as leaf gaps and branch shadows. This ensures that the topology network is constructed on the correct physical entities, prevents registration field tearing caused by anchor point drift, and guarantees the stability of feature extraction under complex lighting and background interference.

[0045] Furthermore, the steps for constructing the clustered manifold topology include: after generating an initial triangular mesh using the Delaunay triangulation algorithm, performing an edge filtering operation based on Euclidean distance, traversing all edges in the mesh, cutting off long edge connections exceeding a preset neighborhood threshold, and eliminating pseudo-associations in disconnected regions of the image; calculating the connectivity of the remaining nodes, and removing isolated nodes and their associated edges with connectivity below a preset value; the final generated topology is defined as a set of vertices and edges describing the local deformation relationships of the image, used to constrain subsequent image space transformations.

[0046] Specifically, the following steps are taken: Using the two-dimensional coordinates of all ultraviolet extreme value anchor nodes as input, their Delaunay triangulation is calculated to generate an initial triangular mesh (this mesh is an undirected graph structure, the node set is the anchor node coordinates, and the edge set is the three sides of the triangle); traversing all edges in the initial triangular mesh, the Euclidean distance of each edge is calculated, and the long edges exceeding a preset neighborhood threshold are cut off. The neighborhood threshold is defined as: the average pixel diameter D of the blueberry fruit multiplied by a coefficient of 1.8 to 2.5, i.e., threshold = D × (1.8 - 2.5). Under typical imaging geometry, D is approximately 50-60 pixels, so the threshold range is 90. -150 pixels); the purpose of this edge filtering is to eliminate spurious connections that cross non-connected regions of the image (such as long edges connecting different fruit bunches); connectivity analysis is performed on the graph after edge filtering, the number of connections (degree) of each node is counted, and isolated nodes with fewer than 3 connections and all their associated edges are removed; the final generated topology network is defined as a graph structure G=(V,E), where V is the set of filtered nodes (UV extreme value anchor nodes), and E is the set of edges (representing the topological connections between adjacent nodes). This network defines the adjacency relationship between fruits in the image and is used to constrain subsequent non-rigid registration.

[0047] Specifically, considering that blueberry bunches are typically distributed in discrete clusters, and that the swaying direction and amplitude of different bunches when blown by the wind are inconsistent, the initial Delaunay triangulation often incorrectly connects physically unrelated independent bunches. The neighborhood threshold is set to 2.0 to 2.5 times the average pixel diameter of the blueberry, for example, 120 to 150 pixels in a typical detection field of view. This value is based on the geometric statistics of tightly packed fruits, accurately identifying and cutting long edges that cross leaf gaps or background gaps, decomposing the globally unique rigid mesh into several independent submanifolds that conform to physical entities. This means that when the bunch on the left side of the field of view sways eastward due to wind, while the bunch on the right side remains relatively stationary, the deformation calculations between the two do not interfere with each other. Simultaneously, the connectivity threshold is set to 3, automatically eliminating isolated edge nodes with fewer than 3 connections. This strategy effectively filters out leaf reflection noise that is occasionally misdetected as anchor points or single detached hanging fruits, ensuring that the final topology network only covers the structurally stable core fruit cluster area, thus providing robust geometric constraints for subsequent processing of nonlinear local deformations of up to 15%.

[0048] This invention introduces a topology optimization strategy based on physical distance constraints, solving the problem of erroneous associations established by Delaunay triangulation in disconnected regions. In natural scenes, the movement directions of different fruit clusters caused by wind are often unrelated. By severing long edges and removing isolated points, this method divides the global mesh into several independent subgraphs that conform to physical entities, blocking the propagation of deformation errors between different fruit clusters. This ensures that the registration model conforms to the real physical movement laws of blueberry clusters and avoids non-physical stretching distortion in the image.

[0049] Before calculating the local displacement vectors of the visible light color image relative to the ultraviolet grayscale image at each node, a global rigid pre-registration step is performed: using a random sampling consensus algorithm to filter interior point pairs from the ultraviolet extreme value anchoring node set and the visible light candidate matching point set, and calculating a six-parameter affine transformation matrix describing the global geometric transformation of the image; using the inverse matrix of the affine transformation matrix to perform a global inverse transformation on the visible light color image, generating an intermediate image that is roughly aligned with the ultraviolet grayscale image at the branch scale; the subsequent calculation of the local displacement vectors is based on the intermediate image and is performed for the residual non-rigid deformation.

[0050] In outdoor scenarios with wind speeds of level 3 or higher, to prevent the thin-plate spline interpolation from getting trapped in local extrema due to large branch swaying, this embodiment introduces a global rigid pre-registration stage. Specifically, the system sets the RANSAC algorithm to iterate 2000 times and the distance threshold for interior point determination to 3.0 pixels. Through calculation, a six-parameter affine matrix containing rotation angle, X-axis translation, Y-axis translation, X-axis scaling, Y-axis scaling, and shearing factor is first extracted. For example, when the entire ear of fruit is detected to have moved 45 pixels to the lower right and rotated 5 degrees clockwise, the system first performs reverse translation and rotation operations on the visible light image. This process reduces the original global displacement error of 50 pixels on average to a local residual of less than 5 pixels. Subsequently, TPS is used to perform fine non-rigid correction on these 5-pixel residuals, so that the final registration calculation only needs to handle the slight trembling of the fruit relative to the stalk, rather than the drastic displacement of the entire branch.

[0051] This invention effectively solves the problem of large displacements caused by wind exceeding the convergence range of the Transformer Positioning System (TPS) by decoupling complex hybrid deformation into two steps: global rigid transformation and local non-rigid deformation. Global pre-registration, like coarse adjustment, first aligns the field of view, and subsequent TPS only needs to handle fine adjustments. This not only avoids non-physical tearing and distortion of the image under large displacements, but also significantly improves the computational speed and convergence stability of the registration algorithm.

[0052] Further, calculating the local displacement vector of the visible light color image relative to the ultraviolet grayscale image at each node specifically includes: projecting the ultraviolet extreme value anchoring node onto the coordinate system of the visible light band color image to form an initial search center; setting a search window with a preset radius centered on each initial search center; performing edge detection and Hough circle transform on the visible light image within the search window to extract the geometric center coordinates of the fruit outline and defining them as visible light candidate matching points; if detection fails, using local grayscale maxima points and performing eccentricity compensation based on the light source direction vector as alternative matching points; constructing a system containing position... A weighted objective function is set for the matching cost term and the topological rigidity cost term. The position matching cost term calculates the Euclidean distance between the transformed coordinates of the current node and the coordinates of the visible light candidate matching point. The topological rigidity cost term is calculated based on the node connection relationship defined by the cluster manifold topology network. Specifically, for each node in the network, the statistical difference between the node's displacement vector and the set of displacement vectors of all directly connected neighboring nodes in the cluster manifold topology network is calculated. By minimizing the objective function, the optimal displacement vector that minimizes the sum of position error and deformation cost is solved.

[0053] The eccentricity supplement specifically refers to the following: if the Hough circle transform fails to detect effectively, an alternative matching strategy is used: extracting local gray-level maxima within the search window, which may deviate from the true center of the spherical fruit; based on the estimated light source direction vector v_light (which can be obtained by estimating the principal direction of the global gradient of the image), the gray-level maxima are repositioned along the v_light direction (projected towards the center of the sphere), and this repositioned point is used as an alternative matching point. The purpose of this correction is to compensate for the positional deviation of the maxima caused by the specular highlight deviating from the center of the sphere.

[0054] The specific calculation method for the statistical dissimilarity is as follows: First, obtain the displacement vectors of all directly connected neighboring nodes of the current node in the cluster manifold topology, and calculate the arithmetic mean of these neighboring node displacement vectors; then, calculate the square of the Euclidean distance between the current node's displacement vector and the arithmetic mean. This squared value is the statistical dissimilarity, which is added as a penalty term to the objective function to force the node's movement trend to be consistent with its local neighborhood.

[0055] The step of minimizing the objective function to solve for the optimal displacement vector is specifically implemented using an iterative gradient descent algorithm. The specific process is as follows: Initialize the displacement vectors of all nodes to zero; in each iteration, calculate the partial derivatives of the position matching cost term and the topological rigidity cost term with respect to the displacement vector to obtain the gradient direction of the total energy function; using a preset step size parameter, control the current node to update its displacement coordinates along the opposite direction of the gradient; repeat the above iterative process until the change in the total energy function is less than a preset convergence threshold or the preset maximum number of iterations is reached. At this point, the output displacement vector is the optimal solution.

[0056] Specifically, in field operations, influenced by gusts of 3 to 5 meters per second, the non-rigid displacement of blueberry bunches between image frames typically fluctuates between 10 and 40 pixels and varies in direction. Therefore, the initial search window radius is set to 35 pixels, a range determined based on extensive experimental data, sufficient to cover over 98% of geometric misalignments caused by branch swaying. Within the search window, instead of directly searching for extreme brightness values, local edge detection and Hough circle transform are performed on the brightness components of the visible light image. The center of the fitted circle is extracted as a candidate matching point in the visible light image to ensure physical alignment with the geometric center of the fruit in the ultraviolet image. If contour extraction fails, local brightness maxima are extracted and eccentricity compensation is performed based on the estimated light source direction vector, using these as alternative matching points. Within the window, multiple candidate locations may be detected, including the true geometric center of the fruit, pseudo-centers formed by curled leaf edges, or background noise. In such cases, relying solely on local shape matching can easily lead to local optima, resulting in non-physical distortions in the registered image. Therefore, the topological rigidity weight coefficient in the objective function is set between 0.6 and 0.8, giving structural constraints a dominant position.

[0057] When a fruit node in a shaded area of ​​the topology network struggles to find a clear corresponding outline, or is induced by a false edge of a leaf vein 20 pixels from its true location, the displacement statistics of its 5 to 8 directly connected neighboring nodes are calculated in real time. If the consistent displacement vector of a neighboring node shows a movement of approximately 12 pixels to the lower left with a standard deviation of less than 5 degrees, the large rigid penalty term in the objective function instantly suppresses the node's tendency to move towards the erroneous false edge, forcibly correcting its displacement vector to a value consistent with the population through the neighborhood traction effect. This mechanism ensures that even with local texture loss, the overall mesh deformation curvature remains smooth, and the displacement direction deviation between adjacent nodes is strictly controlled within 15 degrees. Finally, convergence is achieved within 10 to 15 iterations through iterative solving, stabilizing the average registration residual of the entire image below 1.5 pixels, laying a solid foundation for the subsequent differential extraction of small lesions.

[0058] This invention proposes a joint optimization algorithm that integrates position matching and topological rigid constraints: In visible light images, some fruits may not have an accurate corresponding point due to highlight overflow or leaf occlusion. This method utilizes the neighborhood constraint capability of the topological network, and through the neighbor pull mechanism, it can infer the correct position of the occluded node based on the displacement trend of surrounding nodes. This mechanism endows the algorithm with extremely strong fault tolerance, and can still maintain the continuity and accuracy of the entire ear registration field even when local features are missing.

[0059] Furthermore, during pixel-level geometric correction using the thin-plate spline interpolation function, a smoothing regularization constraint for the transform field is introduced: when solving the weighted coefficient matrix of the thin-plate spline function, a regularization term based on bending energy is superimposed on the objective function, where bending energy is defined as the square integral of the second-order partial derivative of the transform function; by adjusting the regularization parameter to control the rigidity of the transform field, when the displacement vector of a local control point causes the grid deformation curvature to exceed a preset threshold, the regularization parameter is increased to suppress the deformation component whose curvature exceeds the preset threshold, generating a continuous mapping field and preventing pixel folding and / or tearing in local areas of the registered image.

[0060] Based on displacement vectors, these vector endpoints are used as control points for the thin-plate spline interpolation function. The smoothing regularization parameter λ of the thin-plate spline interpolation is initially set to the range of 0.01 to 0.1. When the deformation curvature of a local mesh (defined as the directional change of the displacement vector of adjacent nodes) exceeds a preset threshold (e.g., 15 degrees), the regularization parameter is automatically increased to the range of 1.0 to 10.0 in that local area to suppress non-physical drastic deformation. For all pixels with non-integer coordinates, the corresponding grayscale value is calculated using the cubic spline interpolation function to generate a registered visible light color image. Registration quality assessment: 10% of the anchor nodes (approximately 50-100 nodes) are randomly sampled, and the positional deviation of these nodes before and after registration is calculated. If the deviation of more than 95% of the nodes is less than 1.0 pixel, the registration is successful; otherwise, re-registration or a fault alarm is triggered.

[0061] The regularization parameter is not a globally uniform constant, but a spatial distribution matrix generated based on image texture features. Specific steps include: calculating the structure tensor of each pixel in the ultraviolet grayscale image and extracting the trace of the structure tensor as a local texture complexity index; establishing a monotonically increasing mapping function from texture complexity to the regularization parameter, such that a first value of the regularization parameter is assigned to fruit regions with texture complexity greater than a first preset value, and a second value of the regularization parameter is assigned to background regions with texture complexity less than a second preset value, where the first value is greater than the second value; and substituting the generated parameter matrix into the energy functional of thin-plate spline interpolation to impose pixel-level differential constraints on the rigidity of the transformed field.

[0062] Scope and Limitations of the Thin-Panel Spline Non-rigid Registration Method: The thin-plate spline non-rigid registration method used in this invention is suitable for scenarios where the fruit deformation amplitude does not exceed twice its pixel diameter. When the wind speed exceeds 5 m / s or the overall translation of the fruit bunch exceeds 50 pixels, global rigid pre-registration (affine transformation based on RANSAC fitting) must be performed first to control the residual deformation within a reasonable range. In fruit areas obscured by leaves, this method may produce a local registration error of 5-10 pixels due to the lack of clear registration features; however, the subsequent application of the weight decay mechanism (calculating weights based on gradients) in these areas can effectively suppress false edge artifacts caused by inaccurate registration, and the final disease detection accuracy can still be maintained at an acceptable level (90%).

[0063] Specifically, due to the complexity of the field environment, occasionally individual UV anchoring nodes may produce incorrect displacement estimates due to leaf shading or sudden strong light interference. For example, when the displacement vectors of the vast majority of fruit nodes point southeast with an amplitude of approximately 15 pixels, a single node may mismatch and point northwest with an amplitude of up to 50 pixels. Without constraints, this abrupt change can cause the local image mesh to fold like paper, resulting in pixels in that area disappearing or overlapping after registration.

[0064] To address this, the baseline value of the regularization parameter is set between 0.01 and 0.1 to allow for natural, flexible deformation of the entire fruit bunch. However, once an abnormally high local bending energy density is detected—for example, if the deviation of a node's displacement vector from its neighborhood mean exceeds three times the standard deviation or the absolute distance exceeds 20 pixels—an adaptive penalty mechanism is automatically activated, instantly increasing the regularization parameter of that region to 5.0 or even 10.0. This operation is equivalent to hardening an elastic thin plate in a specific local area of ​​the image, forcing the transformation motion of that node to remain smooth and consistent with its neighborhood, thereby suppressing non-physical high-frequency noise. Actual measurement data shows that after introducing this constraint, in continuous monitoring videos with wind speeds around level 3, the fruit edge tearing rate caused by registration errors decreased from 4.2% to below 0.3%, effectively preventing missed detection of lesions caused by geometric distortion and ensuring the naturalness and realism of the fruit morphology in the synthesized image.

[0065] To ensure that the fruit shape is not distorted during registration, this embodiment constructs an anisotropic regularization field. Specifically, a 5×5 window is used to calculate the structure tensor of the ultraviolet image, and the sum of its eigenvalues ​​is calculated. For the blueberry fruit body region, due to the rich powdery texture, the trace of its structure tensor typically remains in the range of 800 to 1200; while for the background between the fruit or blurred moving leaves, this value is usually below 200. A mapping function is set, forcing the regularization parameter of the fruit region to 8.5, giving it extremely high rigidity, allowing it to only translate and rotate like a rigid body; while the regularization parameter of the background gaps is set to 0.1, giving it extremely high elasticity. Thus, when displacement compensation is needed, the deformation is mainly borne by the gaps between the fruits, thereby mathematically simulating a realistic physical and mechanical model of hard fruits hanging on soft branches.

[0066] This technical solution achieves conformal registration by introducing adaptive regularization based on texture features. It overcomes the shortcomings of traditional TPS, which treats the image as a uniform rubber sheet, causing the fruit to be stretched into an elliptical or irregular shape. By giving the fruit area high rigidity and the background area high elasticity, the geometric authenticity of the disease detection area (fruit surface) is ensured, avoiding the distortion of lesion texture caused by registration deformation.

[0067] This invention adds a smooth protection layer to non-rigid registration by introducing a bending energy regularization term into thin-plate spline interpolation. When local anchor points experience detection errors or abrupt changes, the regularization term automatically limits the second derivative of the transform field, forcing the image to transition smoothly rather than distort sharply. In engineering, this effectively prevents the registration image from collapsing or pixel folding due to individual noise points, ensuring that the final registered image retains true deformation while possessing good visual quality and data fidelity.

[0068] Before calculating the algebraic difference, a preprocessing step based on illumination component compensation is included: convolving the brightness channel of the visible light color image and the inverted ultraviolet image using a large-scale Gaussian kernel function to extract their respective low-frequency illumination components; calculating the ratio map of the ultraviolet illumination component to the visible light illumination component; using the ratio map to perform nonlinear gain correction on the pixel values ​​of the shadow area of ​​the brightness channel of the visible light color image, so that the illumination attenuation characteristics of the corrected visible light shadow area are consistent with those of the ultraviolet image, and then performing histogram specification and difference operation.

[0069] The nonlinear gain correction employs an exponentially-based adaptive correction model. For each pixel in the shadow region, its corrected brightness value is calculated as follows: First, the ratio of the ultraviolet light component to the visible light component at that pixel location is calculated. This ratio is then exponentially calculated, with the exponent set to a preset adjustment coefficient (preferably 0.5 to 0.8). Next, the original brightness value of that pixel in the visible light image is multiplied by the exponentially calculated ratio to obtain the final corrected pixel brightness value. This method simulates the scattering and filling effect of ultraviolet light in the shadow region, preventing noise amplification caused by direct linear brightening.

[0070] To address the difference in shadow depth caused by the different scattering characteristics of ultraviolet and visible light under outdoor tree shade, an illumination component compensation mechanism was introduced. Specifically, a Gaussian kernel with a standard deviation of 80 pixels was selected to extract the background illumination layer of the two images. Due to the strong Rayleigh scattering effect in the ultraviolet band, the gray value of its shadow area (e.g., 45) is often higher than that of the same shadow area in the visible light image (e.g., 20). The illumination ratio of this area was calculated to be approximately 2.25, and a gain mask was generated accordingly. Before performing differential analysis, this mask was used to nonlinearly enhance the brightness of the deep black shadow areas in the visible light image, simulating the scattering and supplementary lighting effect under ultraviolet light. After this correction step, the pseudo-highlight areas in the differential analysis, which were originally caused by excessively deep shadows in the visible light and shallower shadows in the ultraviolet light, were completely eliminated, ensuring that the high-confidence differential signal originated only from the diseased target that does not have ultraviolet absorption characteristics, rather than from uneven illumination.

[0071] Applicable conditions and limitations of the illumination component compensation method: The illumination component compensation mechanism is based on the atmospheric Rayleigh scattering model under natural light and is effective under the following conditions: the light source is natural sunlight (not a unidirectional artificial light source), making the scattered light isotropic; the shadow depth does not exceed 70% of the light attenuation in the bright area (i.e., the gray value of the shadow area is not less than 30% of the gray value of the brightest area), and the compensation mechanism may fail in excessively dark shadow areas; the standard deviation of the Gaussian kernel used in the compensation (80 pixels) corresponds to images with resolutions of VGA and above. If the image resolution is different, the kernel size needs to be adjusted proportionally; the exponent parameter (0.5-0.8) in the compensation should be dynamically adjusted according to the environmental optical conditions: calculate the ratio of the peak to the minimum value of the brightness histogram of the entire image (defined as the light ratio). When the light ratio exceeds 3, increase the exponent parameter from 0.5 to 0.8 to enhance the compensation of dark areas. This compensation mechanism works best for tree shadows that are uniformly illuminated by transmitted light (such as diffuse light through leaves); its effect is limited for deep background shadows that have no light source of their own.

[0072] This technical solution utilizes illumination decomposition and compensation technology to effectively solve the differences in physical and optical properties caused by different wavelengths in multispectral imaging. It eliminates false edges and shadow artifacts commonly found in complex lighting environments (such as dappled shade), significantly reduces the interference of environmental lighting changes on disease detection, and enables the method to maintain an extremely low false detection rate under all-weather natural light conditions.

[0073] Further, the specific steps for generating the spectral residual saliency map by performing weighted difference operation are as follows: convert the visible light color image to the luminance channel, and perform grayscale inversion transformation on the ultraviolet grayscale image; based on the grayscale distribution of the luminance channel, perform histogram specification processing on the inverted ultraviolet image so that the pixel grayscale values ​​of the two channels are in the same dynamic range; calculate the algebraic difference between corresponding pixels of the two channels, and at the same time, calculate the gradient magnitude of each pixel in the luminance channel, and normalize the gradient magnitude to a preset numerical range; calculate the weight coefficient by subtracting the normalized gradient magnitude from the preset unit constant; perform a dot product operation on the algebraic difference and the mapped weight coefficient, and output the spectral residual saliency map.

[0074] The specific steps are as follows: Color space conversion and luminance extraction: The visible light color image is converted from the RGB color space to a color space that includes luminance components (such as YCbCr or Lab space), and the independent luminance channel components are separated, which serve as the visible light reference image; Ultraviolet image grayscale inversion preprocessing: An inversion transformation is performed on the ultraviolet grayscale image, that is, the original grayscale value of each pixel is subtracted from the maximum grayscale value allowed by the current image bit depth (e.g., 255), thereby converting the originally dark areas that absorb ultraviolet light into bright areas, so as to maintain consistency with the luminance logic of the visible light image; Histogram specification matching: The grayscale distribution characteristics (including mean and standard deviation) of the visible light reference image are statistically analyzed. Using this as the target distribution, histogram specification processing is performed on the inverted ultraviolet image to forcibly adjust its grayscale distribution, so that it is consistent with the allowable grayscale distribution in terms of numerical dynamic range. The visible light reference image maintains statistical consistency; gradient-based weighting coefficient calculation: the gradient magnitude of each pixel in the visible light reference image is calculated using an edge detection operator (such as the 3×3 Sobel operator), and the calculated gradient magnitude is normalized to a value range of zero to one; then, the difference between the preset unit constant (value 1) and the normalized gradient magnitude is calculated, and this difference is defined as the weighting coefficient (this step aims to reduce the weight of high-frequency areas such as fruit edges or branch outlines, thereby suppressing edge artifacts caused by small registration errors); weighted difference operation: for each pixel position in the image, the algebraic difference between the pixel value of the visible light reference image and the pixel value of the ultraviolet image after inversion and specification is first calculated, and then the algebraic difference is multiplied by the corresponding weighting coefficient. The final output is the spectral residual saliency map.

[0075] Specifically, for typical 5-megapixel industrial acquisition data, the visible light image is first converted from the RGB color space to the Lab or YCrCb color space, and only the L or Y luminance component containing texture details is extracted as a baseline. In the histogram specification stage, the gray mean (e.g., 128) and standard deviation (e.g., 45) of the luminance component are dynamically calculated, and the inverted ultraviolet image is forced to be mapped to this statistical distribution to ensure that the overall gray levels of the two images are strictly aligned numerically.

[0076] Taking a ripe blueberry with intact bloom as an example, it appears bright white in the visible light spectrum, with a grayscale value of approximately 220. In the ultraviolet spectrum, due to the strong absorption of ultraviolet light by the bloom, it appears dark black (grayscale value of approximately 30). After inversion, the grayscale value jumps to 225. Performing a difference operation at this point, the algebraic difference between the two is only 5, almost completely canceling each other out, thus achieving the invisibility of a healthy background. Conversely, for early powdery mildew lesions, the visible light grayscale value is also 220, but because the mycelium lacks ultraviolet absorption characteristics, it appears bright in the ultraviolet spectrum (grayscale value of approximately 200), turning dark after inversion (grayscale value of approximately 55). The difference result is as high as 165, forming a very strong significant signal. Furthermore, by introducing a 3×3 Sobel operator to calculate gradient weights, when a high-frequency gradient (normalized amplitude greater than 0.8) is detected at the fruit edge, the weight coefficient is reduced to below 0.2, thereby effectively shielding the contour artifacts generated at the edge due to small registration errors (1-2 pixels). This improves the signal-to-noise ratio of the disease target in the final output salient map from 1.8 in a single band to over 10.0.

[0077] This invention designs an adaptive difference algorithm based on histogram specification and gradient weighting: by using histogram matching, it solves the benchmark deviation problem caused by inconsistent photosensitive characteristics of different sensors; by reducing the difference weight in high-gradient edge regions, it cleverly suppresses artifact bright circles caused by small registration residuals at the fruit edges. This signal processing-level optimization, while preserving high-frequency information of lesions, suppresses structural high-frequency noise to the greatest extent, significantly improving the signal-to-noise ratio and reducing the false alarm rate in subsequent segmentation steps.

[0078] Further, the specific steps for threshold segmentation of the spectral residual saliency map include: calculating the gray-level histogram of the spectral residual saliency map; calculating a first threshold and a second threshold based on the statistical properties of the histogram, wherein the first threshold is greater than the second threshold; extracting all pixels with gray-level values ​​greater than the first threshold to form a high-confidence marker map; extracting all pixels with gray-level values ​​greater than the second threshold to form a generalized candidate mask map; using the high-confidence marker map as the marker image for morphological geodesic dilatation operation, and using the generalized candidate mask map as the mask image for morphological geodesic dilatation operation; performing iterative geodesic dilatation operation to segment the connected components in the marker image. Spatial propagation is performed within the constraints of the mask image until the connected components no longer change, and the resulting image is defined as the initial segmentation mask. An Euler-based hole-filling operation is performed on the initial segmentation mask, flipping background pixels completely surrounded by foreground pixels in a four-connected sense to foreground pixels, generating a solid binary mask. The compactness factor and mean edge gradient of each connected component in the binary mask are calculated. Connected components with a compactness factor less than the preset shape factor threshold are removed using a preset shape factor threshold, and connected components with a mean edge gradient greater than the preset gradient threshold are removed using a preset gradient threshold. The remaining connected components are identified as abnormal target regions in the image.

[0079] Thresholding segmentation is performed on the spectral residual saliency map to extract anomalous targets: Threshold calculation: The gray-level histogram of the spectral residual saliency map is calculated; the maximum inter-class variance method is used, that is, all possible gray-level thresholds are traversed, the inter-class variance between foreground and background pixels is calculated, and the gray-level value that maximizes the inter-class variance is selected as the global baseline threshold T_base. Dual threshold setting and adaptive adjustment: A first threshold T1 is defined to extract high-confidence core anomalous regions, and a second threshold T2 is defined to extract generalized candidate regions. Initially, T1 is set to 1.5 times T_base, and T2 is set to 0.6 times T_base. At the same time, the standard deviation of the gray-level histogram is calculated to evaluate image contrast: If the standard deviation is greater than the preset high contrast threshold (e.g., 50), it indicates that the image contrast is strong, so the factor of T1 is reduced (e.g., adjusted to 1.3), and the factor of T2 is reduced (e.g., adjusted to 0.5); if the standard deviation is less than the preset low contrast threshold (e.g., 30), the factor is increased accordingly to adapt to the segmentation requirements under different lighting conditions. Preliminary segmentation and morphological reconstruction: Pixel sets with gray values ​​greater than T1 are extracted to form a high-confidence label set, and pixel sets with gray values ​​greater than T2 are extracted to form a generalized candidate set. Using the high-confidence label set as the seed image and the generalized candidate set as the mask image, an iterative geodesic dilation operation is performed, gradually spreading the connected components in the seed image outward within the constraints of the mask until the connected components no longer change. Hole filling and geometric filtering: Hole filling based on Euler numbers is performed on the dilation results, flipping background pixels completely surrounded by foreground pixels in a four-connected sense to become foreground pixels, generating solid connected regions. The compactness factor (defined as 4π multiplied by the area and divided by the square of the perimeter) and the mean edge gradient of each connected region are calculated. Connected regions with a compactness factor less than a preset range (e.g., 0.35 to 0.55) (such as thin scratches) and connected regions with a mean edge gradient greater than a preset threshold (e.g., 40 to 60) (such as reflections or fresh damage) are removed. The remaining connected regions are identified as detected anomalous target regions.

[0080] Specifically, in field validation of over 2000 samples, for 8-bit spectral residual maps quantized to 0-255 gray levels, the global baseline threshold T was first calculated using the Otsu's method. The first threshold (high confidence) was set to 1.5 times T (typical gray values ​​of 160-180), locking only the core region with the strongest central reflection of the gray mold hyphae; the second threshold (generalized mask) was set to 0.6 times T (typical gray values ​​of 60-80) to cover the blurred diffusion halo around the lesion edges. Through this dual constraint, geodesic dilatation can precisely stop the growth of the core lesion at the weak edge, effectively preventing lesion breakage or underestimation of area that often occurs with single-threshold segmentation. In the post-processing stage, for common blueberry surface twig scratches and highlight noise, the shape factor (compactness) threshold was set to 0.45. This means that thin, elongated scratches with an aspect ratio exceeding 3:1 (whose compactness is typically below 0.3) will be automatically removed. Meanwhile, utilizing the biological principle that disease edges exhibit diffuse reflectivity, a threshold of 50 was set for the mean edge gradient. Water droplet reflections or fresh mechanical damage with drastic edge gradient abrupt changes (mean value exceeding 60) were classified as non-disease interference. Statistical data shows that after processing with this cascaded screening strategy, the detection rate of tiny early lesions with a diameter less than 1 mm increased to 96.5%, and the false detection rate due to powdery mildew was controlled to within 2%.

[0081] This invention employs a cascaded segmentation strategy combining dual-threshold hysteresis judgment and morphological reconstruction, overcoming the drawbacks of traditional single-threshold segmentation which is either one or the other. A high threshold is used to lock in high-confidence core lesions, while a low threshold combined with spatial connectivity is used to recall weak edges. This method ensures the integrity of lesion extraction while avoiding interference from isolated noise. Combined with geometric feature filtering such as compactness, non-disease interference such as branch reflections is further eliminated, resulting in accurate and reliable disease area contours that can adapt to complex and changing light environments in the field.

[0082] This invention innovatively utilizes the ultraviolet absorption characteristics of blueberry bloom as a physical benchmark against disturbances by constructing ultraviolet extreme value anchoring nodes and cluster manifold topology networks. This solves the problem of pixel-level registration of dense small targets under dynamic wind conditions. Simultaneously, by generating spectral residual saliency maps, this method uses the subtractive logic of physical optics to directly eliminate visual interference from healthy bloom at the data level. Thus, without relying on high-performance deep learning models, it achieves accurate differentiation between highly similar-looking bloom and fungal diseases, significantly improving detection robustness and accuracy.

[0083] Example 2:

[0084] This embodiment is applied to an automated mobile robot inspection scenario in an outdoor blueberry plantation. In this scenario, the operating environment faces three severe challenges: First, the natural wind force reaches level 3 to 4, causing the blueberry bunches to sway and deform at high frequency and irregular amplitude within the field of view; second, the lighting conditions change from sunny to cloudy, creating mottled dynamic shadows between the canopy, and the scattering characteristics of the ultraviolet and visible light bands differ significantly in the shadow area; third, the detection targets are early blueberry powdery mildew mycelia and tiny plum weevil eggs. These two types of targets are typically less than 1 mm in diameter and exhibit a white or translucent color characteristic under visible light that is almost identical to the bloom on healthy blueberries.

[0085] As one embodiment of the present invention, refer to Figure 1 A flowchart of a machine vision-based blueberry tree pest and disease detection method, refer to... Figure 2 Based on the non-rigid registration logic diagram of the cluster-based manifold topology network, referencing Figure 3 The principle of generating spectral residual saliency maps and signal flow graphs.

[0086] In the hardware imaging and data acquisition stage, to combat motion blur caused by wind at its source, this embodiment employs an acquisition strategy based on spectral splitting and hardware-level nanosecond synchronization. The system uses a high-throughput dichroic beam splitter with a cutoff wavelength of 380nm to physically decompose the incident light field along the same optical axis into two channels: ultraviolet (center wavelength 365nm) and visible light (400-700nm). To capture the instantaneous shape of the ears of fruit swaying rapidly in the wind, the controller sends a synchronization trigger signal to two global shutter CMOS sensors, strictly locking the exposure time within 5 milliseconds, and using a high-intensity ultraviolet LED strobe light source for instantaneous supplemental lighting. This strategy ensures that even when the ears of fruit are swaying at a speed of tens of centimeters per second, the images acquired by the two channels are strictly frozen and aligned on the time axis, eliminating the "ghosting" effect caused by time differences and laying the physical foundation for subsequent algorithm registration.

[0087] In the image registration stage, the core challenge of this invention is resolving the contradiction between non-rigid deformation and local occlusion. Under strong winds, blueberry bunches do not move as rigid bodies but exhibit complex dynamic characteristics: the main branches sway at low frequencies with large amplitudes, while individual fruits hanging on them tremble randomly around the stem at high frequencies, accompanied by intermittent occlusion from leaves. To address this challenge, this embodiment first applies Gaussian-Laplace filtering to the ultraviolet grayscale image, utilizing the dark spot characteristics formed by the strong absorption of ultraviolet light by blueberry powder to extract the geometric extremum point at the center of the fruit. Unlike edge detection, which is susceptible to light interference, this spot response-based anchor node extraction method is extremely stable; even when the fruit edges are blurred due to motion, the response value at the center remains sharp and reliable. Next, Delaunay triangulation is used to construct a cluster manifold topology network describing the spatial structure of the bunch. During this process, the system performs a dynamic edge filtering operation based on physical neighborhood: when it detects that the edge length connecting two nodes exceeds 2.5 times the average diameter of blueberries, it determines that the two nodes belong to different bunch branches or background noise, and automatically cuts the connection. This step decomposes the complex mesh of the entire image into several independent submanifolds that conform to physical entities, blocking the non-physical propagation of deformation errors caused by wind between different bunches, and preventing the error phenomenon of the left-hand fruit swinging and pulling the right-hand fruit.

[0088] Furthermore, when calculating the displacement vector of the visible light image relative to the ultraviolet image, this embodiment introduces an adaptive thin-plate spline interpolation correction mechanism for windy scenarios. In normal scenarios, TPS strives for global smoothness, but in the strong wind scenario of this embodiment, the displacement of local fruits may exhibit abrupt changes. Therefore, a regularization constraint based on "bending energy" is introduced. When the system detects a sudden increase in the deformation curvature of a local triangular mesh (for example, the displacement vector of a fruit deviates from that of its neighboring nodes, which is physically highly illogical and usually caused by mismatched leaf reflections), it automatically increases the regularization parameter for that region from the default 0.1 to 5.0 or even higher. This adaptive parameter adjustment is equivalent to mathematically "hardening" the mesh in that region instantaneously, forcibly correcting the motion estimation of the abnormal node using the correct displacement trends of surrounding neighboring nodes. Actual measurement data shows that, during continuous monitoring at a wind speed of 5 m / s, this mechanism successfully reduced the fruit edge tearing rate caused by registration errors from 4.5% to below 0.2%, ensuring the spatial overlap between the registered visible light image and the ultraviolet image at the sub-pixel level (error < 0.5 pixels).

[0089] In the core disease feature extraction stage, this embodiment demonstrates how to solve the "visual confusion" problem using spectral residual technology. In visible light images, early powdery mildew mycelia appear as white flocculents, with a gray value (approximately 220) that is almost indistinguishable from the healthy white fruit bloom (approximately 215). Furthermore, due to uneven lighting, the gray value of the highlight areas of the fruit is as high as 230. Traditional deep learning models based on color or texture are prone to misclassifying healthy fruit bloom as disease or omitting disease as background. This invention breaks this deadlock by utilizing the subtractive logic of physical optics: the long-chain alkanes contained in the healthy fruit bloom layer have a strong absorption effect on ultraviolet light in the 300-400nm wavelength band, thus appearing as a deep black in ultraviolet images (gray value approximately 30); while powdery mildew mycelia or physically damaged areas do not have this absorption characteristic, mainly exhibiting diffuse reflection under ultraviolet light, appearing as a brighter medium gray (gray value approximately 150). This embodiment performs a weighted difference operation: First, the ultraviolet image is inverted, turning healthy fruit powdery areas into bright areas (grayscale value 225) and diseased areas into dark areas (grayscale value 105). Then, the visible light image (bright areas) is subtracted from the inverted ultraviolet image (bright areas). For healthy fruit powdery areas, the values ​​are similar, and the difference result approaches zero, perfectly suppressing the background; while for diseased areas, the bright areas of the visible light minus the dark areas of the inverted ultraviolet light produces a strong significant residual signal of over 100. This method does not rely on complex feature learning but directly utilizes the spectral fingerprint of material composition to achieve a dimensionality reduction attack on "metachromatic" targets.

[0090] Furthermore, to address the interference from dappled sunlight caused by outdoor trees, this embodiment specifically introduces a compensation strategy based on illumination components. Due to the Rayleigh scattering effect, ultraviolet light has a stronger scattering and filling ability in shadow areas than visible light, resulting in lighter shadows in ultraviolet images and darker shadows in visible light images. Direct difference would produce false edge artifacts in shadow areas. Therefore, this embodiment extracts the low-frequency illumination components of both images before difference, calculates the illumination ratio map, and performs nonlinear gain compensation on the dark areas of the visible light image to simulate the scattering effect of ultraviolet light. This processing ensures that even against a complex background of intersecting light and shadow, the difference image maintains a pure black background, retaining only the true disease signals.

[0091] Finally, in the target segmentation and post-processing stage, this embodiment employs a strategy combining dual-threshold hysteresis judgment and morphological reconstruction to address the problem of easily fragmented small lesions. For early infection points with a diameter of only 0.5 mm, the system sets a high threshold to lock the core lesion, uses a low threshold to recall blurred edges, and connects the two through geodesic dilatation. Simultaneously, for common mechanical scratches on the fruit surface (such as branch scratches), a "compactness factor" is used for geometric screening: lesions are usually circular or irregularly shaped clumps with high compactness; while scratches are thin and elongated, with low compactness. By eliminating connected regions with a compactness less than 0.3, this invention ensures a small lesion detection rate exceeding 95% while keeping the false alarm rate caused by wind-blown branch scratches to an extremely low level.

[0092] In summary, this embodiment solves the problem of difficulty in distinguishing between static and dynamic pests and diseases and the difficulty in identifying true and false pests by constructing a disturbance-resistant topological manifold registration field and combining it with spectral residual analysis of material properties. It is not only theoretically consistent, but also demonstrates its significant advantages in harsh field environments, providing solid technical support for achieving all-weather, high-precision automated agricultural inspection.

[0093] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A machine vision-based method for detecting diseases and pests in blueberry trees, characterized in that, include: Acquire ultraviolet grayscale images and visible light color images simultaneously collected for the same blueberry target area; The ultraviolet grayscale image is subjected to multi-scale spot detection filtering processing, and the pixel coordinates of the local grayscale minimum points of the blueberry bloom layer are extracted as ultraviolet extreme value anchoring nodes. Based on the spatial adjacency relationship of the ultraviolet extreme value anchor node set, the Delaunay triangulation algorithm is used to construct a clustered manifold topology network describing the spatial distribution structure of fruits in the image; Using the cluster-system manifold topology network as the deformation control skeleton, the local displacement vector of the visible light color image relative to the ultraviolet grayscale image at each node is calculated, and the visible light color image is geometrically corrected at the pixel level using the thin plate spline interpolation function to generate a registered image spatially aligned with the ultraviolet grayscale image. A weighted difference operation is performed on the luminance component of the registered image and the grayscale component of the ultraviolet grayscale image to generate a spectral residual saliency map that suppresses the background and highlights abnormally bright areas. Threshold segmentation is performed on the spectral residual saliency map to extract the binarized mask and determine the abnormal target region.

2. The blueberry tree disease and pest detection method based on machine vision according to claim 1, characterized in that, The simultaneous acquisition of ultraviolet grayscale images and visible light color images specifically involves: constructing a coaxial imaging optical path; using a beam splitter to spatially decompose the incident light field into a first optical path and a second optical path, which are then projected onto a first image sensor and a second image sensor, respectively; synchronously locking the exposure start time and integration time of the two sensors through a hardware trigger signal, and outputting a pair of original images with a fixed homography mapping relationship; performing distortion correction preprocessing on the original image pair, using a pre-calibrated camera intrinsic parameter matrix to eliminate radial and tangential distortion, and outputting multi-channel image data to be processed without geometric distortion.

3. The blueberry tree disease and pest detection method based on machine vision according to claim 1, characterized in that, The steps of performing multi-scale speckle detection filtering on an ultraviolet grayscale image and extracting local grayscale minima specifically include: performing multi-scale morphological operations on the ultraviolet grayscale image, using isotropic structuring elements to suppress salt-and-pepper noise in the image, and generating a pre-processed image with a smooth background; calculating the grayscale gradient field of the pre-processed image, and using the determinant response value of the Hessian matrix to locate speckle-like structures in the image; searching for pixels with grayscale minima within the speckle-like structure region, and verifying whether the grayscale gradient direction in the pixel's neighborhood exhibits an isotropic convergent distribution. If the convergence condition is met and the grayscale value is lower than a preset adaptive threshold, the pixel is marked as a valid ultraviolet extremum anchoring node.

4. The blueberry tree disease and pest detection method based on machine vision according to claim 1, characterized in that, The steps for constructing the clustered manifold topology include: after generating an initial triangular mesh using the Delaunay triangulation algorithm, performing an edge filtering operation based on Euclidean distance, traversing all edges in the mesh, cutting long edge connections that exceed a preset neighborhood threshold, and eliminating pseudo-associations in disconnected regions of the image; calculating the connectivity of the remaining nodes, and removing isolated nodes and their associated edges with connectivity below a preset value; the final generated topology is defined as a set of vertices and edges describing the local deformation relationships of the image, used to constrain subsequent image space transformations.

5. The blueberry tree disease and pest detection method based on machine vision according to claim 1, characterized in that, Calculating the local displacement vector of the visible light color image relative to the ultraviolet grayscale image at each node specifically includes: projecting the ultraviolet extreme value anchoring node onto the coordinate system of the visible light band color image to form an initial search center; setting a search window with a preset radius centered at each initial search center; performing edge detection and Hough circle transform on the visible light image within the search window to extract the geometric center coordinates of the fruit outline, defining them as visible light candidate matching points; if detection fails, using local grayscale maxima points and performing eccentricity compensation based on the light source direction vector as alternative matching points; constructing a system including position matching... The objective function is a weighted sum of the position matching cost and the topological rigidity cost. The position matching cost calculates the Euclidean distance between the transformed coordinates of the current node and the coordinates of the candidate matching point in visible light. The topological rigidity cost is calculated based on the node connection relationship defined in the cluster manifold topology network. Specifically, for each node in the network, the statistical difference between the node's displacement vector and the set of displacement vectors of all directly connected neighboring nodes in the cluster manifold topology network is calculated. By minimizing the objective function, the optimal displacement vector that minimizes the sum of position error and deformation cost is solved.

6. The blueberry tree disease and pest detection method based on machine vision according to claim 1, characterized in that, In the process of pixel-level geometric correction using thin-plate spline interpolation functions, a smoothing regularization constraint for the transform field is introduced: when solving the weighted coefficient matrix of the thin-plate spline function, a regularization term based on bending energy is superimposed on the objective function. The bending energy is defined as the square integral of the second-order partial derivative of the transform function. By adjusting the regularization parameter, the rigidity of the transform field is controlled. When the displacement vector of a local control point causes the grid deformation curvature to exceed a preset threshold, the regularization parameter is increased to suppress the deformation component whose curvature exceeds the preset threshold, thereby generating a continuous mapping field and preventing pixel folding and / or tearing in local areas of the registered image.

7. The blueberry tree disease and pest detection method based on machine vision according to claim 1, characterized in that, The specific steps for performing weighted difference operation to generate a spectral residual saliency map are as follows: convert the visible light color image to the brightness channel, and perform grayscale inversion transformation on the ultraviolet grayscale image; Based on the grayscale distribution of the brightness channel, histogram specification processing is performed on the inverted ultraviolet image to make the pixel grayscale values ​​of the two channels in the same dynamic range. Calculate the algebraic difference between corresponding pixels in the two channels. Simultaneously, calculate the gradient magnitude of each pixel in the brightness channel and normalize the gradient magnitude to a preset numerical range. Calculate the weighting coefficient by subtracting the normalized gradient magnitude from a preset unit constant. Perform a dot product operation between the algebraic difference and the mapped weighting coefficient to output a spectral residual saliency map.

8. The blueberry tree disease and pest detection method based on machine vision according to claim 1, characterized in that, The specific steps for threshold segmentation of the spectral residual saliency map include: calculating the gray-level histogram of the spectral residual saliency map; calculating a first threshold and a second threshold based on the statistical properties of the histogram, wherein the first threshold is greater than the second threshold; extracting all pixels with gray values ​​greater than the first threshold to form a high-confidence marker map; extracting all pixels with gray values ​​greater than the second threshold to form a generalized candidate mask map; using the high-confidence marker map as the marker image for morphological geodesic dilatation operation, and using the generalized candidate mask map as the mask image for morphological geodesic dilatation operation; performing iterative geodesic dilatation operation to transpose the connected components in the marker image onto the mask. Spatial propagation is performed within the constraints of the image until the connected components no longer change, and the resulting image is defined as the initial segmentation mask. An Euler-based hole-filling operation is performed on the initial segmentation mask, flipping background pixels completely surrounded by foreground pixels in a four-connected sense to foreground pixels, generating a solid binary mask. The compactness factor and mean edge gradient of each connected component in the binary mask are calculated. Connected components with a compactness factor less than the preset shape factor threshold are removed using a preset shape factor threshold, and connected components with a mean edge gradient greater than the preset gradient threshold are removed using a preset gradient threshold. The remaining connected components are identified as abnormal target regions in the image.

Images