Visual detection method, device and equipment for diseases and insect pests based on apple phenotype criteria

Abstract

Embodiments of the present disclosure disclose a pest and disease visual detection method, device and equipment based on apple phenotype standard. A specific embodiment of the method comprises: performing double-branch feature extraction on an apple image tensor to generate geometric topology normal vector, highlight core coordinates, global spatial topology atlas and disease spot candidate high-frequency feature atlas; performing surface measurement mapping on a two-dimensional image space corresponding to the apple image tensor to obtain a phenotype feature space; performing scale constraint on the phenotype feature space to generate phenotype constraint parameters; performing illumination component separation and gradient calibration on the apple image tensor to generate pixel gradient features; performing calibration fusion on the pixel gradient features to generate calibration feature information; and performing abnormal residual detection on the calibration feature information to generate a pest and disease detection result. This embodiment reduces the false detection rate of apple disease spots caused by shadows and highlights, and reduces the error rate of detection results under different fruit sizes and lighting conditions.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to a method, apparatus, and device for visual detection of pests and diseases based on apple phenotypic standards. Background Technology

[0002] Early identification of apple diseases and pests typically relies on variations in peel color, texture, and edge patterns. However, the direction, brightness, and shading of light in the natural environment of an orchard are constantly changing, and the curved surface of an apple can cause the same lesion to exhibit different pixel features at different locations, increasing the difficulty of visual detection. Currently, convolutional neural networks are commonly used to directly extract color, texture, and gradient features from two-dimensional apple images, which are then used to identify diseases and pests through classification or object detection networks. Some methods also employ histogram equalization, color space transformation, or image filtering to reduce the impact of lighting changes on the detection results.

[0003] However, existing convolutional neural networks primarily identify lesions in two-dimensional image space. When the light intensity and incident direction in the orchard change, the shadows of branches and leaves, highlights, and reflections from the curved surface of apples can cause abrupt changes in local color and edges, which can easily be misidentified as lesions. At the same time, different fruit sizes and shooting postures can change the scale and location of lesions in the image, leading to missed detection of small lesions in the early stages, misdetection of light and shadow areas, and deviations in lesion localization. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure propose methods, apparatuses, electronic devices, and computer-readable media for visual detection of pests and diseases based on apple phenotypic standards to address the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a visual detection method for pests and diseases based on apple phenotypic standards. The method includes: performing bi-branch feature extraction on an acquired apple image tensor to generate a geometric topological normal vector, specular core coordinates, a global spatial topological map, and a candidate high-frequency feature map of lesions; performing surface metric mapping on the two-dimensional image space corresponding to the apple image tensor based on the global spatial topological map, the candidate high-frequency feature map of lesions, the geometric topological normal vector, and the specular core coordinates to obtain a phenotypic feature space; and performing a surface metric mapping on the phenotypic feature space based on the geometric topological normal vector and the biological anchor points in the global spatial topological map. The aforementioned phenotypic feature space is scale-constrained to generate phenotypic constraint parameters; based on the aforementioned candidate high-frequency feature map of lesions, the aforementioned apple image tensor is subjected to illumination component separation and gradient calibration to generate pixel gradient features; based on the aforementioned phenotypic constraint parameters, the aforementioned geometric topological normal vector, the aforementioned specular core coordinates, and the aforementioned candidate high-frequency feature map of lesions, the aforementioned pixel gradient features are calibrated and fused to generate calibration feature information; the aforementioned calibration feature information is subjected to anomaly residual detection to generate pest and disease detection results, wherein the aforementioned pest and disease detection results include anomaly residual information, pest and disease area information, pest and disease type labels, and location coordinate bounding boxes.

[0007] Secondly, some embodiments of this disclosure provide a visual detection device for pests and diseases based on apple phenotypic standards. The device includes: a bi-branch feature extraction unit configured to perform bi-branch feature extraction on an acquired apple image tensor to generate a geometric topological normal vector, specular core coordinates, a global spatial topological map, and a candidate high-frequency feature map of lesions; a surface metric mapping unit configured to perform surface metric mapping on the two-dimensional image space corresponding to the apple image tensor based on the global spatial topological map, the candidate high-frequency feature map of lesions, the geometric topological normal vector, and the specular core coordinates, to obtain a phenotypic feature space; and a scale constraint unit configured to perform scale constraint on the upper... The phenotypic feature space is subjected to scale constraints to generate phenotypic constraint parameters; the illumination component separation and gradient calibration unit is configured to perform illumination component separation and gradient calibration on the apple image tensor based on the aforementioned candidate high-frequency feature map of lesions to generate pixel gradient features; the calibration fusion unit is configured to perform calibration fusion on the aforementioned pixel gradient features based on the aforementioned phenotypic constraint parameters, the aforementioned geometric topological normal vector, the aforementioned specular core coordinates, and the aforementioned candidate high-frequency feature map of lesions to generate calibration feature information; the generation unit is configured to perform abnormal residual detection on the aforementioned calibration feature information to generate pest and disease detection results, wherein the aforementioned pest and disease detection results include abnormal residual information, pest and disease area information, pest and disease type labels, and location coordinate bounding boxes.

[0008] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0009] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0010] The various embodiments disclosed above have the following beneficial effects: the visual detection method for pests and diseases based on apple phenotypic standards according to some embodiments of this disclosure can reduce the false detection rate of lesions caused by shadows and highlights, improve the detection rate and bounding box positioning accuracy of early small-area lesions, and reduce the fluctuation of detection results under different fruit sizes and lighting conditions. Specifically, the reason for the high false detection rate of related lesions and the low detection rate of small-area lesions is that existing convolutional neural networks mainly identify lesions in two-dimensional image space. When the light intensity and incident direction in the orchard change, the shadows of branches and leaves, highlights, and reflections from the curved surface of apples can cause local color and edge abrupt changes, which are easily misidentified as lesions. At the same time, different fruit sizes and shooting postures will change the scale and position of lesions in the image, leading to missed detection of early small-area lesions, false detection of light and shadow areas, and deviation in lesion positioning. Based on this, some embodiments of the visual detection method for pests and diseases based on apple phenotypic standards disclosed herein first perform bi-branch feature extraction on the acquired apple image tensor to generate geometric topological normal vectors, specular core coordinates, a global spatial topological map, and a high-frequency feature map of lesion candidates. Thus, by preserving the overall structure of the apple, such as its outline, central axis, and biological anchor points, as well as local details such as color anomalies, texture breaks, and gradient abrupt changes, the smooth loss of small-area lesions during continuous downsampling can be reduced. Then, based on the aforementioned global spatial topological map, the aforementioned high-frequency feature map of lesion candidates, the aforementioned geometric topological normal vectors, and the aforementioned specular core coordinates, a surface metric mapping is performed on the two-dimensional image space corresponding to the aforementioned apple image tensor to obtain the phenotypic feature space. Therefore, by rotating and aligning apple images under different shooting postures and converting the two-dimensional pixel positions into uniform spherical surface positions, relatively stable coordinates can be achieved for similar regions on the stem side, waist side, and navel side, thereby reducing mapping deviations and node registration errors at the same phenotypic position. Subsequently, based on the aforementioned geometric topological normal vectors and biological anchor points in the aforementioned global spatial topological map, scale constraints are applied to the aforementioned phenotypic feature space to generate phenotypic constraint parameters. Thus, by adjusting the feature sampling scale through the fruit stalk, fruit navel, edge curvature, and fruit projected area, apples of different sizes or growth stages are assigned similar phenotypic neighborhood ranges, thereby reducing the scaling of lesion features caused by fruit size changes and lowering the detection stability rate affected by size variations in the examined fruit. Next, based on the aforementioned lesion candidate high-frequency feature map, illumination component separation and gradient calibration are performed on the aforementioned apple image tensor to generate pixel gradient features. Thus, by separating environmental brightness variations in the image from the fruit peel's own color and texture, and weakening abnormal gradients generated by highlight and shadow boundaries, while preserving true color abrupt changes and texture breaks, the false detection rate of lesions in the light and shadow areas is reduced. Finally, based on the aforementioned phenotypic constraint parameters, the aforementioned geometric topological normal vectors, the aforementioned highlight core coordinates, and the aforementioned lesion candidate high-frequency feature map, the aforementioned pixel gradient features are calibrated and fused to generate calibration feature information.Therefore, by using local surface normal vectors, highlight positions, and phenotypic scales to determine whether high-frequency features conform to the light-receiving laws of curved surfaces, the false detection rates of shadows and highlights on the fruit are reduced. Finally, abnormal residual detection is performed on the above-mentioned calibration feature information to generate pest and disease detection results. These results include abnormal residual information, pest and disease area information, pest and disease type labels, and location coordinate bounding boxes. By comparing the calibrated local color, texture, and gradient with the normal variation range of healthy apples, residual areas exceeding the normal range are extracted, and pest and disease categories and locations are output, thus improving the accuracy of lesion bounding box localization. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0012] Figure 1 This is a flowchart of some embodiments of the visual detection method for pests and diseases based on apple phenotypic standards according to the present disclosure;

[0013] Figure 2 This is a flowchart illustrating the phenotypic feature construction process in the visual detection method for pests and diseases based on apple phenotypic standards disclosed in this paper.

[0014] Figure 3 This is a schematic diagram of the structure of some embodiments of the visual detection device for pests and diseases based on apple phenotypic standards according to the present disclosure;

[0015] Figure 4 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0018] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0019] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0020] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0021] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] Figure 1 A flowchart 100 of some embodiments of a visual detection method for pests and diseases based on apple phenotypic criteria according to this disclosure is shown. This visual detection method for pests and diseases based on apple phenotypic criteria includes the following steps:

[0023] Step 101: Perform bi-branch feature extraction on the acquired apple image tensor to generate geometric topological normal vectors, specular core coordinates, global spatial topological map, and lesion candidate high-frequency feature map.

[0024] In some embodiments, the execution entity (e.g., a computing device) of the visual detection method for pests and diseases based on apple phenotypic standards can perform bi-branch feature extraction on the acquired apple image tensor to generate geometric topological normal vectors, specular core coordinates, a global spatial topological map, and a candidate high-frequency feature map of lesions. The apple image tensor can be red, green, and blue three-channel color image data output by a monocular image acquisition device (e.g., a monocular camera). Each pixel position in the apple image tensor corresponds to a red channel value, a green channel value, and a blue channel value. For example, for an apple image of size H×W, the red, green, and blue channel values ​​corresponding to each pixel position in the apple image can be combined to form a three-dimensional data structure as the apple image tensor. The global spatial topological map can include the overall outline edge of the fruit, the central axis of symmetry of the fruit, the anchor point of the fruit stalk, and the navel depression area, and records the spatial positional relationships between these structures. The candidate high-frequency feature map of lesions is used to characterize local high-frequency features on the apple surface that may be related to pests and diseases, and can include local abnormal color patches, edge texture break areas, and microscopic gradient mutation point clouds. The aforementioned geometric topological normal vectors are used to characterize the local surface orientation corresponding to different topological positions on the visible surface of the apple. The geometric topological normal vectors can be composed of multiple local unit direction vectors corresponding to each valid topological node. The aforementioned specular core coordinates can be the pixel coordinates corresponding to the position on the visible surface of the apple where the specular reflection response is most concentrated.

[0025] It should be noted that the aforementioned computing devices can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed on the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here. It should be understood that the number of the aforementioned computing devices can be arbitrary, depending on the implementation requirements.

[0026] In practice, the aforementioned execution entity can perform color thresholding on the apple image tensor to obtain the target fruit region, and then fit an ellipse to the target fruit region, using the major axis of the ellipse as the central axis of symmetry of the fruit. Contour nodes are selected at equal intervals along the fruit contour, and the geometric topological normal vector is determined by the tangent directions of adjacent contour nodes. The contour nodes, the central axis of symmetry, and the biometric locations at both ends of the contour are then combined to form a global spatial topological map. Simultaneously, the aforementioned execution entity can extract color anomaly regions, texture anomaly regions, and gradient anomaly points using color difference, local binary mode, and the Laplacian operator, respectively, to form a candidate high-frequency feature map of lesions, and the center of the connected region with the highest brightness is determined as the coordinates of the specular core.

[0027] In some optional implementations of certain embodiments, the aforementioned execution entity may perform bi-branch feature extraction on the acquired apple image tensor through the following steps to generate geometric topological normal vectors, specular core coordinates, global spatial topological map, and lesion candidate high-frequency feature map:

[0028] The first step involves extracting multi-scale phenotypic structural features from the apple image tensor using the first feature extraction branch to generate a global spatial topology map. This first feature extraction branch is used to analyze the overall structural information of the fruit in the apple image and may include a multi-scale backbone network, a cross-scale feature fusion layer, a structure prediction layer, and a local normal vector prediction layer. In practice, the execution entity first inputs the apple image tensor into the multi-scale backbone network included in the first feature extraction branch. This multi-scale backbone network sequentially sets multiple convolutional processing layers and downsampling processing layers, which can employ a combination of residual neural networks and feature pyramid networks. Different convolutional processing layers output hierarchical feature maps with different spatial resolutions. Hierarchical feature maps with higher spatial resolution preserve the fruit's outline boundaries and local structural positions, while hierarchical feature maps with lower spatial resolution represent the overall semantic differences between the apple fruit and the background. The cross-scale feature fusion layer upsamples the hierarchical feature maps with lower spatial resolution, and then concatenates and convolves the upsampled hierarchical feature maps with the hierarchical feature maps with higher spatial resolution to obtain multi-scale phenotypic structural features. As an example, the cross-scale feature fusion layer can adopt the top-down fusion structure in the feature pyramid network, or it can adopt the bidirectional feature pyramid network.

[0029] Then, the aforementioned execution entity can input multi-scale phenotypic structural features into the structure prediction layer, which includes the contour prediction layer, axis prediction layer, fruit stem keypoint prediction layer, and fruit navel region prediction layer. The contour prediction layer is used to predict the fruit category probability for each pixel and output the corresponding fruit region probability map. It can use a fully convolutional network or the semantic segmentation output head of a DeepLab deep learning network model. Afterward, the aforementioned execution entity can mark pixels whose corresponding fruit region probability is greater than or equal to a preset fruit probability threshold (e.g., 0.85) as fruit pixels, perform connected component analysis (e.g., eight-neighbor connected component analysis) on each marked fruit pixel, and perform Canny edge detection on the binary mask corresponding to the retained target fruit region to obtain each contour edge pixel, and then combine the contour edge pixels to form the overall contour edge of the fruit.

[0030] The axis prediction layer predicts the probability that each pixel belongs to the fruit's central axis of symmetry and outputs a central axis of symmetry probability map. This can be achieved using a heatmap regression head from either an HRNet (High-Resolution Network) model or an SHN (Stacked Hourglass Networks) model. Next, the execution body retains pixels whose central axis of symmetry probability is greater than or equal to a preset axis of symmetry probability threshold (e.g., 0.85). The retained pixels are then thinned to maintain a single-pixel width. Random sampling consistency filtering and least-squares line fitting are performed on the thinned axis pixel coordinates to obtain the fruit's central axis of symmetry.

[0031] The fruit stalk keypoint prediction layer is used to predict the fruit stalk position response map corresponding to the fruit stalk connection position output by the prediction layer. The position with the largest response value can be used as the fruit stalk anchor point, and the same network structure as the axis prediction layer can be used. Secondly, the aforementioned execution entity can determine the position of the largest response in the fruit stalk position response map and determine the position of the largest response as the fruit stalk anchor point corresponding to the connection position between the fruit stalk and the fruit body.

[0032] The navel region prediction layer predicts the probability that each pixel belongs to the navel region and outputs a navel region probability map. This can be achieved using a fully convolutional neural network model or the region segmentation output header in a DeepLab deep learning network model. Then, the execution unit identifies pixels with a navel region probability greater than or equal to a preset navel probability threshold (e.g., 0.9) as navel regions. Furthermore, the center of the navel region can be determined by weighting the pixel coordinates based on the navel region probabilities of each pixel within the navel region.

[0033] Subsequently, the aforementioned execution entity can define the pixel positions of each contour edge in the overall outline of the fruit, the pixel positions of each axis in the central axis of symmetry of the fruit, the anchor point of the fruit stem, and the pixel coordinates of each region in the fruit navel depression as global structure nodes. Furthermore, the aforementioned execution entity can define the pixel positions of each contour edge as contour nodes, the pixel positions of each axis as axis nodes, the anchor point of the fruit stem as a fruit stem anchor node, and the pixel coordinates of each region in the fruit navel depression as a fruit navel region node.

[0034] Finally, adjacent contour nodes can be connected according to the arrangement order of the fruit's overall outline edges (i.e., from smallest to largest ordinate) to form contour connection relationships. Adjacent axis nodes can be connected according to the extension order of the fruit's central symmetry axis (i.e., the end closest to the fruit stalk anchor point is determined as the axis start point, and the end closest to the center of the fruit navel depression is determined as the axis end point; the extension order is from the fruit stalk side to the fruit navel side) to form axis connection relationships. The fruit stalk anchor point and the center of the fruit navel depression are respectively connected to the nearest axis node to form biological anchoring connection relationships. The aforementioned global structural nodes, corresponding node features, and node connection relationships together form a global spatial topology map. Node features can be composed of the pixel coordinates corresponding to the global structural nodes, node network features (i.e., feature vectors read from the corresponding positions in the aforementioned multi-scale phenotypic structural features), and node confidence (i.e., obtained by concatenating the corresponding fruit region probability, central symmetry axis probability, fruit stalk position response value, and fruit navel region probability).

[0035] The second step involves extracting high-resolution pixel features from the apple image tensor using a second feature extraction branch to generate a candidate high-frequency feature map of lesions. This second feature extraction branch preserves and analyzes local color, texture, and gradient information in the apple image and may include a high-resolution preservation network, a high-frequency feature prediction layer, and a specular coordinate extraction layer. In practice, the execution entity first inputs the apple image tensor into the high-resolution preservation network included in the second feature extraction branch. As an example, the high-resolution preservation network can be an HRNet (High-Resolution Network) model.

[0036] The aforementioned high-resolution preservation network can be configured with parallel high-resolution and low-resolution feature pathways. The high-resolution feature pathway continuously preserves local color, texture edges, and subtle gradient variations in the apple image, while the low-resolution feature pathway extracts the contextual relationships between adjacent pixel regions. Feature information can be exchanged between different resolution feature pathways through upsampling, downsampling, and channel fusion operations to obtain high-resolution pixel features.

[0037] Then, the aforementioned execution entity can input high-resolution pixel features into the color anomaly prediction layer, texture breakage prediction layer, and gradient abrupt change prediction layer, respectively. The color anomaly prediction layer processes the high-resolution pixel features through one or more convolutional layers, and then generates a color anomaly response map through the output convolutional layer. A pixel classification head with a fully convolutional network as its backbone can be used. Then, the aforementioned execution entity combines pixels whose corresponding color anomaly response values ​​are greater than or equal to a preset color response threshold (e.g., 0.87) into color anomaly connected regions, and determines each color anomaly connected region as a local color anomaly patch.

[0038] The texture break prediction layer is used to fuse edge features from different convolutional layers and output a texture continuity response map or a texture break response map. A Holistic-Nested Edge Detection (HED) network can be used. Then, the execution unit compares the texture orientation and texture intensity of adjacent locations, and groups pixels with abrupt changes in texture orientation or texture continuity response values ​​below a preset continuity threshold (e.g., 0.6) as edge texture break regions.

[0039] The gradient mutation prediction layer can employ a structure combining fixed-gradient convolutional kernels and trainable convolutional layers. For example, the horizontal and vertical gradients are first extracted using the Sobel or Scharr operator, and then a gradient mutation response map is generated using a fixed-gradient convolutional kernel and a trainable convolutional layer. Next, the execution entity can identify each pixel whose gradient mutation response value is greater than or equal to a preset gradient response threshold (e.g., 0.85) as a gradient mutation point, and record each gradient mutation point according to its spatial location to obtain a micro-gradient mutation point cloud.

[0040] Secondly, the aforementioned execution entity can identify local color anomaly patches, edge texture breakage regions, and micro-gradient abrupt change points as high-frequency feature nodes. Each high-frequency feature node can use its corresponding image coordinates, color response value, texture breakage response value, gradient abrupt change response value, and original color value as its node features.

[0041] Subsequently, for every two high-frequency feature nodes whose spatial distance (i.e., the Euclidean distance between the image coordinates corresponding to the two high-frequency feature nodes) is less than or equal to a preset adjacency distance, the aforementioned execution entity establishes a spatial adjacency relationship between the two high-frequency feature nodes. Finally, the aforementioned execution entity can combine each high-frequency feature node, its corresponding node features, and its spatial adjacency relationships to form a candidate high-frequency feature map of lesions.

[0042] It should be noted that the aforementioned preset adjacency distance is used to limit the establishment of connections between high-frequency feature nodes that are spatially close. In practice, the preset adjacency distance can be determined by the apple image resolution and the minimum size of the target lesion. Specifically, the pixel length corresponding to the expected minimum lesion size can be used as a reference distance and adjusted according to the image resolution. For example, for an apple image with a resolution of 1024×1024 pixels, a distance within the range of 5 to 15 pixels can be used as the preset adjacency distance.

[0043] The third step involves extracting local normal vectors from the aforementioned global spatial topology map to generate geometric topological normal vectors. In practice, the execution entity first determines the circumscribed rectangle and its center position of the target fruit region based on the overall outline edge of the fruit. Then, it determines the lateral and longitudinal offsets of each global structural node relative to the center of the circumscribed rectangle. These offsets are then normalized using the width and height of the circumscribed rectangle, resulting in relative lateral and longitudinal positions that together constitute the relative node positions. Adjacent node features can be the node features corresponding to other global structural nodes directly connected to the corresponding global structural node through node connection relationships.

[0044] It should be noted that for edge contour nodes, adjacent nodes are the preceding and following contour nodes in the edge contour pixel sequence. For axis nodes, adjacent nodes are the preceding and following axis nodes along the axis extension direction. For fruit stem anchoring nodes and fruit navel region nodes, adjacent nodes are the axis nodes with which they establish biological anchoring connections. Then, the aforementioned execution entity can process each global structure node in the global spatial topology map through the local normal vector prediction layer included in the first feature extraction branch. The local normal vector prediction layer can read the multi-scale phenotypic structure features, node relative positions, and adjacent node features corresponding to each global structure point. It aggregates adjacent node features through graph convolutional layers or graph attention layers, and reads the multi-scale phenotypic structure features corresponding to each global structure node, converting the node relative positions into positional features. Then, it weights and accumulates the features of each adjacent node according to the node connection relationship to obtain the neighborhood aggregation features. Subsequently, the multi-scale phenotypic structure features, positional features, and neighborhood aggregation features are concatenated and input into a fully connected layer or convolutional layer, outputting three directional components corresponding to the global structure nodes. The three directional components represent the directional response values ​​of the local surface in the horizontal direction of the image, the vertical direction of the image, and the direction of the surface outward normal, respectively.

[0045] Next, the execution entity determines the combined length (i.e., the root of the sum of squares) of the three directional components, and then divides each of the three directional components by this combined length to ensure that the lengths of the processed directional vectors remain consistent. This eliminates the differences in the amplitude of the directional responses of different nodes, retaining only the local surface orientation. Furthermore, to avoid opposite directions for the normal vectors of adjacent nodes, the execution entity compares the directional vector corresponding to each node with the reference direction pointing from the center of the fruit to that node. When the two directions are opposite, all directional components of the node's directional vector are reversed. Finally, the execution entity arranges the corresponding local unit directional vectors according to the spatial position of each effective topological node in the global topological graph, thus obtaining the geometric topological normal vector.

[0046] It should be noted that each prediction layer (e.g., contour prediction layer, axis prediction layer, fruit stem keypoint prediction layer, fruit navel region prediction layer, color anomaly prediction layer, texture breakage prediction layer, and gradient mutation prediction layer) can share the features output by the first or second feature extraction branch and are trained using a multi-task joint training method. The contour prediction layer and fruit navel region prediction layer can use region masks as supervision information, the axis prediction layer and fruit stem keypoint prediction layer can use heatmaps as supervision information, and the local normal vector prediction layer can use pre-labeled or fitted local normal directions as supervision information.

[0047] The fourth step involves extracting the highlight coordinates from the aforementioned candidate high-frequency feature map of lesions to generate highlight core coordinates. In practice, firstly, the executing entity can determine the original color value corresponding to each high-frequency feature node in the candidate high-frequency feature map of lesions and convert the original color value into a brightness value. For example, the brightness value Y = 0.299R + 0.587G + 0.114B can be determined using the following expression. Here, R, G, and B represent the red, green, and blue color channel values, respectively. Y represents the pixel brightness. Then, the executing entity can determine the brightness value distribution of each high-frequency feature node within the target fruit area and identify each high-frequency feature node whose corresponding brightness value reaches a preset highlight quantile value (e.g., 0.75 quantile) as a highlight candidate node. Then, according to the spatial adjacency relationship between high-frequency feature nodes, the interconnected highlight candidate nodes are combined into a highlight candidate region. Afterward, for each highlight candidate region, the executing entity accumulates the brightness excess of each high-frequency feature node within the region. The brightness excess can be obtained by subtracting the highlight screening threshold (e.g., 0.82) from the corresponding brightness value, and the highlight candidate region with the largest cumulative value is determined as the target highlight region. Finally, the execution entity converts the brightness excess of each high-frequency feature node in the target highlight region into a brightness weight (i.e., the ratio of the brightness excess to the sum of the brightness excesses of all nodes in the target highlight region), and then weights the pixel coordinates corresponding to each high-frequency feature node to obtain the horizontal coordinates and vertical coordinates of the highlight core, thus forming the highlight core coordinates.

[0048] Step 102: Based on the global spatial topology map, the candidate high-frequency feature map of lesions, the geometric topological normal vector, and the coordinates of the specular core, perform surface metric mapping on the two-dimensional image space corresponding to the apple image tensor to obtain the phenotypic feature space.

[0049] In some embodiments, the execution entity may perform surface metric mapping on the two-dimensional image space corresponding to the apple image tensor based on the global spatial topology map, the candidate high-frequency feature map of lesions, the geometric topology normal vector, and the specular core coordinates to obtain the phenotypic feature space.

[0050] In practice, the aforementioned execution entity can perform ellipse fitting on the overall outline edge of the fruit to obtain the ellipse center, major axis, and minor axis, and then normalize each image position using the major and minor axes. Subsequently, the normalized image positions are mapped onto a pre-established semi-ellipsoidal surface, and the global spatial topology map, lesion candidate high-frequency feature map, geometric topological normal vector, and specular core coordinates are written into the corresponding semi-ellipsoidal grid positions, thereby forming a phenotypic feature space.

[0051] In some optional implementations of certain embodiments, the execution entity may perform surface metric mapping on the two-dimensional image space corresponding to the apple image tensor based on the global spatial topology map, the lesion candidate high-frequency feature map, the geometric topological normal vector, and the specular core coordinates to obtain the phenotypic feature space:

[0052] The first step is to construct an affine transformation matrix based on the angular offset between the fruit's central axis of symmetry and the horizontal baseline. This affine transformation matrix is ​​used to rotate the fruit's central axis of symmetry to the direction of a preset vertical baseline. In practice, firstly, the executing entity can determine the pixel coordinates of each axis node corresponding to the fruit's central axis of symmetry in the global spatial topology map. Then, the executing entity can select axis nodes located at both ends of the fruit's central axis of symmetry and determine the direction angle of the fruit's central axis of symmetry through the difference in horizontal and vertical coordinates between the two axis nodes. Next, the executing entity can determine the horizontal direction of the image as the horizontal baseline and determine the angular offset between the fruit's central axis of symmetry and the horizontal baseline. Then, the executing entity can determine the geometric center of the region enclosed by the overall outline edge of the fruit as the rotation center. Subsequently, the executing entity can determine the angle between the fruit's central axis of symmetry and the preset vertical baseline as the rotation correction angle. The preset vertical baseline can be a straight line passing through the rotation center and extending along the vertical direction of the image. Finally, the aforementioned execution entity can write the rotation center, rotation correction angle, and the lateral and longitudinal translations generated during the rotation into a two-dimensional affine transformation matrix to obtain the affine transformation matrix.

[0053] The second step involves rotating and aligning the global spatial topology map, the candidate high-frequency feature map of lesions, the geometric topological normal vector, and the specular core coordinates using the aforementioned affine transformation matrix. In practice, the executing entity can process the global spatial topology map, the candidate high-frequency feature map of lesions, the geometric topological normal vector, and the specular core coordinates respectively using the aforementioned affine transformation matrix.

[0054] For the aforementioned global spatial topology map, the executing entity can perform coordinate transformation on the pixel coordinates corresponding to each global structure node using an affine transformation matrix to obtain the rotated pixel coordinates of each global structure node. It should be noted that the network features, node confidence, and node connectivity of each global structure node remain unchanged.

[0055] For the aforementioned candidate high-frequency feature maps of lesions, the execution entity can perform coordinate transformation on the image coordinates corresponding to each high-frequency feature node using an affine transformation matrix to obtain the rotated image coordinates corresponding to each high-frequency feature node. It should be noted that the color response value, texture breakage response value, gradient abrupt change response value, and original color value of each high-frequency feature node remain unchanged.

[0056] For the aforementioned geometric topological normal vectors, the execution entity can synchronously rotate the image horizontal and vertical components of each local unit direction vector using the aforementioned rotation correction angle, while keeping the surface outward normal direction component unchanged. This ensures that the rotated local unit direction vectors maintain spatial correspondence with their corresponding global structure nodes. For the aforementioned specular core coordinates, the execution entity can perform a coordinate transformation on the specular core coordinates using an affine transformation matrix to obtain the rotated specular core coordinates. Thus, the global spatial topological map, the lesion candidate high-frequency feature map, the geometric topological normal vectors, and the specular core coordinates are unified into the same rotated and aligned image coordinate system.

[0057] The third step involves determining the vertical and horizontal biological scales based on the rotated and aligned global spatial topology. In practice, firstly, the execution entity can read the pixel coordinates corresponding to the fruit stem anchoring node and each fruit navel region node from the rotated and aligned global spatial topology. Then, the execution entity can use the fruit navel region probabilities corresponding to each fruit navel region node within the target fruit navel depression region as the original position weights, and normalize each original position weight by summing them. Finally, the rotated horizontal and vertical pixel coordinates of each fruit navel region node are weighted and summed using the normalized position weights to obtain the center of the rotated and aligned fruit navel depression region.

[0058] Then, the aforementioned execution entity can determine the discrete pixel distance between the rotated and aligned fruit stem anchor point and the center of the rotated and aligned fruit navel depression region. Since the fruit stem anchor point and the center of the fruit navel depression region are basically located on the preset vertical baseline after rotation and alignment, the vertical pixel distance between them can be determined as the vertical biological scale. Finally, the aforementioned execution entity can read the horizontal pixel coordinates corresponding to each edge contour node after rotation and alignment, and determine the minimum and maximum horizontal pixel coordinates respectively. The distance between the maximum and minimum horizontal pixel coordinates is the width of the maximum horizontal bounding rectangle corresponding to the overall contour edge of the fruit, and the width of the maximum horizontal bounding rectangle is determined as the horizontal biological scale.

[0059] The fourth step involves constructing a spherical projection operator based on the aforementioned vertical and horizontal biological scales. This spherical projection operator establishes the correspondence between the position of the two-dimensional image and the relative position of the fitted sphere on the surface. In practice, the execution entity can pre-establish a normalized fitted sphere and divide the visible surface of the fitted sphere into various spherical topological nodes according to a preset spherical grid density. Each spherical topological node corresponds to a relative latitude, relative longitude, and a spherical node number. For example, the preset spherical grid density can be set to 500 to 1200 spherical topological nodes per unit surface area.

[0060] First, the aforementioned implementing entity can determine the midpoint between the fruit stem anchor point and the center of the fruit navel depression area as the origin of the biological coordinate system, determine the central symmetry axis of the fruit after rotation and alignment as the longitudinal reference axis, and determine the straight line passing through the origin of the biological coordinate system and perpendicular to the longitudinal reference axis as the transverse reference axis.

[0061] Then, for each image position in the rotated and aligned target fruit region, the aforementioned execution entity can determine the longitudinal pixel distance of the image position along the longitudinal reference axis relative to the origin of the biological coordinates, and normalize this longitudinal pixel distance using a 1 / 2 vertical biological scale to obtain the longitudinally normalized position. Afterwards, the aforementioned execution entity can also determine the lateral pixel distance of the image position to the longitudinal reference axis, and normalize this lateral pixel distance using a 1 / 2 horizontal biological scale to obtain the laterally normalized position. The side closer to the fruit stem can be set as the positive longitudinal direction, and the side closer to the fruit navel can be set as the negative longitudinal direction. One side of the longitudinal reference axis can be set as the positive lateral direction, and the other side can be set as the negative lateral direction.

[0062] Subsequently, the aforementioned execution entity can convert the longitudinally normalized position (i.e., the longitudinally normalized position and the laterally normalized position) into relative latitude on the fitted sphere using inverse trigonometric functions. Specifically, the execution entity can restrict the longitudinally normalized position to a value range of -1 to 1, and convert it into relative latitude on the fitted sphere using an arcsine function. Then, the execution entity can determine the cosine value of the relative latitude as the normalized cross-sectional radius of the fitted sphere at that relative latitude, and use this normalized cross-sectional radius to correct the laterally normalized position (i.e., the ratio of the laterally normalized position to the normalized cross-sectional radius is determined as the laterally corrected position), thus obtaining the laterally corrected position. Subsequently, the execution entity can restrict the laterally corrected position to a value range of -1 to 1, and convert it into relative longitude using an arcsine function.

[0063] Next, the execution entity can select the spherical topology node whose relative latitude and longitude are closest to the image position from a preset set of spherical topology nodes, and record the correspondence between the current image position, relative latitude, relative longitude, and spherical node number. Finally, the execution entity can combine the coordinate transformation rules and the correspondence between all image positions and spherical topology nodes into a spherical projection operator.

[0064] The fifth step involves using the aforementioned spherical projection operator to perform spherical projection on the rotated and aligned global spatial topology map and the candidate high-frequency feature map of lesions, thereby generating a spherical topology feature map. In practice, the aforementioned execution entity can utilize the spherical projection operator to perform spherical projection on the rotated and aligned global spatial topology map and the candidate high-frequency feature map of lesions, respectively.

[0065] For each global structure node in the rotated and aligned global spatial topology graph, the aforementioned execution entity can determine the spherical topology node corresponding to the rotated pixel coordinates of the global structure node using the spherical projection operator, and write the node network features and node confidence of the global structure node into the global structure features of the corresponding spherical topology node.

[0066] For each high-frequency feature node in the candidate high-frequency feature map of the lesion after rotation and alignment, the above-mentioned execution entity can determine the spherical topology node corresponding to the rotated image coordinates of the high-frequency feature node through the spherical projection operator, and write the color response value, texture break response value, gradient mutation response value and original color value corresponding to the high-frequency feature node into the high-frequency pixel features of the corresponding spherical topology node.

[0067] When multiple nodes of the same type (e.g., edge contour nodes, axis nodes) are projected onto the same spherical topology node, the aforementioned execution entity can perform a weighted average of the node confidence or response value of each node on the network features of each node, and write the weighted average result into the corresponding feature (e.g., global structural features or high-frequency pixel features).

[0068] Finally, the aforementioned execution entity can also establish spherical node connection relationships according to the surface adjacency relationships between various spherical topological nodes in the spherical topological mesh, and combine various spherical topological nodes, spherical node connection relationships, global structural features and high-frequency pixel features into a spherical topological feature map.

[0069] The sixth step involves attaching the rotated and aligned geometric topological normal vectors and specular core coordinates to the corresponding topological nodes in the aforementioned spherical topological feature map to generate a phenotypic feature space. In practice, firstly, the aforementioned execution entity can determine the corresponding spherical topological nodes by using the spherical projection operator, based on the global structural node positions corresponding to each local unit direction vector in the rotated and aligned global spatial topological map.

[0070] Then, the aforementioned execution entity can use each local unit direction vector as the local surface normal vector of the corresponding spherical topology node. When multiple local unit direction vectors correspond to the same spherical topology node, the local unit direction vectors can be averaged separately, and then the averaged local unit direction vectors can be normalized to obtain the local surface normal vector corresponding to the spherical topology node.

[0071] Subsequently, the aforementioned execution entity can determine the relative latitude, relative longitude, and spherical node number corresponding to the rotated specular core coordinates using the spherical projection operator, and mark this spherical topology node as a specular core node, as well as record the specular core marker, specular core response value, and corresponding relative latitude and longitude. Thus, each spherical topology node can include a spherical node number, relative latitude, relative longitude, global structural features, high-frequency pixel features, local surface normal vector, and specular core marker; values ​​that do not exist can be null.

[0072] Next, the aforementioned execution entity can concatenate the global structural features, high-frequency pixel features, and local surface normal vectors corresponding to each spherical topological node to obtain the node's high-dimensional features. These high-dimensional features are then input into an adaptive gating network. The adaptive gating network generates retained weights for each feature dimension in the node's high-dimensional features through a fully connected layer. These retained weights are then used to perform dimension-wise weighting on the corresponding feature dimensions. Finally, a linear mapping layer converts the weighted node high-dimensional features into phenotypic features of a preset dimension. For example, the high-dimensional features corresponding to each spherical topological node can be converted into 128-dimensional phenotypic features.

[0073] Finally, the aforementioned executing entity can jointly determine the phenotypic feature space by the various spherical topological nodes, the connection relationships between spherical nodes, the relative surface coordinates, and the phenotypic features corresponding to each spherical topological node.

[0074] like Figure 2 As shown in the figure, the steps for performing affine rotation alignment on an apple image, synchronous alignment between the global spatial topology map and the candidate high-frequency feature map of lesions, biological scaling determination, spherical projection, generation of spherical topology feature map, and mounting geometric topology normal vectors and specular core coordinates to spherical topology nodes to generate phenotypic feature space are illustrated.

[0075] Step 103: Based on the geometric topological normal vector and the biological anchor points in the global spatial topological map, scale constraints are applied to the phenotypic feature space to generate phenotypic constraint parameters.

[0076] In some embodiments, the execution entity can apply scale constraints to the phenotypic feature space based on the geometric topological normal vector and the biological anchor points in the global spatial topological map to generate phenotypic constraint parameters. The biological anchor points can be the fruit stalk anchor point and the center of the fruit navel depression region.

[0077] In practice, the aforementioned implementing entity can determine the distance from the fruit stalk anchor point to the center of the fruit navel depression as the longitudinal reference scale, and statistically analyze the fruit outline width at different longitudinal positions to obtain a transverse scale sequence. Subsequently, the shape and orientation of the feature sampling window are corrected using the geometric topological normal vectors corresponding to each surface position, and the sampling range in the phenotypic feature space is adjusted according to the longitudinal reference scale and the transverse scale sequence. The adjusted longitudinal scale, transverse scale, and sampling window parameters are then determined as phenotypic constraint parameters.

[0078] In some optional implementations of certain embodiments, the execution entity may perform scale constraints on the phenotypic feature space based on the geometric topological normal vector and the biological anchor points in the global spatial topological map to generate phenotypic constraint parameters:

[0079] The first step is to extract curvature features from the overall contour edges of the fruit included in the aforementioned global spatial topology map to generate edge curvature points. In practice, the execution entity can first read the pixel coordinates corresponding to each edge contour node from the aforementioned global spatial topology map, and then, starting from any contour node, sequentially search for contour nodes adjacent to the current contour node that have not yet been sorted, according to the eight-neighborhood chain code sorting method, until returning to the starting contour node, to obtain a closed contour node sequence arranged clockwise or counterclockwise.

[0080] Then, for each contour node in the closed contour node sequence, the execution entity can select two contour nodes separated by a preset number of steps before and after the current contour node. For example, it can select the 5th contour node before and the 5th contour node after the current contour node. Subsequently, the execution entity can determine the first contour direction pointing from the current contour node to the previous contour node, and the second contour direction pointing from the current contour node to the next contour node, and determine the angle between the first and second contour directions. Next, the execution entity can divide the angle by the contour arc length between the previous and next contour nodes to obtain the curvature response value corresponding to the current contour node. Then, the execution entity can retain all contour nodes whose curvature response values ​​are greater than or equal to a preset curvature response threshold, and perform local maxima filtering on the retained contour nodes. For example, only the contour node with the largest curvature response value is retained within a preset contour neighborhood to avoid generating multiple adjacent edge curvature points in the same curved region. Finally, the execution entity can determine the retained contour nodes as edge curvature points. Each edge curvature point can correspond to pixel coordinates, spherical node number, curvature response value, contour extension direction, and node network features.

[0081] It should be noted that the aforementioned preset curvature response threshold can be determined using statistical results of the curvature of a healthy apple contour. Specifically, the curvature values ​​corresponding to the contour nodes of multiple standard apple samples can be calculated, and the normal range of contour curvature variation can be determined. The curvature values ​​corresponding to positions exceeding the normal range of variation can be used as the preset curvature response threshold. For example, the preset curvature response threshold can be determined by adding twice the standard deviation to the mean curvature of a healthy apple contour. That is, if the mean curvature of a healthy apple contour is 0.12 and the standard deviation is 0.04, the preset curvature response threshold can be 0.2.

[0082] The second step is to determine the aforementioned biological anchor points and edge curvature points as each phenotypic feature node. In practice, the executing entity can determine the fruit stalk anchor point (i.e., the biological anchor point) as the fruit stalk phenotypic feature node, the center of the fruit navel depression area as the fruit navel phenotypic feature node, and each edge curvature point as a curvature phenotypic feature node.

[0083] For each biological anchor point or edge curvature point, the aforementioned execution entity can convert the corresponding rotated pixel coordinates into relative latitude, relative longitude, and spherical node number using the spherical projection operator. Then, the execution entity can read the phenotypic features of the corresponding spherical topological node from the phenotypic feature space and determine the read phenotypic features as the node network features of that phenotypic feature node. Thus, each phenotypic feature node can include node type, rotated pixel coordinates, spherical node number, relative latitude, relative longitude, node network features, and node confidence. Specifically, the node confidence of the fruit stalk phenotypic feature node can be the fruit stalk position response value. The node confidence of the fruit navel phenotypic feature node can be the average of the probabilities of each fruit navel region in the fruit navel depression region. The node confidence of the curvature phenotypic feature node can be the normalized curvature response value.

[0084] The third step is to bind the aforementioned geometric topological normal vectors to the corresponding phenotypic feature nodes to generate node orientation attributes. In practice, for each phenotypic feature node, firstly, the execution entity can read the spherical node number corresponding to the phenotypic feature node and search for spherical topological nodes with the same spherical node number in the phenotypic feature space. Then, the execution entity can determine the local surface normal vector corresponding to the spherical topological node and bind the read local surface normal vector to the current phenotypic feature node as the node orientation attribute of the aforementioned phenotypic feature node.

[0085] It should be noted that when the spherical topology node corresponding to the above-mentioned phenotypic feature node does not have a local surface normal vector, the above-mentioned execution entity can read the local surface normal vectors corresponding to each adjacent spherical topology node directly connected to the above-mentioned spherical topology node.

[0086] The aforementioned execution entity can set interpolation weights based on the surface distances from each adjacent spherical topology node to the current spherical topology node. For example, the smaller the surface distance, the larger the corresponding interpolation weight. The specific numerical settings for the weights will not be elaborated here. Subsequently, the local surface normal vectors of each adjacent spherical topology node are weighted and averaged, and the resulting direction vector is normalized to obtain the node direction attribute of the current phenotypic feature node. Thus, each phenotypic feature node corresponds to a node direction attribute, which represents the local surface orientation of the fitted spherical position where the phenotypic feature node is located.

[0087] The fourth step involves establishing graph-structured connections between the bound phenotypic feature nodes to construct a phenotypic feature topology network. In practice, the aforementioned execution entities can establish biological axial connectivity, contour curvature connectivity, and local surface connectivity, respectively.

[0088] For biological axial connections, the aforementioned execution entity can establish a connection edge representing the contour curvature connection relationship between the fruit pedicle phenotypic feature node and the fruit navel phenotypic feature node, which is used to represent the biological axial structural relationship of the apple along the direction from the fruit pedicle to the fruit navel.

[0089] For contour curvature connection relationships, the aforementioned execution entity can sort each curvature phenotype feature node according to the order of each edge curvature point in the closed contour node sequence, and establish connection edges representing contour curvature connection relationships between two adjacent curvature phenotype feature nodes.

[0090] For local surface connectivity, the aforementioned execution entity can determine the surface distance between any two phenotypic feature nodes on the fitted sphere and determine the directional similarity between the directional attributes of the two nodes. This directional similarity can be determined by the spatial dot product between the normal vectors of the two local surfaces. When the surface distance between two phenotypic feature nodes is less than or equal to a preset surface adjacency distance, and their directional similarity is greater than or equal to a preset directional similarity threshold (e.g., 0.8), the aforementioned execution entity can establish a connection edge representing the local surface connectivity between the two phenotypic feature nodes.

[0091] For each connection edge, the aforementioned execution entity can generate node connection weights using the surface distance, directional similarity, and node confidence between the two nodes. For example, the smaller the surface distance, the higher the directional similarity, and the higher the node confidence between the two nodes, the larger the generated node connection weights. Subsequently, the aforementioned execution entity can normalize the node connection weights corresponding to the same phenotypic feature node, so that the sum of the node connection weights corresponding to that phenotypic feature node is 1.

[0092] Furthermore, the aforementioned execution entity can set a dynamic multi-scale contextual phenotypic information propagation operator in the first feature extraction branch. This dynamic multi-scale contextual phenotypic information propagation operator can be implemented using a graph convolutional network or a graph attention network, and can respectively set small-scale, medium-scale, and large-scale surface neighborhoods. The small-scale surface neighborhood can include all neighboring nodes directly connected to the current phenotypic feature node. The medium-scale surface neighborhood can include all phenotypic feature nodes separated from the current phenotypic feature node by two connection levels. The large-scale surface neighborhood can include all phenotypic feature nodes that are the furthest reachesable along biological axis connections or contour curvature connections.

[0093] Subsequently, for each phenotypic feature node, the aforementioned execution entity can utilize the node connection weights corresponding to each scale (i.e., large, medium, and small scales) to perform weighted aggregation of the node network features and node orientation attributes of each phenotypic feature node in the corresponding neighborhood, obtaining the neighborhood aggregated features corresponding to each scale. Finally, the aforementioned execution entity can concatenate the neighborhood aggregated features corresponding to each scale with the node network features of the current phenotypic feature node, and generate updated node network features through graph convolutional layers or graph attention layers.

[0094] Fifth, based on the aforementioned phenotypic feature topology network and the fruit projection area corresponding to the aforementioned apple image tensor, the aforementioned phenotypic feature space is scaled to obtain phenotypic constraint parameters. In practice, firstly, the aforementioned execution entity can determine the number of pixels marked as fruit pixels in the binary mask corresponding to the determined target fruit region, thereby obtaining the fruit projection area corresponding to the current apple image tensor.

[0095] Then, the aforementioned executing entity can pre-set a standard fruit projection area. This standard fruit projection area can be determined by statistically analyzing the fruit projection areas corresponding to pre-collected standard apple image samples and using the median of each fruit projection area. Afterward, the executing entity can determine the area ratio between the current fruit projection area and the standard fruit projection area, and then apply the square root of this ratio to obtain an initial normalization scale adjustment coefficient.

[0096] Subsequently, the aforementioned execution entity can utilize a heuristic relaxation algorithm based on residual approximation (e.g., a combination of Lagrange relaxation and local search algorithms) to adjust the initial normalized scaling coefficients, thereby obtaining normalized scaling coefficients. Specifically, the execution entity can read the surface distances between the fruit stem phenotypic feature nodes and the fruit navel phenotypic feature nodes, as well as the surface distances between adjacent curvature phenotypic feature nodes, from the phenotypic feature topology network. The execution entity can then perform scaling conversion on each surface distance using the normalized scaling coefficients, and compare the converted surface distances with pre-set standard phenotypic node distances to obtain phenotypic distance residuals. The aforementioned standard phenotypic node distances can be obtained by statistically analyzing the phenotypic feature topology network corresponding to standard apple image samples and determining the median surface distances between nodes of the same type.

[0097] When the phenotypic distance residual exceeds a preset residual threshold, the aforementioned execution entity can increase or decrease the normalization scale adjustment coefficient according to a preset adjustment step size, and then recalculate the phenotypic distance residual. The preset adjustment step size controls the magnitude of change in each residual correction or parameter update and can be determined by the parameter value range. Specifically, the allowable range of parameter variation can be divided into multiple adjustment intervals, and the single adjustment ratio can be set to 1% to 10% of the parameter range. For example, for a calibration parameter with a value range of 0 to 1, a preset adjustment step size of 0.05 can be set, meaning the parameter is increased or decreased by 0.05 each time it is adjusted. When the adjusted phenotypic distance residual is less than the original phenotypic distance residual, the normalization scale adjustment coefficient is modified along the current adjustment direction. When the adjusted phenotypic distance residual is greater than the original phenotypic distance residual, the adjustment direction is changed, and the adjustment step size is decreased. This adjustment process can continue until the phenotypic distance residual is less than or equal to the preset residual threshold, or the number of adjustments reaches a preset maximum number of adjustments (e.g., 15 times). The normalization scaling factor corresponding to the last processing step can be determined as the target normalization scaling factor.

[0098] It should be noted that the aforementioned execution entity can determine the preset residual threshold by calculating the color residual distribution using multiple pre-collected healthy apple samples and diseased apple samples. Specifically, the residual between the color center value corresponding to each pixel position in a healthy apple sample and the standard chromaticity of a healthy apple can be calculated separately. The mean and standard deviation of the residuals corresponding to all healthy samples can be statistically analyzed and set as the mean of the residuals of healthy samples plus three times the standard deviation. When the abnormal residual corresponding to a local area is greater than or equal to this threshold, the area is identified as an abnormal area.

[0099] Finally, the aforementioned implementing entity can use the target normalized scale adjustment coefficient to adjust the image feature sampling range corresponding to each spherical topological node in the phenotypic feature space, and simultaneously adjust the surface distance range corresponding to the small-scale, medium-scale, and large-scale surface neighborhoods. The target normalized scale adjustment coefficient, the node information of each phenotypic feature node, the node orientation attribute, the node connection relationship, the node connection weight, and the adjusted range of each scale surface neighborhood are jointly determined as phenotypic constraint parameters.

[0100] Step 104: Based on the candidate high-frequency feature map of lesions, perform illumination component separation and gradient calibration on the apple image tensor to generate pixel gradient features.

[0101] In some embodiments, the execution entity can perform illumination component separation and gradient calibration on the apple image tensor based on the aforementioned candidate high-frequency feature map of lesions to generate pixel gradient features. In practice, the execution entity can perform homomorphic filtering on the apple image tensor, identifying the low-frequency brightness component as the illumination component and the high-frequency component after filtering out the low-frequency brightness as the peel reflection component. Subsequently, according to the image coordinates in the candidate high-frequency feature map of lesions, the corresponding local region is extracted from the peel reflection component, and the horizontal and vertical gradients are calculated using the Schar operator. Finally, the local gradient is normalized using the average gradient of the normal peel surrounding the candidate region to obtain the pixel gradient features.

[0102] In some optional implementations of certain embodiments, the execution entity may perform illumination component separation and gradient calibration on the apple image tensor based on the aforementioned candidate high-frequency feature map of lesions to generate pixel gradient features:

[0103] The first step is to perform illumination component decomposition on the apple image tensor to generate ambient illumination field components and intrinsic reflectivity components of the object surface. In practice, firstly, the execution entity can normalize the pixel values ​​of each color channel in the apple image tensor to a value range of 0 to 1, and then input the normalized apple image tensor into the illumination component separation unit set at the front end of the second feature extraction branch.

[0104] The aforementioned illumination component separation unit can employ a pre-trained image decomposition network consisting of an encoder and a decoder. For example, a convolutional neural network model composed of convolutional layers, downsampling layers, upsampling layers, and skip connection layers can be used.

[0105] The aforementioned illumination component separation unit can be configured with an illumination field output and an intrinsic reflectance output. The illumination field output extracts slowly varying brightness information from the apple image through convolution processing and outputs a single-channel ambient illumination field component. Each location in the ambient illumination field component records the normalized illumination intensity corresponding to that image location. The intrinsic reflectance output is used to eliminate the influence of the ambient illumination field component on the intensity of the red, green, and blue channels pixel by pixel, outputting the intrinsic reflectance component of the object surface. During the training process of the illumination component separation unit, the ambient illumination field component and the intrinsic reflectance component of the object surface can be recombined, and the difference between the recombined image and the input image can be compared to iteratively update the illumination component separation unit.

[0106] The second step involves determining the illumination-affected region in the apple image tensor based on the ambient light field components. In practice, the execution entity can separately determine the highlight region and the shadow boundary region. For the highlight region, the execution entity can read the ambient light field components corresponding to each pixel within the target fruit region and arrange them in ascending order of light intensity. Then, the light intensity value corresponding to the preset highlight quantile is determined as the highlight region threshold. For example, the light intensity value corresponding to the 90th percentile can be determined as the highlight region threshold. The execution entity can mark each pixel whose ambient light field component is greater than or equal to the highlight region threshold as a highlight candidate pixel, and then use eight-neighbor connected region analysis to form a highlight region from the interconnected highlight candidate pixels, while isolated regions with an area smaller than the preset highlight area threshold can be removed as noise.

[0107] For the shadow boundary region, the aforementioned execution entity can use the Sobel operator or the Scharr operator to determine the amount of illumination change of the ambient illumination field component in the horizontal and vertical directions of the image, respectively, and determine the illumination field gradient magnitude from the amount of illumination change in the two directions.

[0108] Then, the aforementioned execution entity can mark pixels with illumination field gradient magnitudes greater than or equal to a preset shadow boundary threshold as shadow boundary pixels, and perform morphological dilation processing on each shadow boundary pixel to make the resulting area cover the transition range of illumination from light to dark. Finally, the aforementioned execution entity can jointly determine the highlight area and shadow boundary area as the illumination-affected area, and record highlight area markers or shadow boundary area markers for each pixel in the illumination-affected area.

[0109] The third step involves extracting the intrinsic grayscale gradient from the candidate high-frequency feature map of the lesion based on the intrinsic reflectance components of the object's surface. In practice, the executing entity can read the original image coordinates retained in each high-frequency feature node and use these original image coordinates to determine the corresponding position in the intrinsic reflectance components of the object's surface.

[0110] It should be noted that for high-frequency feature nodes composed of a single micro-gradient abrupt change point, the aforementioned execution entity can directly read the intrinsic reflectance corresponding to the aforementioned image coordinates. For high-frequency feature nodes composed of local color aberration patches or edge texture break regions, the aforementioned execution entity can read the pixel coordinates of each pixel in the corresponding region and the intrinsic reflectance corresponding to each pixel coordinate.

[0111] Then, the aforementioned execution entity can weight the three intrinsic reflectivity channels according to the brightness contribution ratios of the red, green, and blue channels to obtain the intrinsic grayscale values. Then, the changes in the intrinsic grayscale values ​​in the horizontal and vertical directions of the image can be determined using the Sobel or Scharr operator, yielding the intrinsic horizontal gradient and intrinsic vertical gradient, respectively.

[0112] Finally, the intrinsic gradient magnitude and intrinsic gradient direction can be determined by the intrinsic horizontal gradient and intrinsic vertical gradient. The intrinsic gray-level gradient corresponding to each high-frequency feature node is composed of the intrinsic horizontal gradient, intrinsic vertical gradient, intrinsic gradient magnitude, and intrinsic gradient direction.

[0113] The fourth step involves performing gradient calibration on the intrinsic grayscale gradient in the illumination-affected region based on the chromaticity calibration factors corresponding to the ambient light field components, thereby generating pixel gradient features. In practice, the execution entity can pre-construct a chromaticity calibration factor table. Specifically, the normalized ambient light field components can be divided into multiple continuous illumination intensity intervals, and the intrinsic gradient amplitude of the highlight region in each illumination intensity interval can be statistically analyzed using healthy apple images acquired under different illumination conditions. For each illumination intensity interval, the execution entity can calculate the ratio between the median intrinsic gradient amplitude of the highlight region and the median intrinsic gradient amplitude of the standard diffuse region within that interval, and determine this ratio as the initial chromaticity calibration factor for the corresponding illumination intensity interval.

[0114] Then, the aforementioned execution entity can check each initial chromaticity calibration factor from low to high illumination intensity. When the initial chromaticity calibration factor of the later illumination intensity range is less than or equal to the chromaticity calibration factor of the previous illumination intensity range, the chromaticity calibration factor of the later illumination intensity range can be adjusted to the sum of the previous chromaticity calibration factor and the preset step value. Thus, a chromaticity calibration factor that increases non-linearly with the increase of the ambient light field component can be formed. As an example, the ambient light field component can be divided into low illumination range, medium illumination range, and high illumination range, and set to chromaticity calibration factors of 1, 1.2, and 1.5 respectively.

[0115] For high-frequency feature nodes located within highlight areas, the execution entity can read the ambient light field component corresponding to the high-frequency feature node, determine its corresponding illumination intensity range, and then divide the intrinsic lateral gradient and intrinsic longitudinal gradient by the corresponding chromaticity calibration factor, respectively. For high-frequency feature nodes located within shadow boundary areas, the execution entity can read the intrinsic reflectance components of the object surface on both sides of the shadow boundary along the illumination field gradient direction. When the difference in intrinsic reflectance on both sides of the shadow boundary is less than a preset reflectance difference threshold, it indicates that the grayscale abrupt change at that location is mainly caused by illumination changes. The execution entity can replace the intrinsic grayscale gradient at that location with the average intrinsic grayscale gradient of the non-boundary pixels on both sides of the shadow boundary. When the difference in intrinsic reflectance on both sides of the shadow boundary is greater than or equal to a preset reflectance difference threshold, it indicates that there may be a change in true color or texture breakage at that location. The execution entity can retain the chromaticity-calibrated intrinsic grayscale gradient at that location. For high-frequency feature nodes outside the illumination-affected area, the execution entity can directly retain the corresponding intrinsic grayscale gradient.

[0116] Finally, the aforementioned execution entity can recalculate the gradient magnitude and gradient direction corresponding to each high-frequency feature node, and arrange the calibrated horizontal gradient, vertical gradient, gradient magnitude and gradient direction according to the image coordinates of each high-frequency feature node, thereby obtaining the pixel gradient features.

[0117] Step 105: Based on the phenotypic constraint parameters, geometric topological normal vector, specular core coordinates, and lesion candidate high-frequency feature map, the pixel gradient features are calibrated and fused to generate calibration feature information.

[0118] In some embodiments, the execution entity can perform calibration and fusion of the pixel gradient features based on the phenotypic constraint parameters, the geometric topological normal vector, the specular core coordinates, and the lesion candidate high-frequency feature map to generate calibration feature information. In practice, firstly, the execution entity can map the specular core coordinates to the phenotypic feature space and calculate the surface distance between each lesion candidate high-frequency feature and the specular core position to obtain the specular influence coefficient. Then, it calculates the angle between the geometric topological normal vector and the specular core direction to obtain the surface orientation calibration coefficient, and adjusts the calibration range using the phenotypic constraint parameters. The execution entity can correct the pixel gradient features using the specular influence coefficient and the surface orientation calibration coefficient, and then weightedly concatenate the corrected pixel gradient features with the lesion candidate high-frequency features to generate calibration feature information. The weight settings can be configured according to the actual application scenario and will not be elaborated here.

[0119] In some optional implementations of certain embodiments, the execution entity may perform calibration and fusion of the pixel gradient features based on the phenotypic constraint parameters, the geometric topological normal vector, the specular core coordinates, and the lesion candidate high-frequency feature map to generate calibration feature information through the following steps:

[0120] The first step involves attaching the candidate high-frequency feature maps of lesions and the pixel gradient features to the corresponding surface metric coordinate nodes in the phenotypic feature space, according to the aforementioned phenotypic constraint parameters. In practice, the executing entity first reads the image coordinates corresponding to each high-frequency feature node in the candidate high-frequency feature map of lesions, and the image coordinates corresponding to each gradient feature in the pixel gradient features. Then, the executing entity uses the aforementioned spherical projection operator to convert each image coordinate into relative latitude, relative longitude, and spherical node numbers, thereby determining the surface metric coordinate nodes corresponding to each high-frequency feature node and each pixel gradient feature. Next, the executing entity adjusts the image feature sampling range corresponding to each surface metric coordinate node using the target normalization scale adjustment coefficient included in the aforementioned phenotypic constraint parameters. Within the adjusted sampling range, weighted averages are performed on the color response value, texture breakage response value, gradient abrupt change response value, original color value, and calibrated horizontal gradient, vertical gradient, gradient magnitude, and gradient direction. Finally, the executing entity writes the obtained high-frequency pixel features and pixel gradient features into the corresponding surface metric coordinate nodes to obtain the corresponding phenotypic feature space. Thus, the high-frequency feature maps of lesion candidates and pixel gradient features output by the two feature extraction branches are unified into the same surface metric coordinate system.

[0121] The second step involves generating the virtual environment intrinsic light source direction vector based on the relative positions of the aforementioned specular core coordinates within the aforementioned phenotypic feature space. In practice, firstly, the executing entity can determine the relative latitude, relative longitude, spherical node number, and corresponding specular core node corresponding to the aforementioned specular core coordinates in the phenotypic feature space. Then, the executing entity can determine the direction from the fitted spherical center to the specular core node and normalize the determined direction to obtain the local spherical normal direction corresponding to the specular core node. Subsequently, the executing entity can also determine the optical axis direction of the monocular image acquisition device as the camera observation direction. This camera observation direction can be pre-written into the executing entity after the image acquisition device has been installed and calibrated. Afterward, the executing entity can reverse-map the camera observation direction using the symmetric relationship between the incident direction and the observation direction during specular reflection with respect to the local spherical normal direction to obtain the virtual environment intrinsic light source direction vector.

[0122] It should be noted that the aforementioned virtual environment intrinsic light source direction vector is used to represent the main illumination direction in the current image potential space, and is not the actual physical light source position measured by the light sensor.

[0123] The third step involves determining the local surface normal vector corresponding to each surface metric coordinate node based on the aforementioned geometric topological normal vectors. In practice, for each surface metric coordinate node with local high-frequency pixel features, the execution entity can read the spherical node number corresponding to that node. Then, the execution entity can search for local surface normal vectors with the same spherical node number in the phenotypic feature space and determine the found local surface normal vector as the local surface normal vector corresponding to the current surface metric coordinate node. When multiple local unit direction vectors are attached to the same surface metric coordinate node, the execution entity can average each direction component separately, and then normalize the averaged direction vectors to obtain a local surface normal vector corresponding to the current surface metric coordinate node.

[0124] The fourth step involves determining the local curvature variation corresponding to each surface metric coordinate node based on the pre-defined non-destructive phenotypic manifold curvature baseline. In practice, the aforementioned execution entity can pre-construct the non-destructive phenotypic manifold curvature baseline using healthy apple images. Specifically, referring to the implementation methods in steps 101 and 102, each healthy apple image is mapped to the same spherical topology mesh. For each spherical topology node, the degree of local surface curvature between that node and its adjacent spherical topology nodes is determined, and the degree of local surface curvature of the same spherical topology nodes in each healthy apple sample is averaged to obtain the reference curvature corresponding to that spherical topology node. Arranging the reference curvatures according to the spherical node numbers yields the non-destructive phenotypic manifold curvature baseline.

[0125] For each surface metric coordinate node, the aforementioned execution entity can determine the current local curvature by using the relative surface coordinates of the surface metric coordinate node and its adjacent spherical topology nodes. Then, it reads the reference curvature corresponding to the same spherical node number from the non-destructive phenotypic manifold curvature baseline. The aforementioned execution entity can determine the absolute difference between the current local curvature and the reference curvature, and normalize this difference using the reference curvature to obtain the local curvature variability.

[0126] The fifth step involves generating noise-resistant manifold interferometric values ​​based on the real-time pixel gradient change rate, the fitted spherical curvature derivative distribution, the aforementioned virtual environment intrinsic light source direction vector, and the normal vectors of each local surface. In practice, for surface metric coordinate nodes where the local curvature variability exceeds the curvature abrupt change threshold, the aforementioned execution entity can first determine the real-time pixel gradient change rate. Specifically, the execution entity can determine the gradient magnitude corresponding to the surface metric coordinate node, as well as the gradient magnitudes corresponding to each adjacent spherical topology node directly connected to the surface metric coordinate node. Furthermore, the real-time pixel gradient change rate can be obtained by dividing the difference between the gradient magnitude of the surface metric coordinate node and the average gradient magnitude of its neighboring nodes by the average surface distance between the current node and its neighboring nodes.

[0127] It should be noted that the aforementioned curvature abrupt change threshold is used to determine whether the curvature change between adjacent spherical topological nodes exceeds the normal range of apple skin variation. In practice, a set of normal curvature changes can be constructed using healthy apple samples, and the curvature differences between adjacent nodes can be statistically analyzed. The mean of the normal curvature differences plus three times the standard deviation is determined as the curvature abrupt change threshold. For example, if the mean curvature change of adjacent nodes on the surface of a healthy apple is 0.05 and the standard deviation is 0.02, then the curvature abrupt change threshold can be 0.11. When the curvature difference between two adjacent nodes is greater than 0.11, it is considered that there is an abnormal curvature change in that area.

[0128] Then, the aforementioned execution entity can read the pre-stored distribution of the fitted spherical curvature derivative. This distribution can be obtained by determining the local curvature differences between adjacent spherical topological nodes and normalizing them to the corresponding surface distances. Next, the execution entity can determine the spatial dot product between the virtual environment's intrinsic light source direction vector and the current local surface normal vector, and subtract this spatial dot product to obtain the interference margin. This interference margin is then multiplied by the shadow area compensation adjustment coefficient to obtain the compensation term. For example, the shadow area compensation adjustment coefficient can be set to 0.08. Finally, the execution entity can use the real-time pixel gradient change rate as the numerator and the sum of the fitted spherical curvature derivative and the compensation term as the denominator to obtain the noise-resistant manifold interference value. This compensation term can prevent computational anomalies when the local curvature derivative of the fitted spherical surface approaches zero.

[0129] Step 6: Based on the aforementioned noise-resistant manifold interferometry values, determine the feature flow weights corresponding to each local high-frequency pixel feature in the candidate high-frequency feature map of the lesion. In practice, the aforementioned execution entity can pre-set a diffuse reflection consistency interval. For example, the diffuse reflection consistency interval can be set to 0.85 to 1.15. When the noise-resistant manifold interferometry value corresponding to a certain local high-frequency pixel feature is within the aforementioned diffuse reflection consistency interval, it indicates that the change in the local high-frequency pixel can be explained by the relationship between the current illumination direction and the orientation of the local curved surface, and the feature flow weight of the local high-frequency pixel feature can be set to 0. When the noise-resistant manifold interferometry value deviates from the diffuse reflection consistency interval, it indicates that the change in the local high-frequency pixel cannot be fully explained by the illumination relationship, and the feature flow weight of the local high-frequency pixel feature can be set to 1. For local high-frequency pixel features whose local curvature variation does not exceed the curvature abrupt change threshold, their feature flow weight can be kept at 1.

[0130] Therefore, local high-frequency pixel features with a feature flow weight of 0 are regarded as shadows or highlights, while local high-frequency pixel features with a feature flow weight of 1 are used for abnormal residual detection.

[0131] Step 7: Based on the aforementioned feature flow weights, perform closed-loop calibration fusion on the aforementioned candidate high-frequency feature maps of lesions and the aforementioned pixel gradient features to generate calibration feature information. In practice, for each surface metric coordinate node, the aforementioned execution entity can multiply the high-frequency pixel features and pixel gradient features corresponding to the surface metric coordinate node with the feature flow weights to obtain calibrated high-frequency pixel features and calibrated pixel gradient features. Then, the aforementioned execution entity can concatenate the calibrated high-frequency pixel features, calibrated pixel gradient features, and the phenotypic features of the aforementioned surface metric coordinate nodes in the phenotypic feature space, and convert the concatenated features into node calibration features of a preset dimension through a fully connected layer. Finally, the aforementioned execution entity can arrange the node calibration features according to the spherical node number and simultaneously record the feature flow weights, original image coordinates, and spherical node connection relationships corresponding to each spherical node, thereby generating calibration feature information.

[0132] Step 106: Perform abnormal residual detection on the calibration feature information to generate pest and disease detection results.

[0133] In some embodiments, the execution entity can perform abnormal residual detection on the calibration feature information to generate pest and disease detection results. These results include abnormal residual information, pest and disease region information, pest and disease type labels, and location coordinate bounding boxes. In practice, firstly, the execution entity can pre-establish a healthy feature sample set using healthy apple images. For each surface position in the calibration feature information, the distance between the corresponding calibration feature and the nearest feature in the healthy feature sample set is calculated, and this distance is determined as the abnormal residual. Then, the execution entity can group adjacent positions where the abnormal residual is greater than a preset residual threshold into a pest and disease region, and input the region features into a support vector machine or nearest neighbor classifier to obtain the pest and disease type label. Finally, the original image coordinates corresponding to the pest and disease region are read, the location coordinate bounding boxes are determined, and the abnormal residual, pest and disease region, pest and disease type label, and location coordinate bounding boxes are combined to form the pest and disease detection results.

[0134] In some optional implementations of certain embodiments, the aforementioned execution entity may perform abnormal residual detection on the aforementioned calibration feature information through the following steps to generate pest and disease detection results:

[0135] The first step is to determine the local pixel color center value corresponding to a local area of ​​the apple based on the aforementioned calibration feature information. In practice, the execution entity can first filter the surface metric coordinate nodes whose feature flow weights are not zero from the calibration feature information.

[0136] Then, the aforementioned execution entity can perform connectivity analysis on spatially adjacent surface metric coordinate nodes according to the spherical node connection relationship, and determine each connected region as a local region of the apple. For each local region of the apple, the aforementioned execution entity can read the original color value or the intrinsic reflectance color value of the object surface corresponding to each surface metric coordinate node within the region. Then, the aforementioned execution entity can determine the color weight through the feature flow weight and node confidence corresponding to each surface metric coordinate node, and then perform a weighted average of the red channel value, green channel value, and blue channel value respectively to obtain the local pixel color center value corresponding to the current local region of the apple.

[0137] The second step involves generating a baseline chromaticity envelope based on a pre-defined standard chromaticity for healthy apples. In practice, the executing entity can pre-acquire several images of healthy apples without blemishes and, referring to the specific implementation methods in steps 101 to 104, convert each healthy apple image to the same spherical topology mesh. Then, for each spherical topology node in the mesh, the executing entity can determine the color value corresponding to that node for each healthy apple image and determine the normal value range for each color channel. For example, the 5th percentile value of the color value of each healthy apple sample can be determined as the lower limit, and the 95th percentile value as the upper limit. The executing entity can then arrange the lower and upper limits of the colors corresponding to each spherical topology node according to the spherical node number to obtain the baseline chromaticity envelope.

[0138] Therefore, the reference chromaticity envelope can represent the normal color differences that are allowed to exist in the apple stem area, waist area, and navel area under healthy conditions, instead of using a fixed color value to constrain the entire apple surface.

[0139] The third step involves generating abnormal residual information based on the deviation between the local pixel color center value and the reference chromaticity envelope. In practice, for each local area of ​​the apple, the execution entity can read the corresponding lower and upper color limits from the reference chromaticity envelope according to the spherical node numbers included in that area. When the local pixel color center value is between the corresponding lower and upper color limits, the color deviation can be set to zero. When the local pixel color center value is lower than the lower color limit or higher than the upper color limit, the execution entity can determine the color distance between the local pixel color center value and the nearest envelope boundary, and define this color distance as the color deviation. Subsequently, the execution entity can compose abnormal residual information by combining the spherical node numbers corresponding to the local area of ​​the apple, the color deviation, the color deviation direction, the texture breakage response value, and the gradient abrupt change response value.

[0140] The fourth step involves generating a multidimensional residual tensor corresponding to the spherical topological mesh based on the aforementioned anomalous residual information. In practice, the execution entity can pre-set a multi-channel tensor corresponding to the spherical topological mesh, with each spatial location in the multi-channel tensor corresponding to a spherical topological node. For each local apple region in the anomalous residual information, the execution entity can write the color deviation, texture breakage response value, gradient abruptness response value, and calibrated gradient magnitude into the corresponding tensor channels according to the spherical node number. For spherical topological nodes where no anomalous residual information has been generated, the corresponding tensor positions can be filled with zeros. Finally, the execution entity can combine the spatial locations of the spherical topological mesh and each residual feature channel to form a multidimensional residual tensor.

[0141] The fifth step involves using a residual convergence network to perform residual convergence processing on the aforementioned multidimensional residual tensor to generate pest and disease area information. In practice, the execution entity can input the multidimensional residual tensor into a pre-trained residual convergence network. This residual convergence network can employ a convolutional neural network comprising multiple residual convolutional units and a spatial attention layer. First, the residual convolutional units extract features layer by layer from the multidimensional residual tensor to obtain the lesion response values ​​corresponding to each spherical topological node. Then, the residual convergence network performs connected component analysis on interconnected high lesion response nodes and calculates the cumulative response value of each residual channel within each connected component. When the cumulative response value exceeds a preset first-level safety threshold (e.g., 0.15), the spatial attention layer increases the feature weights corresponding to that connected component.

[0142] After processing by the residual convolutional unit, the residual convergence network can retain connected regions whose cumulative response value is greater than a preset secondary danger threshold (e.g., 0.35) and whose area accounts for a greater than a preset pixel percentage threshold of the target fruit area. For example, the preset pixel percentage threshold can be set to 0.03%. Finally, the aforementioned execution entity can use the spherical node number, original image pixel position, area, and boundary of the retained connected regions to jointly determine the pest and disease area information.

[0143] The sixth step involves classifying the pest and disease area information using the classification layer within the residual convergence network to generate pest and disease type labels and location coordinate bounding boxes. In practice, the execution entity can input the multidimensional residual tensor features corresponding to each pest and disease area into the classification layer of the residual convergence network. This classification layer can sequentially include a global average pooling layer, a fully connected layer, and a normalized exponential classification layer. The global average pooling layer aggregates the residual features corresponding to each spherical topological node within the pest and disease area into a region feature vector. The fully connected layer and the normalized exponential classification layer output the classification probability corresponding to each preset pest and disease type. The execution entity can determine the pest and disease type with the highest classification probability as the pest and disease type label corresponding to that pest and disease area. Then, the execution entity can read the original image pixel coordinates retained by each spherical topological node within the pest and disease area and determine the minimum horizontal pixel coordinate, maximum horizontal pixel coordinate, minimum vertical pixel coordinate, and maximum vertical pixel coordinate, respectively. Finally, the aforementioned implementing entity can combine the four coordinates to form a location coordinate bounding box, and jointly determine the pest and disease area information, pest and disease type label, and location coordinate bounding box as the pest and disease detection result.

[0144] Further reference Figure 3 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a visual detection device for pests and diseases based on apple phenotypic standards. These device embodiments are similar to... Figure 1Corresponding to the method embodiments shown, this visual detection device for pests and diseases based on apple phenotypic standards can be specifically applied to various electronic devices.

[0145] like Figure 3 As shown, a visual detection device 300 for pests and diseases based on apple phenotypic standards in some embodiments includes: a bi-branch feature extraction unit 301, a surface metric mapping unit 302, a scale constraint unit 303, an illumination component separation and gradient calibration unit 304, a calibration fusion unit 305, and a generation unit 306. The bi-branch feature extraction unit 301 is configured to perform bi-branch feature extraction on the acquired apple image tensor to generate a geometric topological normal vector, specular core coordinates, a global spatial topological map, and a high-frequency feature map of lesion candidates. The surface metric mapping unit 302 is configured to perform surface metric mapping on the two-dimensional image space corresponding to the apple image tensor based on the global spatial topological map, the high-frequency feature map of lesion candidates, the geometric topological normal vector, and the specular core coordinates to obtain a phenotypic feature space. The scale constraint unit 303 is configured to apply scale constraints to the phenotypic feature space based on the geometric topological normal vector and the biological anchor points in the global spatial topological map to generate a phenotypic approximation. The illumination component separation and gradient calibration unit 304 is configured to perform illumination component separation and gradient calibration on the apple image tensor based on the aforementioned candidate high-frequency feature map of lesions to generate pixel gradient features; the calibration fusion unit 305 is configured to perform calibration fusion on the aforementioned pixel gradient features based on the aforementioned phenotypic constraint parameters, the aforementioned geometric topological normal vector, the aforementioned specular core coordinates, and the aforementioned candidate high-frequency feature map of lesions to generate calibration feature information; the generation unit 306 is configured to perform abnormal residual detection on the aforementioned calibration feature information to generate pest and disease detection results, wherein the aforementioned pest and disease detection results include abnormal residual information, pest and disease area information, pest and disease type labels, and location coordinate bounding boxes.

[0146] It is understandable that the units described in the apple phenotypic standard-based visual inspection device 300 for pests and diseases are similar to the reference units. Figure 1 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the pest and disease visual inspection device 300 based on apple phenotypic standards and the units contained therein, and will not be repeated here.

[0147] The following is for reference. Figure 4 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 400 suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0148] like Figure 4 As shown, the electronic device 400 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory 402 or a program loaded from a storage device 408 into a random access memory 403. The random access memory 403 also stores various programs and data required for the operation of the electronic device 400. The processing unit 401, the read-only memory 402, and the random access memory 403 are interconnected via a bus 404. An input / output interface 405 is also connected to the bus 404.

[0149] Typically, the following devices can be connected to the input / output interface 405: input devices 406 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 408 including, for example, magnetic tape, hard disk, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 4 Each box shown can represent a device or multiple devices as needed.

[0150] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a read-only memory 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of some embodiments of this disclosure.

[0151] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0152] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0153] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: perform bi-branch feature extraction on the acquired apple image tensor to generate geometric topological normal vectors, specular core coordinates, a global spatial topological map, and a lesion candidate high-frequency feature map; perform surface metric mapping on the two-dimensional image space corresponding to the aforementioned apple image tensor based on the aforementioned global spatial topological map, the aforementioned lesion candidate high-frequency feature map, the aforementioned geometric topological normal vectors, and the aforementioned specular core coordinates to obtain a phenotypic feature space; and perform bioanchoring based on the aforementioned geometric topological normal vectors and the aforementioned global spatial topological map. The above phenotypic feature space is scale-constrained to generate phenotypic constraint parameters. Based on the above candidate high-frequency feature map of lesions, the apple image tensor is subjected to illumination component separation and gradient calibration to generate pixel gradient features. Based on the above phenotypic constraint parameters, the above geometric topological normal vector, the above specular core coordinates, and the above candidate high-frequency feature map of lesions, the above pixel gradient features are calibrated and fused to generate calibration feature information. The above calibration feature information is subjected to abnormal residual detection to generate pest and disease detection results, wherein the above pest and disease detection results include abnormal residual information, pest and disease area information, pest and disease type labels, and location coordinate bounding boxes.

[0154] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and Python, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0155] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0156] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0157] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A visual detection method for pests and diseases based on apple phenotypic standards, characterized in that, include: A bi-branch feature extraction was performed on the acquired apple image tensor to generate geometric topological normal vectors, specular core coordinates, global spatial topological map, and high-frequency feature map of lesion candidates; Based on the global spatial topology map, the candidate high-frequency feature map of lesions, the geometric topology normal vector, and the specular core coordinates, a surface metric mapping is performed on the two-dimensional image space corresponding to the apple image tensor to obtain the phenotypic feature space. Based on the geometric topological normal vector and the biological anchor points in the global spatial topological map, scale constraints are applied to the phenotypic feature space to generate phenotypic constraint parameters. Based on the candidate high-frequency feature map of the lesions, the apple image tensor is subjected to illumination component separation and gradient calibration to generate pixel gradient features; The pixel gradient features are calibrated and fused based on the phenotypic constraint parameters, the geometric topology normal vector, the specular core coordinates, and the lesion candidate high-frequency feature map to generate calibration feature information; Anomaly residual detection is performed on the calibration feature information to generate pest and disease detection results, wherein the pest and disease detection results include anomaly residual information, pest and disease area information, pest and disease type labels, and location coordinate bounding boxes.

2. The method according to claim 1, characterized in that, The step of performing bi-branch feature extraction on the acquired apple image tensor to generate geometric topological normal vectors, specular core coordinates, global spatial topological map, and lesion candidate high-frequency feature map includes: Through the first feature extraction branch, multi-scale phenotypic structure features are extracted from the apple image tensor to generate a global spatial topology map; Through the second feature extraction branch, high-resolution pixel features are extracted from the apple image tensor to generate a candidate high-frequency feature map of lesions; Local normal vectors are extracted from the global spatial topology map to generate geometric topology normal vectors; High-frequency coordinates are extracted from the candidate high-frequency feature maps of the lesions to generate high-frequency core coordinates.

3. The method according to claim 1, characterized in that, The step of applying scale constraints to the phenotypic feature space based on the geometric topological normal vector and the biological anchor points in the global spatial topological map to generate phenotypic constraint parameters includes: Curvature features are extracted from the overall outline edges of the fruit included in the global spatial topology map to generate edge curvature points; The biological anchor points and the edge curvature points are determined as each phenotypic feature node; The geometric topological normal vector is bound to the corresponding phenotypic feature node to generate the node orientation attribute; Graph structure associations are performed on the bound phenotypic feature nodes to construct a phenotypic feature topology network; Based on the phenotypic feature topology network and the fruit projection area corresponding to the apple image tensor, the phenotypic feature space is scaled to obtain phenotypic constraint parameters.

4. The method according to claim 1, characterized in that, The step of performing illumination component separation and gradient calibration on the apple image tensor based on the candidate high-frequency feature map of the lesions to generate pixel gradient features includes: The apple image tensor is decomposed into illumination components to generate ambient illumination field components and intrinsic reflectivity components of the object surface. Based on the ambient light field components, determine the region of illumination influence in the apple image tensor; Based on the intrinsic reflectance components of the object surface, the intrinsic gray-level gradient is extracted from the candidate high-frequency feature map of the lesion. Based on the chromaticity calibration factor corresponding to the ambient light field component, the intrinsic grayscale gradient in the light-affected region is calibrated to generate pixel gradient features.

5. The method according to claim 1, characterized in that, The step of calibrating and fusing the pixel gradient features based on the phenotypic constraint parameters, the geometric topological normal vector, the specular core coordinates, and the candidate high-frequency feature map of the lesion to generate calibration feature information includes: According to the phenotypic constraint parameters, the candidate high-frequency feature map of the lesion and the pixel gradient features are attached to the corresponding surface metric coordinate nodes in the phenotypic feature space; Based on the relative position of the specular core coordinates in the phenotypic feature space, a virtual environment intrinsic light source direction vector is generated; Based on the geometric topology normal vector, determine the local surface normal vector corresponding to each surface metric coordinate node; Based on the preset non-destructive phenotypic manifold curvature baseline, determine the local curvature variation corresponding to each surface metric coordinate node; Based on the real-time pixel gradient change rate, the fitted spherical curvature derivative distribution, the virtual environment intrinsic light source direction vector, and the normal vectors of each local surface, noise-resistant manifold interference values ​​are generated. Based on the noise-resistant manifold interferometry value, determine the feature flow weights corresponding to each local high-frequency pixel feature in the candidate high-frequency feature map of the lesion; Based on the feature flow weights, the candidate high-frequency feature map of the lesion and the pixel gradient features are fused using closed-loop calibration to generate calibration feature information.

6. The method according to claim 1, characterized in that, The step of performing abnormal residual detection on the calibration feature information to generate pest and disease detection results includes: Based on the calibration feature information, determine the local pixel color center value corresponding to a local area of ​​the apple; Generate a baseline chromaticity envelope based on the pre-defined healthy apple standard chromaticity; Based on the deviation between the local pixel color center value and the reference chromaticity envelope, abnormal residual information is generated; Based on the abnormal residual information, a multidimensional residual tensor corresponding to the spherical topological mesh is generated; The multidimensional residual tensor is subjected to residual convergence processing through a residual convergence network to generate information on pest and disease areas. The classification layer included in the residual convergence network is used to classify the information of the pest and disease area to generate pest and disease type labels and location coordinate bounding boxes.

7. A visual detection device for pests and diseases based on apple phenotypic standards, characterized in that, include: The bi-branch feature extraction unit is configured to perform bi-branch feature extraction on the acquired apple image tensor to generate geometric topological normal vectors, specular core coordinates, global spatial topological map, and lesion candidate high-frequency feature map. The surface metric mapping unit is configured to perform surface metric mapping on the two-dimensional image space corresponding to the apple image tensor based on the global spatial topology map, the lesion candidate high-frequency feature map, the geometric topology normal vector, and the specular core coordinates to obtain the phenotypic feature space. The scale constraint unit is configured to apply scale constraints to the phenotypic feature space based on the geometric topological normal vector and the biological anchor points in the global spatial topological map to generate phenotypic constraint parameters. The illumination component separation and gradient calibration unit is configured to perform illumination component separation and gradient calibration on the apple image tensor based on the candidate high-frequency feature map of the lesion to generate pixel gradient features. The calibration fusion unit is configured to perform calibration fusion on the pixel gradient features based on the phenotypic constraint parameters, the geometric topology normal vector, the specular core coordinates, and the lesion candidate high-frequency feature map to generate calibration feature information; The generation unit is configured to perform abnormal residual detection on the calibration feature information to generate pest and disease detection results, wherein the pest and disease detection results include abnormal residual information, pest and disease area information, pest and disease type labels, and location coordinate bounding boxes.

8. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images