A graph structure deep learning target spectrum detection network applied to seed identification

Abstract

The application relates to the fields of computer vision, deep learning, spectral image processing and agricultural intelligent detection technology, and discloses a graph structure deep learning target spectral detection network applied to seed identification. The network comprises a spectral image input unit, a main feature extraction unit, a multi-scale feature fusion unit, a regional graph structure modeling unit, a graph structure optimization unit and a target detection output unit. The spectral image input unit is used for receiving a seed image collected by a spectral camera or a fusion image formed by multi-band spectral images; the main feature extraction unit is used for extracting texture, edge, morphology and spectral space joint features of a seed target; the multi-scale feature fusion unit is used for fusing seed target features under different scales; the regional graph structure modeling unit is used for dividing an image into multiple regional nodes and constructing a non-directional weighted graph according to spatial adjacency relations and spectral feature similarities; the graph structure optimization unit is used for performing consistency optimization on regional level features based on graph Laplace regularization or graph total variation constraint; and the target detection output unit is used for outputting the position, category and confidence of a seed target. The application can enhance the regional consistency expression and fine-grained recognition ability of a seed target under complex spectral imaging conditions by combining a deep learning target detection network with regional graph structure optimization, and is suitable for seed intelligent identification, automatic sorting and quality detection scenes.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, deep learning, spectral image processing, and intelligent agricultural detection technology, specifically relating to a graph-structured deep learning target spectral detection network for seed identification. This network is suitable for scenarios such as seed category identification, seed target localization, seed quality detection, and automatic agricultural sorting under spectral camera acquisition conditions. Background Technology

[0002] Seed identification is a crucial step in intelligent agricultural inspection and seed quality control. Traditional seed identification typically relies on manual visual inspection or ordinary image processing methods. The identification results are easily affected by seed posture, background differences, lighting conditions, and human experience, making it difficult to meet the requirements of automated, batch, and high-precision inspection.

[0003] Spectroscopic cameras can acquire seed response information across multiple spectral bands, providing richer spectral features compared to ordinary RGB images. For seed categories with similar appearances, subtle texture differences, or indistinct morphological features, spectral images can provide stronger, finer-grained distinguishing criteria.

[0004] Deep learning object detection networks possess end-to-end feature learning capabilities, enabling them to simultaneously perform object localization and category recognition tasks. However, in spectral seed recognition scenarios, seed targets typically exhibit characteristics such as small scale, subtle edge textures, weak inter-category differences, and background interference. Relying solely on convolutional feature extraction can easily lead to issues like insufficient local feature representation, unstable target boundaries, and category confusion.

[0005] Existing object detection networks typically focus on pixel-level or feature map-level representations, lacking explicit modeling of regional structural relationships. For seed images, regions within the same seed usually exhibit strong spatial continuity and spectral similarity, while there are often significant differences in regional boundaries between seeds and the background, and between different seeds. Introducing prior knowledge of regional structure into object spectral detection networks could help improve the accuracy and stability of seed recognition.

[0006] Therefore, it is necessary to construct a target spectral detection network that can simultaneously utilize the feature representation capabilities of deep learning target detection networks, the spectral feature information of spectral images, and the regional consistency constraint capabilities of graph structure models, in order to improve the detection accuracy and generalization ability of seed identification under complex imaging conditions. Summary of the Invention

[0007] The purpose of this invention is to provide a graph-structured deep learning target spectral detection network for seed recognition, in order to solve the problems of existing spectral seed recognition networks such as insufficient regional structure modeling, inadequate fine-grained feature representation, and insufficient stability of detection output.

[0008] To achieve the above objectives, this invention provides a graph-structured deep learning network for target spectral detection applied to seed identification. The network includes a spectral image input unit, a backbone feature extraction unit, a multi-scale feature fusion unit, a region graph structure modeling unit, a graph structure optimization unit, and a target detection output unit; each unit is respectively used to complete spectral image input, deep feature extraction, multi-scale feature fusion, region graph structure modeling, graph structure optimization, and seed target detection output.

[0009] The backbone feature extraction unit includes one or more of the following: a convolutional feature extraction module, a residual connection module, a normalization module, and a nonlinear activation module; the multi-scale feature fusion unit includes one or more of the following: a feature upsampling module, a feature downsampling module, a feature concatenation module, or a feature weighted fusion module, used to fuse shallow texture features, mid-level morphological features, and high-level semantic features.

[0010] The backbone feature extraction unit or multi-scale feature fusion unit is equipped with an attention mechanism module, which includes one or more of the following: channel attention module, spatial attention module, spectral attention module, or spectral-spatial joint attention module; the attention mechanism module applies attention to the input features. Generate attention weights Enhanced features are obtained through element-wise multiplication. It is represented as:

[0011] The region map structure modeling unit divides the input image or intermediate feature map into regions using superpixel segmentation, region growing, cluster segmentation, or region partitioning based on feature map response. A region; if pixels The spectral vector at that location is Then the first Regional-level spectral characteristics of each region Represented as:

[0012] The undirected weighted graph is represented as follows: ,in, Represents a set of region nodes. This represents the set of edges between nodes in a region. Represents the weight matrix; the weight matrix elements in Determined by combining regional feature similarity and spatial proximity:

[0013] The graph structure optimization unit constructs a graph Laplace matrix. And based on graph Laplacian regularization, an optimization objective function is constructed. The graph Laplacian matrix, degree matrix, optimization objective function, and their linear equations are respectively expressed as:

[0014] In another embodiment, the graph structure optimization unit constructs an optimization objective function based on the graph total variation constraint, and fuses the output optimized features with features from the multi-scale feature fusion unit or the target detection output unit. The fusion method includes one or more of weighted summation, channel splicing, residual connection, or attention recalibration, as expressed as:

[0015] The target detection output unit outputs the result. The output includes a detection bounding box for the seed target, a class label, and a confidence score. The detection bounding box and the output are represented as follows:

[0016] The present invention also provides a graph-structured deep learning target spectral detection system for seed identification, comprising an image acquisition module, a spectral detection network module, and a result output module; wherein, the spectral detection network module is used to execute the above-mentioned graph-structured deep learning target spectral detection network, and the result output module is used to output seed detection boxes, category labels, and confidence scores.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the inference process of the graph-structured deep learning target spectral detection network described above.

[0018] Compared with existing technologies, the present invention has the following beneficial effects: First, the present invention combines the spectral features of spectral images with a deep learning object detection network, thereby improving the ability to express fine-grained seed categories; Second, the present invention explicitly utilizes spatial adjacency and spectral similarity relationships in seed images through region graph structure modeling, enabling more stable and consistent representation of regions within the same target; Third, the present invention optimizes region features through graph Laplacian regularization or graph total variation constraints, which helps to enhance the consistency within regions and maintain the seed boundary structure; Fourth, the present invention introduces an attention mechanism into the deep detection network, making the network pay more attention to seed edges, textures, and regions with significant spectral responses, thereby improving the accuracy and stability of object detection output. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of the graph-structured deep learning target spectral detection network provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the graph-structured deep learning target spectral detection network provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the backbone feature extraction and multi-scale feature fusion structure provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the superpixel segmentation and region map structure construction provided by the present invention.

[0023] Figure 5 This is a schematic diagram of graph structure optimization and detection output provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the system module structure provided by the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0026] like Figure 1 As shown, in this embodiment, a seed image is first acquired using a spectral camera, and then the seed image is input into a graph-structured deep learning target spectral detection network. The network sequentially performs processing steps such as backbone feature extraction, multi-scale feature fusion, region graph structure modeling, graph structure optimization, and detection output.

[0027] During the spectral image input stage, the input image can be complete multi-band spectral data or a fused image composed of several key bands. For multi-band data, a multi-channel tensor can be directly input according to the number of input channels of the network, or a fused image suitable for network input can be obtained through band combination. Let the input spectral image be:

[0028] in, Indicates the image height. Indicates the image width. This represents the number of spectral bands. In the backbone feature extraction stage, the network performs multi-layer convolutional feature extraction on the input image to obtain hierarchical features containing edge, texture, morphological, and semantic information. The multi-layered features output by the backbone feature extraction unit can be represented as:

[0029] in, This represents the backbone feature extraction function. Indicates learnable parameters, Indicates the first Layer output features. The multi-scale feature fusion unit fuses features from different levels to form detection features:

[0030] In one specific embodiment, the deep learning object detection part can adopt a Backbone-Neck-Head structure similar to YOLOv8, where the Backbone is used to extract multi-level features, the Neck is used to fuse features at different scales, and the Head is used to output the target location, category, and confidence score. This embodiment is only one optional implementation and does not limit the use of other deep learning object detection structures in this invention.

[0031] During the attention enhancement stage, the network embeds an attention mechanism module in the backbone feature extraction unit or multi-scale feature fusion unit. This attention mechanism module can employ one or more combinations of SE, CBAM, ECA, channel attention, spatial attention, or spectral-space joint attention. The channel attention weights can be expressed as:

[0032] in, Indicates global average pooling. and For learnable weights, Represents a non-linear activation function. This represents the Sigmoid activation function.

[0033] In the region graph structure modeling stage, the network divides the input image or intermediate feature map into multiple regions. Taking superpixel segmentation as an example, each superpixel region corresponds to a graph node, and the spatial adjacency and spectral similarity relationships between regions correspond to graph edges and edge weights. Let pixels... Spatial coordinates are The feature vector is Cluster center The spatial coordinates and eigenvectors are respectively and Then the pixel With cluster center The joint distance between them is expressed as:

[0034] in, For the area spacing parameter, This is a parameter for spatial compactness. For example... Figure 4 As shown, if two regions are spatially adjacent and have similar spectral features, the edge weight between them is larger; if there is a clear boundary between two regions or a large difference in spectral features, the edge weight between them is smaller. The region map structure constructed in this way can reflect the consistency of the internal regions of the seed image and the differences in the target boundary.

[0035] In the graph structure optimization stage, the graph structure optimization unit constructs a graph Laplacian matrix based on the region graph structure and optimizes the initial features at the region level. Graph Laplacian regularization can constrain the feature differences between adjacent and similar regions, making the regions within the same seed have more consistent feature responses.

[0036] In another specific embodiment, the graph structure optimization unit employs a total graph variation constraint. The total graph variation constraint can enhance the consistency within a region while preserving the target boundary structure, making it suitable for scenarios where the seed and background boundaries are clearly defined or where different seeds are in contact.

[0037] In the feature fusion stage, the region features optimized by the graph structure are combined with the multi-scale fused features. The combination method can be weighted summation, channel concatenation, residual connection, or attention recalibration. The fused features are then input into the target detection output unit.

[0038] During the detection output phase, the target detection output unit outputs the detection bounding box, category label, and confidence score of the seed target. The output results can be used in a display interface, a seed quality statistics system, or an automatic sorting actuator.

[0039] like Figure 6As shown, the present invention can also constitute a graph-structured deep learning target spectral detection system for seed identification, including an image acquisition module, a spectral detection network module, and a result output module. The spectral detection network module internally includes a backbone feature extraction unit, a multi-scale feature fusion unit, a region graph structure modeling unit, a graph structure optimization unit, and a target detection output unit.

Claims

1. A graph-structured deep learning target spectral detection network for seed identification, characterized in that... include: The spectral image input unit is used to receive seed images acquired by the spectral camera or fused images formed by multi-band spectral images; The system comprises the following components: a backbone feature extraction unit for deep feature extraction of the input image, yielding texture, edge, morphological, and spectral-space joint features of the seed target; a multi-scale feature fusion unit for fusing deep features at different scales to form fused features for target detection; a region graph structure modeling unit for dividing the input image or intermediate feature map into regions, using the divided regions as graph nodes, and constructing an undirected weighted graph based on spatial adjacency and spectral feature similarity between regions; and a graph structure optimization unit for constructing a graph Laplacian matrix or graph total variation constraint based on the undirected weighted graph, and optimizing the graph structure of the region-level features to obtain the optimized feature representation. The target detection output unit is used to output the location, category, and confidence level of the seed target based on the optimized feature representation.

2. The graph-structured deep learning target spectral detection network for seed identification according to claim 1, characterized in that, The backbone feature extraction unit includes one or more of the following: a convolutional feature extraction module, a residual connection module, a normalization module, and a nonlinear activation module; the multi-scale feature fusion unit includes one or more of the following: a feature upsampling module, a feature downsampling module, a feature concatenation module, or a feature weighted fusion module, used to fuse shallow texture features, mid-level morphological features, and high-level semantic features.

3. The graph-structured deep learning target spectral detection network for seed identification according to claim 1, characterized in that, The backbone feature extraction unit or multi-scale feature fusion unit is equipped with an attention mechanism module, which includes one or more of the following: channel attention module, spatial attention module, spectral attention module, or spectral-spatial joint attention module; the attention mechanism module applies attention to the input features. Generate attention weights Enhanced features are obtained through element-wise multiplication. It is represented as: in, This indicates element-wise multiplication.

4. The graph-structured deep learning target spectral detection network for seed identification according to claim 1, characterized in that, The region map structure modeling unit divides the input image or intermediate feature map into regions using superpixel segmentation, region growing, cluster segmentation, or region partitioning based on feature map response. One region; If pixels The spectral vector at that location is Then the first Regional-level spectral characteristics of each region Represented as: in, Indicates the first Each region Indicates the number of spectral bands. Indicates the first The number of pixels contained in each region.

5. The graph-structured deep learning target spectral detection network for seed identification according to claim 4, characterized in that, The undirected weighted graph is represented as follows: ,in, Represents a set of region nodes. This represents the set of edges between nodes in a region. Represents the weight matrix; the weight matrix elements in Determined through regional feature similarity and spatial proximity: in, and They represent the first The and the first Spectral or depth characteristics of a region and They represent the first The and the first The spatial center of each region and These are scale control parameters.

6. The graph-structured deep learning target spectral detection network for seed identification according to claim 5, characterized in that, The graph structure optimization unit constructs a graph Laplace matrix. And based on graph Laplacian regularization, an optimization objective function is constructed. The graph Laplacian matrix, degree matrix, optimization objective function, and their linear equations are respectively expressed as: in, Indicates the initial region characteristics. This represents the optimized region features. Represents the regularization parameter. It is an identity matrix.

7. The graph-structured deep learning target spectral detection network for seed identification according to claim 1, characterized in that, The graph structure optimization unit constructs an optimization objective function based on the graph total variation constraint, and fuses the output optimized features with the features in the multi-scale feature fusion unit or the target detection output unit. The fusion method includes one or more of weighted summation, channel splicing, residual connection, or attention recalibration, which are expressed as follows: in, Represents a node With nodes Edge weights between them and These represent the regional feature response values ​​at the corresponding nodes. This represents the regional characteristics after graph structure optimization. This represents the feature fusion function.

8. The graph-structured deep learning target spectral detection network for seed identification according to claim 1, characterized in that, The target detection output unit outputs the result. The output includes a detection bounding box for the seed target, a class label, and a confidence score. The detection bounding box and the output are represented as follows: in, Indicates the first The detection bounding box of each seed target, Indicates the first Category labels for each seed target, Indicates the first Confidence scores of each seed target Indicates the number of targets detected. Indicates the coordinates of the center of the detection box. and These represent the width and height of the detection frame, respectively.

9. A graph-structured deep learning target spectral detection system for seed identification, characterized in that, include: The image acquisition module is used to acquire seed images collected by the spectral camera; A spectral detection network module is used to execute the graph-structured deep learning target spectral detection network according to any one of claims 1 to 8; The results output module is used to output the seed detection box, class label, and confidence score.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the inference process of the graph-structured deep learning target spectral detection network according to any one of claims 1 to 8.

Images