Target crop feature classification method and device based on light sensing feature recognition

Abstract

Embodiments of the present disclosure disclose a target crop feature classification method and device based on light sensing feature recognition. A specific embodiment of the method comprises: constructing a space-time spectral tensor; performing light sensing feature extraction on the space-time spectral tensor to obtain a target crop light sensing feature information sequence; performing roughness feature extraction on the space-time spectral tensor to obtain a spatial base element matrix and a control constraint smoothing coefficient; generating target crop edge features and intrinsic component distribution features according to the control constraint smoothing coefficient, the target crop light sensing feature information sequence and the spatial base element matrix; and performing target crop feature classification according to the target crop edge features and the intrinsic component distribution features to obtain a target crop classification result. The embodiment can improve the accuracy of the target crop classification result.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more particularly to the field of image recognition technology and crop classification technology, specifically to a method and apparatus for classifying target crop features based on light-sensing feature recognition. Background Technology

[0002] Crop classification plays a crucial role in crop operation and sales. Currently, the conventional approach often involves adding automated production lines and collecting color images of crops (e.g., apples). Initial identification of surface features in these images is performed, followed by manual secondary screening for crop grading (e.g., classifying crops with undamaged exteriors and acceptable color into the highest grade). However, grading solely based on surface features ignores the internal component content characteristics that influence crop grading. Furthermore, manual secondary screening relies on human experience, which can lead to decreased accuracy when human experience is insufficient, resulting in inaccurate crop classification results. Summary of the Invention

[0003] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0004] Some embodiments of this disclosure propose a method and apparatus for classifying target crop features based on light-sensing feature recognition, in order to solve the technical problems mentioned in the background section above.

[0005] In a first aspect, some embodiments of this disclosure provide a target crop feature classification method based on photosensitive feature recognition. The method includes: constructing a spatiotemporal spectral tensor based on a target crop image sequence acquired by a multispectral camera, wherein the target crop image is a discrete infrared spectral image within a preset wavelength range taken at a fixed angle for the target crop to be classified on a conveyor belt, and the target crop is a single type of crop; extracting photosensitive features from the aforementioned spatiotemporal spectral tensor to obtain a target crop photosensitive feature information sequence, wherein the target crop photosensitive feature information includes: a feature spectral vector and a dynamic brightness gain operator; and classifying the target crop based on the aforementioned photosensitive feature information. The light-sensing feature information sequence is used to extract roughness features from the aforementioned spatiotemporal spectral tensor, resulting in a spatial primitive matrix and control constraint smoothing coefficients. The spatial primitive matrix characterizes the roughness features and spatial structural continuity of the target crop. Based on the control constraint smoothing coefficients, the light-sensing feature information sequence of the target crop, and the spatial primitive matrix, edge features and intrinsic component distribution features of the target crop are generated. Based on these edge features and intrinsic component distribution features, the target crop is classified to obtain a classification result, which includes a target crop component density sequence and a crop-level classification label.

[0006] Secondly, some embodiments of this disclosure provide a target crop feature classification device based on photosensitive feature recognition. The device includes: a construction unit configured to construct a spatiotemporal spectral tensor based on a sequence of target crop images acquired by a multispectral camera, wherein the target crop images are discrete infrared spectral images within a preset wavelength range taken at a fixed angle for the target crop to be classified on a conveyor belt, and the target crop is a single type of crop; a photosensitive feature extraction unit configured to extract photosensitive features from the aforementioned spatiotemporal spectral tensor to obtain a sequence of target crop photosensitive feature information, wherein the target crop photosensitive feature information includes: a feature spectral vector and a dynamic brightness gain operator; and a roughness feature extraction unit configured to extract roughness features based on the aforementioned spatiotemporal spectral tensor. The target crop photosensitive feature information sequence is used to extract roughness features from the aforementioned spatiotemporal spectral tensor, resulting in a spatial primitive matrix and control constraint smoothing coefficients. The spatial primitive matrix characterizes the roughness features and spatial structural continuity of the target crop. A generation unit is configured to generate edge features and intrinsic component distribution features of the target crop based on the control constraint smoothing coefficients, the target crop photosensitive feature information sequence, and the spatial primitive matrix. A target crop feature classification unit is configured to classify the target crop features based on the edge features and intrinsic component distribution features, obtaining a target crop classification result. This result includes a target crop component density sequence and a target crop-level classification label.

[0007] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0008] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0009] The above embodiments of this disclosure have the following beneficial effects: the target crop feature classification method based on photosensitive feature recognition of some embodiments of this disclosure can improve the accuracy of target crop classification results. In practice, classification of target crops based solely on their surface characteristics ignores the component content characteristics that affect the taste of crops. Taking apples as an example, the grade (taste) of apples includes not only the apple's surface color but also its sugar content. Therefore, this disclosure first constructs a spatiotemporal spectral tensor based on the target crop image sequence acquired by a multispectral camera. The target crop image is a discrete infrared spectral image within a preset wavelength range taken at a fixed angle for the target crop to be classified on a conveyor belt, and the target crop is a single type of crop. In practice, the taste and nutritional value of crop types are reflected not only in the surface characteristics of crops but also in the content of their internal components. However, classifying crops based on surface characteristics (e.g., color, smoothness, etc.) can only select phenotypic qualified crops, making it difficult to distinguish the actual taste and nutritional value of crops. As a result, the crop classification results deviate from the actual meaning. Therefore, this application considers the differences in the absorption characteristics of specific components within crop tissues to wave energy in the infrared band. Thus, discrete infrared spectral images of the target crop (e.g., apple) are acquired to facilitate the extraction of internal component content characteristics. Then, photosensitive features are extracted from the aforementioned spatiotemporal spectral tensor to obtain a photosensitive feature information sequence for the target crop. This photosensitive feature information includes a feature spectral vector and a dynamic brightness gain operator. Here, the generated feature spectral vector can be used to characterize the light absorption properties of components within the target crop, thereby facilitating the identification of component content. Next, based on the aforementioned photosensitive feature information sequence, roughness features are extracted from the aforementioned spatiotemporal spectral tensor to obtain a spatial primitive matrix and a control constraint smoothing coefficient. The spatial primitive matrix characterizes the roughness characteristics and spatial structural continuity of the target crop. In practice, factors influencing target crop classification also include the roughness and surface continuity of the target crop. By generating a spatial primitive matrix, the roughness characteristics and spatial structural continuity characteristics of the target crop can be introduced, thereby further improving the accuracy of target crop feature classification. Subsequently, based on the aforementioned control constraint smoothing coefficient, the aforementioned target crop photosensitive feature information sequence, and the aforementioned spatial primitive matrix, edge features and intrinsic component distribution features of the target crop are generated. Finally, target crop feature classification is performed based on the aforementioned target crop edge features and intrinsic component distribution features to obtain the target crop classification result, which includes the target crop component density sequence and the target crop-level classification label. In summary, this method can identify the component density contained in the target crop, thereby classifying the target crop from multiple dimensions and improving the accuracy of the target crop classification results. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0011] Figure 1 This is a flowchart of some embodiments of the target crop feature classification method based on light-sensing feature recognition according to some embodiments of this disclosure; Figure 2 This is a schematic diagram illustrating the generation of the characteristic spectral vector; Figure 3 This is a schematic diagram illustrating the generation of the spatial neighborhood feature matrix; Figure 4 This is a schematic diagram of the structure of some embodiments of the target crop feature classification device based on photosensitive feature recognition according to the present disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0014] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0015] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0017] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] Figure 1 A flowchart 100 is shown, illustrating some embodiments of a target crop feature classification method based on light-sensing feature recognition according to this disclosure. This target crop feature classification method based on light-sensing feature recognition includes the following steps: Step 101: Construct a spatiotemporal spectral tensor based on the target crop image sequence acquired by the multispectral camera.

[0019] In some embodiments, the executing entity (e.g., a computing device) of the target crop feature classification method based on photosensitive feature recognition can construct a spatiotemporal spectral tensor based on the target crop image sequence acquired by a multispectral camera. Here, the target crop is a single type of crop, i.e., an agricultural product that can be selected one by one through visual recognition. The target crop image is a discrete infrared spectral image (i.e., a grayscale image) captured at a fixed angle within a preset wavelength range of the target crop to be classified on the conveyor belt. Here, the multispectral camera is a device used to capture images of a single target crop to be classified on the conveyor belt. The acquisition wavelength range of the multispectral camera is [400 nm, 1000 nm]. Each target crop image in the target crop image sequence corresponds to the same target crop and the same coordinate region. Each target crop image in the target crop image sequence corresponds to a different acquisition wavelength (spectral channel) and is arranged in order of wavelength size.

[0020] In practice, multispectral cameras capture images of target crops being transported sequentially on a conveyor belt at a fixed frequency. The shooting frequency corresponds to the conveyor belt speed, ensuring that images of the target crops are captured. For example, the target crop could be an apple, watermelon, or pear.

[0021] Secondly, following the order of the spectral channels of each target crop image, the pixel values ​​in each target crop image are combined into a spatiotemporal spectral tensor (scale H×W×S). The spatiotemporal spectral tensor is a three-dimensional data cube. Here, H represents the height of the target crop image, W represents the width of the target crop image, and S represents the number of spectral channels.

[0022] It should be noted that the aforementioned computing device can be either hardware or software. When the computing device is hardware, it can be implemented as a single server or a single terminal device. When the computing device is software, it can be installed within the hardware devices listed above. It can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are made here. Specifically, the aforementioned execution entity can be a crop classification system based on light-sensing feature recognition.

[0023] Step 102: Extract photosensitive features from the spatiotemporal spectral tensor to obtain a sequence of photosensitive feature information of the target crop.

[0024] In some embodiments, the aforementioned execution entity can extract photosensitive features from the aforementioned spatiotemporal spectral tensor to obtain a sequence of photosensitive feature information of the target crop. The photosensitive feature information of the target crop includes: a feature spectral vector and a dynamic brightness gain operator. The photosensitive feature information of the target crop is generated based on the light absorption characteristics of the components within the target crop. The feature spectral vector is used to characterize the light absorption characteristics of the components within the target crop. The dynamic brightness gain operator is used to perform inverse logarithmic scaling on each pixel in the subsequent spatial neighborhood feature matrix, thereby filtering out false texture gradients caused by attenuation from external radiation sources or the intervention of stray environmental light.

[0025] As an example, taking apples as the target crop, the feature spectral vector can contain data describing the light absorption characteristics of the apple's internal components. Here, the apple's internal components can include, but are not limited to, at least one of the following: moisture, sugar, organic acids, pigments, etc.

[0026] Specifically, the spatiotemporal spectral tensor is first flattened into a two-dimensional spectral matrix (feature dimension: (1×(H×W))×S) according to the spectral channels. The column vectors of the two-dimensional spectral matrix (feature dimension: 1×(H×W)) correspond to the spectral channels. Then, the arithmetic mean, standard deviation, skewness, kurtosis, and energy value corresponding to each column vector in the two-dimensional spectral matrix are calculated. Finally, the arithmetic mean, standard deviation, skewness, kurtosis, and energy value corresponding to each column vector are combined to form the feature spectral vector (feature dimension: 1×5) in the photosensitive feature information of the target crop. Furthermore, the standard deviation of the pixel values ​​corresponding to each column vector in the two-dimensional spectral matrix can be used as a dynamic gain operator to intuitively reflect the fluctuation of image contrast.

[0027] Here, the arithmetic mean reflects the overall reflectance intensity of the spectral channel (spectral band). The standard deviation reflects the pixel dispersion of the spectral channel, sensitively expressing the spatial non-uniformity of the crop (e.g., bruising on the target crop results in spatial non-uniformity). Skewness reflects the asymmetry of the gray-level distribution. For example, bruising on the target crop causes low gray-level tailing. Kurtosis reflects the steepness of the gray-level distribution curve of the target crop in the same spectral channel, thus distinguishing smooth surfaces from diseased surfaces. The energy value reflects the uniformity of the gray-level distribution of the grayscale image; higher energy indicates more regular texture.

[0028] In some optional implementations of certain embodiments, the execution entity performs photosensitive feature extraction on the spatiotemporal spectral tensor to obtain a sequence of photosensitive feature information of the target crop, including: Step S1: The average grayscale value within the preset background area on the conveyor belt in the aforementioned spatiotemporal spectral tensor is determined as the environmental reference brightness factor. The preset background area is the conveyor belt area captured by the camera. That is, the preset background area is the captured surface of the conveyor belt. The environmental reference brightness factor describes the overall response level of the preset background area to the light radiation intensity of each wavelength channel under the current imaging conditions.

[0029] Step S2: For each coordinate point of the above spatiotemporal spectral tensor, perform the following first processing step: Step S21: Determine each gray value in the spatiotemporal spectral tensor corresponding to the coordinate points as a continuous spectral gray value, thus obtaining a continuous spectral gray value sequence. Specifically, the gray values ​​of each wavelength channel at the coordinate point can be determined as continuous spectral gray values, resulting in a continuous spectral gray value sequence. The continuous spectral gray value sequence is the gray value sequence corresponding to each acquisition wavelength at the coordinate point.

[0030] Step S22: Determine the gray-level response slope sequence corresponding to the above continuous spectral gray-level value sequence. The gray-level response slope characterizes the rate of change of gray-level value with respect to spectral channels. Here, the difference slope between each two adjacent continuous spectral gray-level values ​​in the continuous spectral gray-level value sequence can be determined as the initial gray-level response slope (gray-level response slope = difference between gray-level values ​​of two adjacent continuous spectral lines - wavelength difference of the spectral channels corresponding to gray-level values ​​of two adjacent continuous spectral lines). Finally, since the difference operation amplifies noise, when the multispectral data has high noise, the difference result may be unreliable. Therefore, a Savitzky-Golay filter is used to smooth each initial gray-level response slope to obtain the gray-level response slope sequence.

[0031] Step S23: Based on the aforementioned gray-scale response slope sequence, construct the feature spectral vector of the target crop photosensitive feature information corresponding to the aforementioned coordinate points in the target crop photosensitive feature information sequence. The aforementioned gray-scale response slope sequence can be determined as the feature spectral vector.

[0032] As an example, see Figure 2 The gray values ​​corresponding to the aforementioned coordinate points in the spatiotemporal spectral tensor 202 are determined as continuous spectral line gray values, resulting in a continuous spectral line gray value sequence 202. Then, the gray-level response slope between every two adjacent continuous spectral line gray values ​​in the continuous spectral line gray value sequence 202 is determined, resulting in a gray-level response slope sequence 203. Finally, based on the gray-level response slope sequence 203, a characteristic spectral vector 204 is constructed.

[0033] Step S24: Determine the average gray value of at least one gray value within a preset window corresponding to the coordinate point in the aforementioned spatiotemporal spectral tensor, and use it as the current average gray value. The preset window is a three-dimensional window used to select gray values ​​of three-dimensional coordinates within a preset neighborhood centered on the coordinate point. Next, the average gray value within the preset window can be determined as the current average gray value.

[0034] Step S25: The ratio of the current average grayscale value to the environmental baseline brightness factor is determined as the dynamic brightness gain operator in the target crop light-sensing feature information corresponding to the coordinate point in the target crop light-sensing feature information sequence. The dynamic brightness gain operator characterizes the difference in brightness distortion between the coordinate point and the corresponding image acquisition area (preset background area) of the camera.

[0035] Step 103: Based on the target crop photosensitive feature information sequence, roughness features are extracted from the spatiotemporal spectral tensor to obtain the spatial primitive matrix and control constraint smoothing coefficient.

[0036] In some embodiments, the execution entity can extract roughness features from the spatiotemporal spectral tensor based on the target crop photosensitive feature information sequence, obtaining a spatial primitive matrix and control constraint smoothing coefficients. The spatial primitive matrix characterizes the roughness features and spatial structural continuity of the target crop. The control constraint smoothing coefficients are dimensionless, regularized, and adjustable parameters. Secondly, for each wavelength channel in the spatiotemporal spectral tensor, the variance of the grayscale value corresponding to each pixel coordinate in the wavelength channel can be determined using a sliding window (e.g., 3×3). Then, the variances of the grayscale values ​​can be combined into a spatial primitive matrix according to the order of the pixel coordinates.

[0037] The control constraint smoothing coefficient is generated through the following steps: First, the historical spatiotemporal spectral tensor of the previous frame is obtained. Then, the absolute values ​​of the differences between corresponding pixels in the current frame's spatiotemporal spectral tensor and the historical spatiotemporal spectral tensor are determined, resulting in a set of absolute values. Next, the average value of each absolute value in the set is determined as the initial smoothing coefficient. Then, the average value of the dynamic brightness gain operator included in each target crop photosensitive feature information sequence is determined. Finally, the ratio of the initial smoothing coefficient to this average value is determined as the control constraint smoothing coefficient. The control constraint smoothing coefficient reflects the degree of perturbation in the image's internal contrast.

[0038] In some optional implementations of certain embodiments, the execution entity extracts roughness features from the spatiotemporal spectral tensor based on the target crop photosensitive feature information sequence to obtain a spatial primitive matrix and control constraint smoothing coefficients, including: Step S1: Determine the gray-level gradient feature corresponding to each coordinate point of the aforementioned spatiotemporal spectral tensor. This can be achieved using the Sobel operator, which determines the gray-level gradient magnitude at each coordinate point as the gray-level gradient feature.

[0039] Step S2: For each coordinate point of the above spatiotemporal spectral tensor, perform the following second processing step: Step S21: Select the gray values ​​within the neighborhood window corresponding to the coordinate point from the spatiotemporal spectral tensor above, and use them as the spatial neighborhood feature matrix. The neighborhood window is a two-dimensional window (e.g., a window size of 5×5). Next, in the spatiotemporal spectral tensor, select the gray values ​​within the neighborhood window range centered on the coordinate point, and then use each gray value within the neighborhood window as the spatial neighborhood feature matrix.

[0040] As an example, see Figure 3 For each coordinate point of the aforementioned spatiotemporal spectral tensor 301, the gray value within the neighborhood window 302 corresponding to the coordinate point can be selected from the aforementioned spatiotemporal spectral tensor 301 as the spatial neighborhood feature matrix 303.

[0041] Step S22: Based on the dynamic brightness gain operator included in the target crop light-sensing feature information corresponding to the coordinate points in the above target crop light-sensing feature information sequence, the spatial neighborhood feature matrix is ​​scaled to generate a processed neighborhood matrix. Specifically, the spatial neighborhood feature matrix is ​​scaled inversely using the dynamic brightness gain operator to obtain the processed neighborhood matrix. The inverse logarithmic scaling can be expressed as: processed neighborhood matrix = log(spatial neighborhood feature matrix / dynamic brightness gain operator). Here, the logarithm base is 10.

[0042] In practice, multispectral imaging is subject to two types of external light interference: radiative attenuation caused by light source aging and changes in shooting distance constitutes multiplicative noise, while ambient stray light introduces a fixed gray-level shift, manifesting as additive noise. Performing a logarithmic transformation on the image utilizes the logarithmic decomposition of products to convert the multiplicative attenuation term coupled to the signal into an additive DC component. This low-frequency constant component can be easily removed through band difference and mean subtraction. Simultaneously, the logarithmic transformation exhibits an inverse mapping principle: the logarithmic gray-level value decreases for high-reflectivity bright targets and increases for low-reflectivity dark targets, effectively amplifying the weak spectral features caused by light absorption from biochemical components such as chlorophyll and water. From a frequency domain perspective, radiative attenuation and stray light form a large-scale, gently varying low-frequency background, which is compressed into an approximate constant after the logarithmic transformation, achieving automatic cancellation. Local gray-level abrupt changes caused by real structures such as crop epidermal cell wall protrusions and epidermal cracks are high-frequency components, which can be completely preserved within the logarithmic domain difference features. Therefore, the decoupling and separation of the light interference background and the effective crop signal are achieved.

[0043] Step S23: Based on the processed neighborhood matrix described above, a local gray-level co-occurrence matrix is ​​generated. This local gray-level co-occurrence matrix represents the spatial gray-level co-occurrence topological relationship between coordinate points within the neighborhood window. Furthermore, in the spatiotemporal spectral tensor, according to the coordinate positions corresponding to the processed neighborhood matrix, the probability of occurrence of gray-level value pairs between every two pixels is calculated and used as values ​​in the local gray-level co-occurrence matrix, thus obtaining the local gray-level co-occurrence matrix.

[0044] Step S24: Generate crop surface feature vectors based on the aforementioned local gray-level co-occurrence matrix. First, determine the contrast of the local gray-level co-occurrence matrix to reflect the higher the contrast, indicating a rougher target crop surface (i.e., greater local gray-level differences). Then, determine the energy value of the local gray-level co-occurrence matrix to reflect the spatial continuity of the target crop surface. Rough surfaces with fine textures have lower energy, while continuous, uniform surfaces have higher energy values. Next, determine the entropy and correlation of the local gray-level co-occurrence matrix. Rough crop surfaces have higher entropy, while continuous, ordered crop surfaces have lower entropy values. Correlation reflects the spatial continuity of the target crop; strong spatial continuity indicates a high linear correlation between the gray levels of adjacent pixels. Finally, construct the surface feature vector using contrast, energy, entropy, and correlation as vector values.

[0045] Step S3: Based on the generated sequence of crop surface feature vectors, construct the spatial primitive matrix corresponding to the coordinate points in the spatial primitive matrix. The spatial primitive matrix can be formed by combining the surface feature vectors according to the order of the coordinate points. Here, the spatial primitive matrix is ​​a high-order feature map that numerically and vectorly encodes the microstructure (roughness, continuity) of the crop surface.

[0046] Step S4: Based on the gray-level gradient features corresponding to the processed neighborhood matrices in the obtained processed neighborhood matrix sequence, generate the gray-level gradient discrete variance. The gray-level gradient discrete variance characterizes the spatial non-uniformity of the crop surface texture in the current frame. First, the first moment of the gray-level gradient feature corresponding to each processed neighborhood matrix can be determined. Then, based on the first moment, the second central moment of the gray-level gradient feature corresponding to each processed neighborhood matrix can be determined. The second central moment characterizes the discrete variance corresponding to each coordinate point. Finally, the mean of all second central moments is determined as the gray-level gradient discrete variance.

[0047] In practice, a small discrete variance in the gray-level gradient indicates that the target crop surface is smooth or rough but uniformly distributed (high structural consistency). A large discrete variance in the gray-level gradient indicates that there are obvious local differences on the surface of the target crop (such as half with russeting and half smooth), which usually corresponds to lesions or local damage.

[0048] Step S5: Based on the above-mentioned gray-level gradient discrete variance and benchmark discrete variance, generate the control constraint smoothing coefficient. Wherein, the control constraint smoothing coefficient = (gray-level gradient discrete variance) / (variable variance). (Baseline discrete variance) / (Baseline discrete variance + Steady-state adjustment constant). Here, the value range of the steady-state adjustment constant is [0.001, 0.005].

[0049] Step 104: Based on the control constraint smoothing coefficient, the target crop photosensitive feature information sequence, and the spatial primitive matrix, generate the target crop edge features and intrinsic component distribution features.

[0050] In some embodiments, the execution entity can generate target crop edge features and intrinsic component distribution features based on the control constraint smoothing coefficient, the target crop photosensitive feature information sequence, and the spatial primitive matrix. First, an adaptive decoupling strength is generated based on the control constraint smoothing coefficient. If the control constraint smoothing coefficient is less than or equal to 0.15 (i.e., considered a steady state, decoupling can be skipped), the adaptive decoupling strength is 0; if the control constraint smoothing coefficient is greater than 0.15, the adaptive decoupling strength is 1. Then, the spatial primitive matrix is ​​normalized using a maximum-minimum normalization algorithm to obtain a normalized matrix, which serves as the target crop edge feature. Finally, the feature spectral vectors in the target crop photosensitive feature information are combined into a feature spectral matrix, which, combined with the normalized matrix, generates the intrinsic component distribution features. Here, intrinsic component distribution features = feature spectral matrix - adaptive decoupling strength × feature spectral matrix × target crop edge features.

[0051] In practice, areas with high roughness often experience enhanced diffuse reflection due to surface unevenness (multiplicative interference). In this case, by using adaptive decoupling intensity control to reduce the rejection ratio, this "spurious increment" is subtracted from the original reflectance, retaining only the intrinsic absorption spectrum related to internal biochemical molecules (chlorophyll, water). This improves the accuracy of the feature.

[0052] In some optional implementations of certain embodiments, the execution entity generates target crop edge features and intrinsic component distribution features based on the control constraint smoothing coefficient, the target crop photosensitive feature information sequence, and the spatial primitive matrix, including: Step S1 involves fusing the feature spectral vectors and spatial primitive matrices included in each target crop photosensitive feature information sequence to obtain a fused feature matrix. This can be achieved by concatenating the feature spectral vectors and spatial primitive matrices column-wise. Each row of the fused feature matrix represents the joint feature vector corresponding to a pixel. Here, the first K columns of the fused feature matrix are the spatial primitive matrix, and the last B columns are the fused feature matrix. K is the number of columns in the spatial primitive matrix, and B is the number of columns in the fused feature matrix.

[0053] Step S2: In response to the control constraint smoothing coefficient being less than or equal to a preset safety threshold, singular value decomposition (SVD) is performed on the fused feature matrix according to a preset penalty weight to obtain the target crop edge features and intrinsic component distribution features. The preset safety threshold ranges from [0.15, 0.35]. A control constraint smoothing coefficient less than or equal to the preset safety threshold indicates that the current frame is in a steady state, and external interference can be ignored. The preset penalty weight is the multiplier of the regularization matrix in the cost function during SVD. Here, the fused feature matrix can be decomposed using a singular value decomposition algorithm to obtain a left singular matrix. Finally, the first column of the left singular matrix is ​​reshaped into a two-dimensional matrix according to its original size using the Reshape function, serving as the target crop edge features. The remaining B columns of data are then segmented from the decomposed overall matrix, serving as the intrinsic component distribution features.

[0054] Step S3: In response to the control constraint smoothing coefficient being greater than the preset safety threshold, the preset penalty weight is adjusted according to the control constraint smoothing coefficient to generate perturbation constraint weights. Wherein, a control constraint smoothing coefficient greater than the preset safety threshold indicates that the current frame receives significant external environmental interference, resulting in low confidence of the original data. First, the perturbation magnitude of the control constraint smoothing coefficient is determined (perturbation magnitude = max(0, control constraint smoothing coefficient)). (Preset penalty weight). Then, the preset penalty weight is amplified proportionally using the perturbation magnitude to obtain the perturbation constraint weight (perturbation constraint weight = preset safety threshold × (1 + perturbation magnitude)). Here, the perturbation constraint weight is the magnitude by which the constraint smoothing coefficient deviates from the safety threshold, so as to suppress spurious spectral fluctuations introduced by vibrations or stray light when activated.

[0055] Step S4: Based on the aforementioned perturbation constraint weights, perform singular value decomposition (SVD) on the fused feature matrix to obtain the target crop edge features and intrinsic component distribution features. Specifically, the perturbation constraint weights are fed forward into the iterative constraint loop during the SVD process to dynamically adjust the iterative constraint boundaries. Then, the fused feature matrix is ​​decomposed using the SVD algorithm to obtain a left singular matrix (i.e., a spatial orthogonal basis) and a right singular value matrix (i.e., a spectral orthogonal basis). Using the Reshape function, the first column of the left singular matrix is ​​reshaped into a two-dimensional matrix according to its original size, serving as the target crop edge features. Target column data is then extracted from the right singular value matrix, serving as the intrinsic component distribution features. The target column is the column in the fused feature matrix corresponding to the spatial primitive matrix.

[0056] In practice, abrupt changes in spatial neighborhood (such as edges and roughness) are the primary source of differences between pixels, dominating the largest variance in the data matrix. Therefore, the first left singular vector (the first column in the left singular matrix) captures the largest spatial structural variations in the image (i.e., major contours and texture boundaries). Secondly, the original spectrum is contaminated with diffuse reflection multiplicative interference (spurious gradients) caused by surface roughness. Through low-rank reconstruction (discarding smaller singular values), these "noise patterns" can be removed from the signal, recovering the intrinsic absorption spectrum determined solely by internal biochemical molecules (chlorophyll, water, sugars).

[0057] Step 105: Classify the target crop features based on the marginal features and intrinsic component distribution features of the target crop to obtain the target crop classification results.

[0058] In some embodiments, the aforementioned execution entity can classify the target crop features based on the target crop edge features and the intrinsic component distribution features to obtain target crop classification results. The target crop classification results include a target crop component density sequence and a target crop level classification label. Specifically, pre-calibrated regression coefficients can be used to regress each pixel value in the intrinsic component distribution features to obtain a density distribution map. Here, target component density = pixel value × regression coefficient + intercept. Then, the average density value of each column in the density distribution map is used as the target crop component density to obtain the target crop component density sequence. Next, the target crop edge features and intrinsic component distribution features are aggregated using the Concat function to obtain aggregated features. Then, a linear layer and a Softmax activation function are used to generate a target crop level classification label. Specifically, the target crop component density represents the visual quantification of the overall component distribution of the target crop. The target crop level classification label is a label representing the target crop category. Taking apples as an example, the target crop categories are divided into three categories from best to worst: Category 1, Category 2, and Category 3. If the target crop corresponds to the optimal category, then the target crop's category-level classification label is: "Category 1".

[0059] In some optional implementations of certain embodiments, the execution entity performs target crop feature classification based on the target crop edge features and the intrinsic component distribution features to obtain target crop classification results, including: Step S1 involves encoding the target crop edge features and intrinsic component distribution features using a feature encoder to generate a shared feature vector. This shared feature vector contains encoded features that include both the target crop edge features and spatial structure features. The feature encoder consists of three serially connected 2D convolutional layers (a first, second, and third convolutional layer). All three convolutional layers have the same kernel size (3×3). Here, the intrinsic component distribution features can be channel-concatenated to a spectral orthogonal basis to obtain the input features (shape 128 × 128 × 8). Then, the input features become the input to the first convolutional layer, the output of which becomes the input to the second convolutional layer (shape 128 × 128 × 32), and the output of which becomes the input to the third convolutional layer (shape 64 × 64 × 64). Finally, the third convolutional layer outputs the shared feature vector (shape 32 × 32 × 128).

[0060] In addition, the feature encoder is the backbone network of a pre-trained shallow neural network. The shallow neural network also includes a regression head and a classification head. Here, the loss function for the regression head is the pixel-wise mean squared error function. The loss function for the classification head is the classification cross-entropy loss function.

[0061] Step S2 involves identifying the target crop components from the shared feature vectors to obtain the target crop component density sequence included in the target crop classification results. Specifically, a regression head is used to identify the target crop components from the shared feature vectors to obtain the target crop component density sequence. Here, the regression head consists of two sequentially connected upsampling layers and a linear layer. The two upsampling layers have identical structures (4×4 kernel size). The output of the first upsampling layer serves as the input to the second upsampling layer (shape 64 × 64 × 64), and the output of the second upsampling layer serves as the input to the linear layer (shape 128 × 128 × 32). Finally, the linear layer outputs the target crop component density sequence (shape 128 × 128 × K). K represents the pre-defined number of component types. The target crop component density is essentially the density map of the corresponding component type.

[0062] Step S3 involves classifying the shared feature vectors to obtain the target crop classification results, including the target crop category-level label. Specifically, a classification head is used to classify the shared feature vectors to obtain the target crop category-level label. This classification head consists of a global average pooling layer and two fully connected layers sequentially. The output of the global average pooling layer is the input to the first fully connected layer (1 × 1 × 128 shape). The output of the first fully connected layer is the input to the second fully connected layer (a 64-dimensional vector). Finally, the second fully connected layer outputs the target crop category-level label. Taking apples as an example, the target crop categories are divided into three types from best to worst: Type 1, Type 2, and Type 3. If the target crop corresponds to the best type, then the target crop category-level label is "Type 1".

[0063] Optionally, the above methods also include: Step S1: Based on the target crop classification results and the historical classification result sequence, generate the result error deviation. Specifically, for each target component density corresponding to the target crop in the target crop classification results and the historical component densities in the historical classification results, the root mean square error between the target crop component density and each historical component density is used as the component deviation. Then, the mean of the deviations of each component is determined as the result error deviation. Here, the historical classification result sequence is the classification results of the target crop within a preset time period.

[0064] Step S2: In response to the result error deviation exceeding a preset error threshold, a classification warning signal is generated and sent to the target terminal. The classification warning signal indicates a system drift error in the current classification result sequence. The result error deviation and the corresponding item-level classification label for the target crop can be used to determine the classification warning signal.

[0065] Optionally, the above methods also include: Step S1: Based on the above target crop classification results, generate the target radiant power. First, through the inverse operation of the PLS (Partial Least Squares) model, determine the absorbance sequence of the target crop component density sequence at a preset wavelength in the above target crop classification results. Then, calculate the transmitted radiant power according to Beer-Lambert's law to obtain the target radiant power.

[0066] Step S2: Based on the target radiation power, the infrared radiation power of the multispectral camera, and the control gain coefficient corresponding to the infrared radiation power, a pulse width modulation (PWM) command is generated. The difference between the target radiation power and the infrared radiation power of the multispectral camera can be determined as the infrared radiation power difference. The product of the infrared radiation power difference and the control gain coefficient is determined as the correction amount for the PWM command. Next, this correction amount is filled into a preset modulation command to obtain the PWM command. The preset modulation command is the command for modulating the pulse width, to which a specific correction value is to be added. The control gain coefficient is a parameter used to adjust the pulse width.

[0067] Step S3: Adjust the pulse parameters of the multispectral camera according to the pulse width modulation command, and obtain the real-time radiation power after the pulse parameter adjustment is completed. Specifically, the pulse width modulation command can be sent to the multispectral camera to adjust the infrared pulse width of the multispectral camera.

[0068] Step S4: The difference between the real-time radiated power and the target radiated power is determined as the radiated power residual.

[0069] Step S5: Constrain the control gain coefficient within a gain range according to the aforementioned radiated power residual. Specifically, firstly, if the radiated power residual is greater than zero and the control gain parameter is within the gain range, the control gain parameter is adjusted upwards based on a preset change in control gain parameter. If the radiated power residual is greater than zero and the control gain parameter exceeds the gain range (greater than the maximum gain value), the control gain parameter is adjusted downwards. If the radiated power residual is less than zero and the control gain parameter is within the gain range, the control gain parameter is adjusted downwards based on a preset change in control gain parameter. If the radiated power residual is less than zero and the control gain parameter exceeds the gain range (less than the minimum gain value), the control gain parameter is adjusted upwards based on a preset change in control gain parameter.

[0070] In practice, from the perspective of imaging physics, crop surfaces have complex geometric curvatures and porous structures, resulting in strong scattering of infrared radiation energy. Each pixel in the spectral image is essentially a mixture of edge geometric signals and multi-component energy spectra. However, the continuous movement of high-speed conveyor belts, the random tumbling of targets on the conveyor belt, and the accompanying high-frequency mechanical vibrations cause diffuse reflection halos to exhibit nonlinear and asymmetric spatial diffusion between different spectral channels. Consequently, the static gray-level co-occurrence matrix established by traditional methods suffers spatial degradation and distortion, and unpredictable non-uniform brightness level abrupt changes occur in the energy distribution between adjacent channels. To address this high-frequency channel characteristic degradation disturbance, one approach in the industry is to add high-precision physical vibration damping mechanisms or introduce high-order environmental compensation light sources to strengthen the hardware. However, this approach leads to an increase in the cumulative manufacturing and maintenance costs of sorting equipment, limiting its industrial-scale deployment. Another approach is to suspend the matrix decoupling calculation of the current frame by a hard threshold when a disturbance is detected, and then use the decoupling weight of the previous steady-state frame to perform linear extrapolation replacement of the current distorted frame. Although this method avoids the influence of single-frame distorted data, it disrupts the continuity of the temporal feature flow under continuous vibration conditions. The feature recognition accuracy after extrapolating multiple frames becomes cumulatively divergent, and may even misjudge large-scale changes in image grayscale as drastic changes in crop edge morphology, causing irreversible decision transmission distortion.

[0071] Therefore, the two approaches in the industry still have the following technical problems: First, the visual topological degradation of spatially mixed pixels has asymmetry across spectral channels, resulting in a lack of adaptive constraint on the distortion differences of each channel; Second, frequent transient jitter during continuous sampling repeatedly triggers the suspension of the computation flow and the extrapolation of historical steady-state frames, which disrupts the continuity of feature temporal evolution and makes the recognition decision prone to divergence; Third, in long-term continuous operation, the deviation trend of classification results gradually accumulates with the operation time of the carrier, and there is a lack of self-checking methods; Fourth, after the radiation power control command is issued, the residual between the actual response of the external control device and the command value is not involved in the loop correction, which makes the control gain coefficient of the recognition end prone to exceeding the limit due to long-term drift.

[0072] To address this, the above-described implementation method first introduces a dynamic brightness gain operator, enabling in-situ feedforward reconstruction of asymmetric and nonlinear topological geometric distortions between infrared spectral channels at the high-dimensional spectral tensor level without adding vibration damping hardware or compensation light sources, thus eliminating diffuse reflection halosing interference induced by pipeline mechanical vibration. Then, by generating control constraint smoothing coefficients, the discrete distribution information of the current frame's grayscale gradient is inversely mapped to the dynamic constraint boundary of matrix orthogonal decomposition, avoiding computational divergence caused by missing boundary conditions. Subsequently, by generating perturbation constraint weights, the backward degradation protection method of suspending the current frame and extrapolating from historical steady-state frames upon detecting a perturbation is replaced, ensuring the continuity of temporal features in the temporal computation flow under continuous high-frequency perturbations, improving feature recognition accuracy, and controlling the entire single-frame process time within a preset time threshold to match the real-time requirements of online detection. Then, by generating the result error deviation, it is easy to determine the gradual accumulation of deviation trends in the classification results over long-term continuous operation. This allows for the timely issuance of early warning signals, preventing system drift errors caused by trend deviations and achieving an effective self-checking method. Finally, by adjusting the pulse parameters and constraining the gain range of the control gain coefficient, the actual response of the external infrared wave energy modulation device (used to control the infrared wavelength of the multispectral camera) to the radiation power control command can be incorporated into the closed loop. The radiation power residual is used to correct the control gain coefficient at the feature classification and recognition end, stabilizing the coefficient within the preset gain range. This prevents the control gain coefficient from exceeding its limits due to long-term drift.

[0073] The above-described embodiments of this disclosure have the following beneficial effects: the target crop feature classification method based on photosensitive feature recognition of some embodiments of this disclosure can improve the accuracy of target crop classification results. Specifically, firstly, a spatiotemporal spectral tensor is constructed based on the target crop image sequence acquired by a multispectral camera. The target crop image is a discrete infrared spectral image within a preset wavelength range, taken at a fixed angle, of the target crop to be classified on a conveyor belt. The target crop is a single type of crop. In practice, the taste and nutritional value of crop types are reflected not only in the surface characteristics of the crops but also in the content of their internal components. Classifying crops based on surface characteristics (e.g., color, smoothness, etc.) can only select phenotypic qualified crops, making it difficult to distinguish the actual taste and nutritional value of the crops. This causes the crop classification results to deviate from practical significance. Therefore, this application considers the differences in the absorption characteristics of specific components within crop tissue to wave energy in the infrared band, and thus acquires discrete infrared spectral images of target crops (e.g., apples) to extract the internal component content characteristics of the target crops. Then, photosensitive features are extracted from the aforementioned spatiotemporal spectral tensor to obtain a photosensitive feature information sequence for the target crop. This photosensitive feature information includes a feature spectral vector and a dynamic brightness gain operator. Here, the generated feature spectral vector can be used to characterize the light absorption properties of the components within the target crop, facilitating the identification of component content. Next, based on the aforementioned photosensitive feature information sequence, roughness features are extracted from the aforementioned spatiotemporal spectral tensor to obtain a spatial primitive matrix and control constraint smoothing coefficients. The spatial primitive matrix characterizes the roughness features and spatial structural continuity of the target crop. In practice, factors influencing target crop classification also include the roughness and surface continuity of the target crop. Generating a spatial primitive matrix can introduce these roughness features and spatial structural continuity features, further improving the accuracy of target crop feature classification. Finally, based on the aforementioned control constraint smoothing coefficients, the aforementioned photosensitive feature information sequence, and the aforementioned spatial primitive matrix, edge features and intrinsic component distribution features of the target crop are generated. Finally, based on the aforementioned edge features and intrinsic component distribution characteristics of the target crop, feature classification of the target crop is performed to obtain the target crop classification results. These results include the target crop component density sequence and the target crop-level classification label. In summary, this method can classify target crops from multiple dimensions, improving the accuracy of the classification results.

[0074] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a target crop feature classification device based on light-sensing feature recognition. These device embodiments are similar to... Figure 1Corresponding to the method embodiments shown, this target crop feature classification device based on light-sensing feature recognition can be specifically applied to various electronic devices.

[0075] like Figure 4 As shown, a target crop feature classification device 400 based on photosensitive feature recognition in some embodiments includes: a construction unit 401, a photosensitive feature extraction unit 402, a roughness feature extraction unit 403, a generation unit 404, and a target crop feature classification unit 405. The construction unit 401 is configured to construct a spatiotemporal spectral tensor based on a sequence of target crop images acquired by a multispectral camera. The target crop images are discrete infrared spectral images within a preset wavelength range, taken at a fixed angle, of the target crop to be classified on a conveyor belt. The target crop is a single type of crop. The photosensitive feature extraction unit 402 is configured to extract photosensitive features from the spatiotemporal spectral tensor to obtain a sequence of target crop photosensitive feature information. The target crop photosensitive feature information includes a feature spectral vector and a dynamic brightness gain operator. The roughness feature extraction unit 403 is configured to extract features from the spatiotemporal spectral tensor based on the target crop photosensitive feature information sequence. Roughness features are extracted using the spectral tensor to obtain a spatial primitive matrix and control constraint smoothing coefficients. The spatial primitive matrix represents the roughness features and spatial structural continuity of the target crop. Generation unit 404 is configured to generate edge features and intrinsic component distribution features of the target crop based on the control constraint smoothing coefficients, the target crop photosensitive feature information sequence, and the spatial primitive matrix. Target crop feature classification unit 405 is configured to perform target crop feature classification based on the target crop edge features and intrinsic component distribution features to obtain target crop classification results. The target crop classification results include a target crop component density sequence and a target crop-level classification label.

[0076] It is understandable that the units described in the target crop feature classification device 400 based on light-sensing feature recognition are similar to those in the reference device. Figure 1 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the target crop feature classification device 400 based on light-sensing feature recognition and the units contained therein, and will not be repeated here.

[0077] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0078] like Figure 5As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory 502 or a program loaded from a storage device 508 into a random access memory 503. The random access memory 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, the read-only memory 502, and the random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0079] Typically, the following devices can be connected to the input / output interface 505: input devices 506 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0080] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a read-only memory 502. When the computer program is executed by the processing device 501, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0081] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0082] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0083] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: construct a spatiotemporal spectral tensor based on a sequence of target crop images acquired by a multispectral camera, wherein the target crop images are discrete infrared spectral images of a target crop to be classified on a conveyor belt, taken at a fixed angle within a preset wavelength range, and the target crop is a single type of crop; extract photosensitive features from the aforementioned spatiotemporal spectral tensor to obtain a sequence of target crop photosensitive feature information, wherein the target crop photosensitive feature information includes: a feature spectral vector and a dynamic brightness gain operator; and, based on the aforementioned target… The crop photosensitive feature information sequence is used to extract roughness features from the aforementioned spatiotemporal spectral tensor, resulting in a spatial primitive matrix and control constraint smoothing coefficients. The spatial primitive matrix characterizes the roughness features and spatial structural continuity of the target crop. Based on the control constraint smoothing coefficients, the target crop photosensitive feature information sequence, and the spatial primitive matrix, edge features and intrinsic component distribution features of the target crop are generated. Based on the edge features and intrinsic component distribution features, the target crop is classified to obtain the target crop classification results, which include the target crop component density sequence and the target crop level classification label.

[0084] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0086] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0087] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A target crop feature classification method based on light-sensing feature recognition, characterized in that, include: Based on the target crop image sequence acquired by the multispectral camera, a spatiotemporal spectral tensor is constructed. The target crop image is a discrete infrared spectral image within a preset wavelength range taken at a fixed angle for the target crop to be classified on the conveyor belt. The target crop is a single type of crop. The light-sensing features of the spatiotemporal spectral tensor are extracted to obtain a sequence of light-sensing feature information of the target crop, wherein the light-sensing feature information of the target crop includes: a feature spectral vector and a dynamic brightness gain operator; Based on the target crop photosensitive feature information sequence, roughness features are extracted from the spatiotemporal spectral tensor to obtain a spatial primitive matrix and control constraint smoothing coefficients. The spatial primitive matrix characterizes the roughness features and spatial structural continuity characteristics of the target crop. Based on the control constraint smoothing coefficient, the target crop photosensitive feature information sequence, and the spatial primitive matrix, generate the target crop edge features and intrinsic component distribution features; The target crop is classified based on its edge features and intrinsic component distribution features to obtain a target crop classification result, which includes a target crop component density sequence and a target crop level classification label.

2. The target crop feature classification method based on photosensitive feature recognition according to claim 1, characterized in that, The method further includes: Based on the target crop classification results and the historical classification result sequence, a result error deviation is generated; In response to the result error deviation being greater than a preset error threshold, a classification warning signal is generated and sent to the target terminal, wherein the classification warning signal indicates that there is a system drift error in the current classification result sequence.

3. The target crop feature classification method based on photosensitive feature recognition according to claim 1, characterized in that, The step of extracting photosensitive features from the spatiotemporal spectral tensor to obtain a sequence of photosensitive feature information of the target crop includes: The average gray value in the corresponding preset background area on the conveyor belt in the spatiotemporal spectral tensor is determined as the environmental reference brightness factor. For each coordinate point of the spatiotemporal spectral tensor, the following first processing step is performed: Each gray value in the spatiotemporal spectral tensor corresponding to the coordinate point is determined as a continuous spectral gray value, thus obtaining a continuous spectral gray value sequence. Determine the gray-level response slope sequence corresponding to the gray-level value sequence of the continuous spectral line; Based on the grayscale response slope sequence, construct the feature spectral vector of the target crop photosensitive feature information corresponding to the coordinate point in the target crop photosensitive feature information sequence; The mean gray value of at least one gray value within a preset window corresponding to the coordinate point in the spatiotemporal spectral tensor is determined as the current mean gray value; The ratio of the current grayscale mean to the environmental reference brightness factor is determined as the dynamic brightness gain operator in the target crop light perception feature information corresponding to the coordinate point in the target crop light perception feature information sequence.

4. The target crop feature classification method based on photosensitive feature recognition according to claim 3, characterized in that, The step of extracting roughness features from the spatiotemporal spectral tensor based on the target crop photosensitive feature information sequence to obtain a spatial primitive matrix and control constraint smoothing coefficients includes: Determine the gray-level gradient features corresponding to each coordinate point of the spatiotemporal spectral tensor; For each coordinate point of the spatiotemporal spectral tensor, perform the following second processing step: The gray values ​​within the neighborhood window corresponding to the coordinate points are selected from the spatiotemporal spectral tensor and used as the spatial neighborhood feature matrix. Based on the dynamic brightness gain operator included in the target crop light-sensing feature information corresponding to the coordinate point in the target crop light-sensing feature information sequence, the spatial neighborhood feature matrix is ​​scaled to generate a processed neighborhood matrix. Based on the processed neighborhood matrix, a local gray-level co-occurrence matrix is ​​generated, wherein the local gray-level co-occurrence matrix represents the spatial gray-level co-occurrence topological relationship between each coordinate point within the neighborhood window. Generate crop surface feature vectors based on the local gray-level co-occurrence matrix; Based on the generated sequence of crop surface feature vectors, construct a spatial primitive matrix; Based on the gray-level gradient features corresponding to the processed neighborhood matrix in the obtained processed neighborhood matrix sequence, the discrete variance of the gray-level gradient is generated. Based on the gray-level gradient discrete variance and the baseline discrete variance, control constraint smoothing coefficients are generated.

5. The target crop feature classification method based on photosensitive feature recognition according to claim 4, characterized in that, The step of generating edge features and intrinsic component distribution features of the target crop based on the control constraint smoothing coefficient, the target crop photosensitive feature information sequence, and the spatial primitive matrix includes: The feature spectral vectors and the spatial primitive matrix included in each target crop photosensitive feature information sequence are fused to obtain a fused feature matrix. In response to the control constraint smoothing coefficient being less than or equal to a preset safety threshold, singular value decomposition is performed on the fused feature matrix according to a preset penalty weight to obtain the target crop edge features and intrinsic component distribution features; In response to the control constraint smoothing coefficient being greater than the preset safety threshold, the preset penalty weight is adjusted according to the control constraint smoothing coefficient to generate disturbance constraint weights; Based on the perturbation constraint weights, singular value decomposition is performed on the fused feature matrix to obtain the target crop edge features and intrinsic component distribution features.

6. The target crop feature classification method based on photosensitive feature recognition according to claim 5, characterized in that, The step of classifying the target crop features based on the marginal features and intrinsic component distribution features to obtain the target crop classification result includes: The edge features of the target crop and the distribution features of the intrinsic components are encoded using a feature encoder to generate a shared feature vector; The shared feature vector is used to identify the target crop components, and the target crop classification result includes the target crop component density sequence. The shared feature vector is used for target-based category identification to obtain the target crop classification results, including the target crop item-level classification label.

7. The target crop feature classification method based on photosensitive feature recognition according to claim 6, characterized in that, The method further includes: Based on the target crop classification results, generate the target radiative power; Based on the target radiation power, the infrared radiation power of the multispectral camera, and the control gain coefficient corresponding to the infrared radiation power, a pulse width modulation command is generated. According to the pulse width modulation command, the pulse parameters of the multispectral camera are adjusted, and after the pulse parameters are adjusted, the real-time radiation power is obtained. The difference between the real-time radiated power and the target radiated power is determined as the radiated power residual. The gain range is constrained for the control gain coefficient based on the radiated power residual.

8. A target crop feature classification device based on photosensitive feature recognition, characterized in that, include: The construction unit is configured to construct a spatiotemporal spectral tensor based on the target crop image sequence acquired by the multispectral camera. The target crop image is a discrete infrared spectral image within a preset wavelength range taken at a fixed angle for the target crop to be classified on the conveyor belt. The target crop is a single type of crop. The photosensitive feature extraction unit is configured to extract photosensitive features from the spatiotemporal spectral tensor to obtain a sequence of photosensitive feature information of the target crop, wherein the photosensitive feature information of the target crop includes: a feature spectral vector and a dynamic brightness gain operator; The roughness feature extraction unit is configured to extract roughness features from the spatiotemporal spectral tensor based on the target crop photosensitive feature information sequence, to obtain a spatial primitive matrix and a control constraint smoothing coefficient, wherein the spatial primitive matrix characterizes the roughness features and spatial structural continuity characteristics of the target crop. The generation unit is configured to generate edge features and intrinsic component distribution features of the target crop based on the control constraint smoothing coefficient, the target crop photosensitive feature information sequence, and the spatial primitive matrix. The target crop feature classification unit is configured to classify the target crop features based on the target crop edge features and the intrinsic component distribution features to obtain the target crop classification result, wherein the target crop classification result includes the target crop component density sequence and the target crop level classification label.

9. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images