Dendrobium defect lightweight detection method based on FPGA and reinforcement learning

Abstract

The present application discloses a Dendrobium defect lightweight detection method based on FPGA and reinforcement learning, relates to the technical field of machine learning, and comprises the following steps: collecting original images in the production process of Dendrobium through an industrial image acquisition device, performing parallel preprocessing operations on the original images based on FPGA hardware to obtain preprocessed Dendrobium images; inputting the preprocessed Dendrobium images into a preset lightweight unsupervised reinforcement learning model; performing deep feature learning on the preprocessed Dendrobium images to output standardized Dendrobium feature maps; exploring abnormal features of the Dendrobium feature maps to adaptively generate double-fused internal reward signals representing the abnormal state of the Dendrobium image defects; outputting fine detection action instructions; associating the detection action instructions with the internal reward signals to complete intelligent discrimination and pixel-level accurate positioning of the Dendrobium defects and output final Dendrobium defect detection results; and improving the distortionless precision and spatial resolution of free-form surface three-dimensional imaging.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a lightweight method for detecting defects in Dendrobium officinale based on FPGA and reinforcement learning. Background Technology

[0002] Dendrobium, a precious Chinese medicinal herb used in both food and medicine, relies heavily on appearance defect detection during its industrialized deep processing. This is a core aspect of quality grading and control, directly determining the commercial value and safety of Dendrobium products. Common defects in Dendrobium include mold, insect infestation, fine cracks, breakage, and deformities. Traditional industrial production primarily relies on manual visual sorting, depending on operator experience to judge indicators such as size, color, and integrity. However, manual inspection is highly subjective, lacks standardization, and is easily affected by factors such as operator fatigue, changes in ambient lighting, and differences in individual experience. This results in high false negative rates, frequent false positives, and low efficiency, making it unsuitable for the demands of large-scale, streamlined, and standardized industrial production. Current machine vision defect detection technologies mainly rely on supervised learning models, using deep learning models such as convolutional neural networks, YOLO series, and U-Net to identify and locate defects in agricultural products. There are also detection schemes that combine multimodal sensing fusion with supervised learning, using multi-source data such as visible light, hyperspectral imaging, and X-rays to improve defect detection capabilities.

[0003] However, existing supervised learning schemes have many problems in the Dendrobium defect detection scenario: First, the morphology of minor defects such as microcracks and punctate mold spots in Dendrobium is messy and the boundaries are blurred, making manual annotation extremely difficult and costly. Moreover, the number of defect samples is scarce, making it difficult to build a large-scale, high-precision labeled dataset, which leads to insufficient training of supervised learning models. Second, there are many varieties of Dendrobium, and the lighting and background environment in industrial production scenarios are dynamically changing. Supervised learning models have poor generalization ability and are prone to missing new defects and insufficient scenario adaptability. Third, existing deep learning models have complex structures, a large number of parameters, and high inference computation costs. The software inference latency is also high, making it difficult to adapt to the high-speed, low-power, and real-time detection deployment requirements of industrial production lines, and thus unable to achieve efficient, accurate, and full-category industrial detection of Dendrobium defects. Summary of the Invention

[0004] To address the aforementioned technical issues, a lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning is provided. This technical solution solves the problems mentioned above.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning includes: The raw images of Dendrobium production process are acquired by an industrial image acquisition device, and the raw images are preprocessed by parallel preprocessing based on FPGA hardware to obtain preprocessed Dendrobium images. The preprocessed Dendrobium images are input into a preset lightweight unsupervised reinforcement learning model; Deep feature learning is performed on preprocessed Dendrobium images to extract inherent features of Dendrobium texture, color, and morphology, and output standardized Dendrobium feature maps. Anomaly features are explored in the feature map of Dendrobium, and an intrinsic reward signal with dual fusion is adaptively generated to characterize the defect and abnormal state of Dendrobium image; By combining the Dendrobium feature map with the intrinsic reward signal, the system autonomously explores suspicious defect areas in the image and outputs refined detection action commands. By associating detection action commands with intrinsic reward signals, the system achieves intelligent identification and pixel-level precise positioning of Dendrobium defects, and outputs the final Dendrobium defect detection results.

[0006] Preferably, the parallel preprocessing operation on the original image based on FPGA hardware includes: The original Dendrobium images were subjected to 3×3 window mid-range filtering to remove salt-and-pepper noise and transmission noise. The RGB color image is converted into a single-channel grayscale image by using a grayscale weighted mapping formula, thus completing the grayscale normalization process. Dendrobium outline boundaries are identified based on row and column grayscale projection curves, and the region of interest is adaptively cropped to remove invalid background areas. The image size is unified by bilinear interpolation algorithm and normalized to a preset standard input size image, which is suitable for the model and FPGA parallel inference architecture.

[0007] Preferably, the output of the standardized Dendrobium feature map includes: Receive preprocessed standard Dendrobium images and convert the image data into a four-dimensional tensor; By removing redundant convolutional layers and the four-level deep separable convolutional structure of the SE attention module, the shallow texture features and deep morphological features of Dendrobium are extracted step by step. Based on the self-supervised SimCLR contrastive learning architecture, the pre-training was completed using a large number of unlabeled normal Dendrobium images to learn the distribution of inherent features of normal Dendrobium. The final generated Dendrobium feature map is output in two equally divided paths to achieve feature reuse.

[0008] Preferably, the step of exploring abnormal features in the Dendrobium feature map and adaptively generating a dual-fusion intrinsic reward signal includes: Dendrobium feature maps are processed in parallel by a fixed target network and a learnable prediction network. The Euclidean distance between the output features of the two networks is calculated to generate a normalized RND novelty reward. A lightweight autoencoder is used to perform encoding compression and feature reconstruction on the feature map of Dendrobium. The mean square error of the original features and the reconstructed features is calculated to generate a normalized reconstruction error reward. Extract the feature dispersion of the current feature map, and dynamically adjust the weighted fusion weight of the RND novelty reward and the reconstruction error reward based on the feature dispersion; By merging the dual-path rewards through a weighted summation method, a final adaptive intrinsic reward signal is generated and output separately for defect detection strategy optimization and defect status discrimination.

[0009] Preferred structural configurations of a fixed-target network, a learnable predictive network, and a lightweight autoencoder include: The fixed target network is a 3-layer fully connected network with fixed weight parameters after initialization. The learnable prediction network adopts the same 3-layer fully connected network with parameters updated iteratively in real time. The lightweight autoencoder is symmetrically composed of two fully connected encoding layers and two fully connected decoding layers, retaining only the core feature reconstruction operation to reduce computational overhead; Both the novelty reward and the reconstruction error reward of the generated RND are processed by Min-Max normalization and mapped to a unified interval to ensure the accuracy of reward fusion.

[0010] Preferably, the step of combining the Dendrobium feature map with the intrinsic reward signal to autonomously explore suspicious defect areas in the image and output refined detection action instructions includes: The feature map of Dendrobium is evenly divided into multiple feature blocks, and each feature block is used as an independent detection state to achieve fine detection of local areas of the image. By inputting state features into a lightweight strategy network, it outputs the probability distribution of four types of detection actions: focusing, zooming, moving, and judgment. A lightweight value network that shares input features with the policy network outputs a long-term revenue estimate of the current detection state. Based on the improved pruning-type PPO algorithm, the advantage function and policy gradient are calculated by combining the intrinsic reward signal, and the parameters of the policy network and value network are iteratively updated. Receive defect detection feedback results, dynamically calibrate the strategy exploration rate and reward fusion weight, and achieve end-to-end adaptive optimization.

[0011] Preferably, the iterative update of the policy network and value network parameters includes: By using multi-layer feature mapping and Softmax activation to output a standardized action probability distribution, the action with the highest probability is selected as the optimal detection action. By using linear activation to output state reward values ​​and combining them with time-series discount factors to calculate state advantages, policy gradient optimization can be assisted. A gradient pruning mechanism is introduced during training to limit the range of policy gradient fluctuations, and the SGD optimizer is used to complete the stable iteration of network parameters.

[0012] Preferably, the associated detection action command and the intrinsic reward signal complete the intelligent identification and pixel-level precise positioning of Dendrobium defects, including: By integrating the intrinsic reward signal and the probability of detection action, the defect confidence is calculated for each feature block, and the defect region is determined by combining the dynamic adaptive threshold. Based on the detection action trajectory of focusing and magnification, a mapping relationship between feature block coordinates and original image coordinates is established, and the defect bounding box and defect area are output. Identify and classify five types of Dendrobium defects: mold, insect infestation, cracks, damage, and deformity, and output detection results including defect type, location, and confidence level; The detection results are fed back into the reinforcement learning iteration process to achieve online adaptive iterative optimization of the model.

[0013] Preferably, it also includes a lightweight model deployment step: The convolution and fully connected operations of the model are decomposed into logical units and quantized and compressed to reduce the number of model parameters and computational cost. Adapted to FPGA hardware parallel pipeline architecture, it accelerates the entire process of image preprocessing, feature extraction, reward calculation, policy reasoning, and defect detection in parallel. By optimizing hardware computing power consumption through 16-bit fixed-point quantization, low-latency and low-power industrial real-time inference deployment can be achieved.

[0014] Preferably, it also includes a model training step: Unlabeled normal Dendrobium images were divided into training, validation and test sets according to proportions, and model pre-training and iterative training were completed using only normal samples. During the pre-training phase, the feature encoding parameters are initialized through self-supervised contrastive learning, and the general features of normal Dendrobium are learned. The core training phase relies on the inherent reward closed-loop optimization mechanism to iteratively update network parameters and monitor the model convergence state through the validation set to suppress overfitting. The convergence verification phase determines model convergence based on the stability of the reward distribution and the sample misclassification rate, and the defect detection performance of the test set is quantified.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention adopts an unsupervised reinforcement learning architecture, which can complete model pre-training and iterative optimization based solely on unlabeled normal Dendrobium images. It eliminates the need to collect and label a large number of Dendrobium defect samples, significantly reducing the cost of dataset construction and manual workload. It solves the problems of scarce Dendrobium defect samples, difficulty in labeling subtle defects, and large labeling errors, filling the application gap of unsupervised reinforcement learning in the field of intelligent detection of defects in Dendrobium medicinal materials. 2. This invention integrates a dual discrimination mechanism of RND novelty reward and autoencoder reconstruction error reward, and dynamically and adaptively adjusts the fusion weights based on feature dispersion. It takes into account the model's ability to explore unknown abnormal defects and its ability to identify minute defects such as fine cracks and dotted mold spots. This reduces the problems of insufficient sensitivity and high defect false negative rate of traditional single discrimination mechanisms, and achieves accurate identification and pixel-level localization of various explicit, implicit and unknown defects of Dendrobium, thereby improving the comprehensiveness and accuracy of defect detection. 3. This invention constructs a feature extraction structure based on an improved lightweight MobileNetV3, removes redundant network parameters and invalid modules, and combines it with a lightweight policy network and value network to significantly reduce the number of model parameters and computational overhead. At the same time, it introduces a PPO policy iterative optimization and online adaptive calibration mechanism, which can dynamically adapt to complex industrial scenarios such as different Dendrobium species, light changes, and environmental interference, effectively improving the model's generalization ability and scenario adaptability, and solving the problems of poor robustness and susceptibility to environmental interference in traditional models. 4. This invention performs lightweight optimization on the entire model process. By decomposing logic units and compressing the model's computing power overhead with 16-bit fixed-point quantization, it adapts to the FPGA parallel pipeline architecture, realizing parallel acceleration of the entire process of image preprocessing, feature extraction, reward calculation, policy inference, and defect discrimination. This solves the shortcomings of traditional deep learning models, such as high inference latency, high hardware power consumption, and inability to adapt to high-speed pipeline detection, thus meeting the industrial, large-scale, and real-time production and testing needs of Dendrobium. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of the present invention. Detailed Implementation

[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0018] Example 1: Refer to Figure 1 As shown, a lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning includes: Step 1: Acquire raw images of Dendrobium production process using an industrial image acquisition device, and perform parallel preprocessing operations on the raw images based on FPGA hardware to obtain preprocessed Dendrobium images; Step 2: Input the preprocessed Dendrobium images into a preset lightweight unsupervised reinforcement learning model; Step 3: Perform deep feature learning on the preprocessed Dendrobium images to extract the inherent features of Dendrobium texture, color, and morphology, and output standardized Dendrobium feature maps; Step 4: Explore abnormal features in the Dendrobium feature map and adaptively generate a dual-fusion intrinsic reward signal to characterize the defect and abnormal state of the Dendrobium image; Step 5: Combining the Dendrobium feature map with the intrinsic reward signal, autonomously explore suspicious defect areas in the image and output refined detection action instructions; Step 6: Associate the detection action commands with the inherent reward signals to complete the intelligent identification and pixel-level precise positioning of Dendrobium defects, and output the final Dendrobium defect detection results.

[0019] Through the above scheme, this invention relies on the parallel acceleration capability of FPGA hardware to complete the image preprocessing, solving the problem of high latency in CPU serial processing; it uses a lightweight unsupervised reinforcement learning model to complete feature self-learning based on pure normal samples, without the need for a large amount of defect annotation data; it combines a dual intrinsic reward adaptive fusion mechanism to enhance the perception of minor and latent defects, and optimizes the defect region search strategy through reinforcement learning through autonomous iteration, ultimately achieving high-precision, pixel-level real-time detection of multiple types of defects in Dendrobium, such as mold, insect infestation, cracks, damage, and deformities.

[0020] Through the above-described scheme, this invention improves upon the problems of limited field of view, severe imaging distortion, and low spatial degrees of freedom in fixed-surface optical imaging. It achieves accurate digital representation of the geometric information of the target scene through three-dimensional voxel discretization, and fully characterizes the high-dimensional transmission law of the light field in the freeform optical system based on the four-dimensional ray transmission tensor. By eliminating redundant information of the optical path through tensor dimensionality reduction decomposition and retaining the core optical modulation features, combined with adaptive rank preservation strategy, multi-wavelength optical correction, frequency domain filtering correction, spatiotemporal joint modulation, and hardware deviation compensation mechanism, it achieves high-precision and highly adaptable modulation drive of the light field signal. Finally, it accurately converges in the target three-dimensional space to form distortion-free and highly uniform spatial three-dimensional pixels, effectively improving the three-dimensional imaging accuracy, spatial resolution, and scene adaptability of the freeform optical system.

[0021] In some embodiments, based on the depth geometry of the target spatial scene, the depth geometry is discretized into a set of three-dimensional voxels containing direction vectors according to a specified resolution threshold. Color and brightness parameters corresponding to the three-dimensional voxel set at each discrete viewpoint are extracted to form optical emission parameters. A three-dimensional physical coordinate sequence is then constructed based on the discretized three-dimensional voxel set, specifically including: The depth geometry model of the target space scene is obtained, and a spatial resolution threshold ΔS is set. The continuous three-dimensional space is uniformly divided along the three orthogonal directions X, Y, and Z to complete the three-dimensional voxel discretization. For each independent voxel, the spatial ray direction vector is calculated to construct a three-dimensional voxel set containing position and direction information. All discrete viewpoints are traversed, and the RGB color three-channel parameters and brightness grayscale parameters corresponding to each voxel are collected. After normalization, standardized optical emission parameters are formed. The three-dimensional spatial coordinates of all effective voxels are summarized and arranged in an orderly manner to construct a complete three-dimensional physical coordinate sequence.

[0022] Specifically, this embodiment uses a uniform voxel discretization method to digitally sample the continuous depth geometry. The spatial resolution threshold ΔS is set as the side length of a single voxel in mm. According to the number of segments along the X, Y, and Z axes, several cubic three-dimensional voxels are uniformly divided. The center coordinates of each voxel are obtained by multiplying the axial segmentation number by the resolution threshold. This determines the spatial position coordinates of all voxels and completes the discretization modeling of the continuous space.

[0023] For each discrete voxel, the starting point of the light source and the target point of light propagation within the voxel are selected. By solving the spatial vector between the two points and normalizing it, the unit direction vector of light propagation corresponding to the voxel is obtained, which accurately represents the light transmission posture of a single voxel.

[0024] Furthermore, the RGB color and brightness parameters of each voxel at each discrete viewpoint are collected. To eliminate differences in numerical dimensions and deviations in value range, maximum and minimum value normalization processing is performed on all original color and brightness parameters, uniformly mapping all parameters to the standard range of 0~1, forming standardized and uniformly calculable optical emission parameters. Finally, the three-dimensional coordinates of all effective voxels are arranged in an orderly manner according to their spatial positions to generate a continuous and complete three-dimensional physical coordinate sequence, providing a precise coordinate reference for subsequent image space definition and optical path mapping calculation, solving the problems of missing three-dimensional coordinate definitions and ambiguous optical path mapping references in the original technology.

[0025] In some embodiments, the discrete pixel array of the original image plane is obtained by deflecting the optical path transmission matrix to the target image space through a freeform surface. The optical path transmission matrix is ​​then arranged according to four dimensions: the x-coordinate of the light source, the y-coordinate of the light source, the x-coordinate of the target space, and the y-coordinate of the target space, to obtain an initial four-dimensional ray transmission tensor, including: Establish the optical path mapping relationship between the original image plane pixel coordinate system and the target image space three-dimensional coordinate system, solve the freeform surface light deflection and transmission matrix, and expand the two-dimensional matrix data into a four-dimensional tensor structure, corresponding to the four dimensions of light source x-coordinate U, light source y-coordinate V, target space x-coordinate X, and target space y-coordinate Y, respectively.

[0026] Specifically, the light deflection on the freeform surface follows the laws of reflection and refraction in geometric optics. By combining the personalized surface shape equation of the freeform surface, local curvature parameters, and the incident angle of the light, the optical path deflection transmission matrix corresponding to a single pixel is obtained. The coefficients in the matrix are the optical path mapping weights, used to quantify the energy distribution and position mapping relationship of the light emitted from a single pixel in the original image plane after being modulated by the freeform surface and projected to different positions in the target image space.

[0027] By traversing all discrete pixels in the original image plane, the two-dimensional optical path transmission matrix corresponding to each pixel is stacked and integrated in a hierarchical manner according to the four dimensions of the light source horizontal and vertical coordinates and the target space horizontal and vertical coordinates to construct a complete four-dimensional ray transmission tensor. This tensor can completely cover the optical path transmission mapping relationship of all pixels, accurately characterize the high-dimensional transmission characteristics of freeform surface light fields, and provide complete data support for subsequent tensor decomposition and light field modulation.

[0028] In some embodiments, the invocation of the tensor decomposition operator to reduce the dimensionality of the four-dimensional ray transmission tensor to generate a spatial modulation feature basis matrix and a core tensor includes: An expansion matrix is ​​constructed along multiple modal directions of the four-dimensional ray transmission tensor; a covariance matrix is ​​obtained for each expansion matrix; the eigenvector arrays of each covariance matrix are extracted as factor matrices; the factor matrices are subjected to principal component retention according to the preset singular value truncation rank to generate spatial modulation feature basis matrix and core tensor.

[0029] Specifically, the four-dimensional ray transmission tensor is modally expanded along the four modal directions of the light source x-coordinate, light source y-coordinate, target space x-coordinate, and target space y-coordinate, respectively. This reduces the dimensionality of the high-dimensional tensor into four corresponding two-dimensional expansion matrices, thereby decoupling the high-dimensional data and facilitating subsequent feature statistics and calculations.

[0030] For each modal expansion matrix, the overall mean of the matrix is ​​first calculated and then the mean is normalized to zero. Then, the covariance matrix corresponding to each mode is calculated using the covariance calculation formula. This matrix is ​​used to characterize the correlation and discrete distribution characteristics of the optical path data in each dimension. The core calculation formula is as follows:

[0031] in: For the first Modal covariance matrix; For the first Modal two-dimensional expansion matrix; The expanded matrix is ​​the mean matrix; This represents the number of matrix samples.

[0032] Eigenvalue decomposition is performed on the covariance matrix of each mode to obtain the corresponding eigenvector matrix and eigenvalue diagonal matrix. The eigenvector matrix serves as the mode factor matrix, representing the independent optical path modulation features of each dimension. The eigenvalues ​​are used to characterize the contribution weight of the corresponding features. Finally, based on the preset singular value truncation rank, high-order and high-contribution principal component eigenvectors in the factor matrix are selected and retained, while redundant low-order features are removed. This process generates the spatial modulation feature basis matrix of each dimension and reconstructs a simplified core tensor, which significantly reduces the computational redundancy of high-dimensional tensors while retaining the core optical path modulation features.

[0033] In some embodiments, the principal component retention of the factor matrix based on a preset singular value truncation rank includes: Generate the surface curvature gradient of the freeform surface in multiple defined physical coordinate regions; establish a retention strategy based on the surface curvature gradient, increase the retention value of the singular value truncated rank in the physical mapping region where the surface curvature gradient exceeds a specific threshold, and decrease the retention value of the singular value truncated rank in the physical mapping region where the surface curvature gradient does not exceed a specific threshold.

[0034] Specifically, for the Gaussian curvature of each physical region of the freeform surface, the partial derivatives of curvature in the horizontal and vertical directions are solved respectively, and the local curvature gradient is calculated by summing the squares and taking the square root, which is used to quantitatively characterize the degree of local deformation of the surface.

[0035] A pre-defined curvature gradient determination threshold is used to implement an adaptive rank preservation strategy: when the curvature gradient of a region is greater than the threshold, it indicates that the surface deformation in that region is severe and the optical path distortion is complex. In this case, the singular value truncation rank needs to be increased to retain more high-order features to accurately fit the complex optical path distortion. When the curvature gradient of a region is less than or equal to the threshold, it indicates that the surface shape is gentle and the optical path transmission is stable. In this case, the truncation rank can be appropriately reduced to eliminate redundant features and reduce the amount of computation. This strategy fully covers all boundary scenarios where the threshold is equal to, greater than, or less than the threshold, taking into account both the imaging accuracy of complex regions and the inference efficiency of gentle regions.

[0036] In some embodiments, the four-dimensional ray transport tensor includes multiple wavelength channels; obtaining the initial four-dimensional ray transport tensor includes: Obtain the refractive index parameters for the specified red, green, and blue bands; generate wavelength transmission matrices for red, green, and blue light as they pass through the freeform surface based on the refractive index parameters; combine multiple wavelength transmission matrices hierarchically to construct a four-dimensional light transmission tensor with multiple wavelength dimensions.

[0037] Specifically, based on the dispersion characteristics of the optical medium and the correspondence between the basic refractive index, dispersion coefficient, and light wavelength, the center wavelengths of red, green, and blue light are substituted to calculate the refractive index of the medium corresponding to the three visible light channels. Based on the refractive index differences of different wavelengths and combined with the deflection law of freeform surface light paths, the independent optical path transmission matrices of the RGB three bands are solved. Finally, the three single-wavelength four-dimensional transmission matrices are stacked and fused in a hierarchical manner to construct a multi-wavelength four-dimensional light transmission tensor, realizing accurate modeling of the multi-channel optical path of the color light field and effectively eliminating the imaging color difference problem caused by medium dispersion.

[0038] In some embodiments, generating the modulation drive stream based on the optical field source data and the spatial modulation feature basis matrix includes: The light field source data is transformed into a source data matrix in tensor form; a feature fusion matrix is ​​generated based on the core tensor and the source data matrix; a multimodal tensor is obtained based on the feature fusion matrix and the spatial modulation feature basis matrices of each dimension; the multimodal tensor is expanded row by row and serialized and recombined according to the pixel arrangement order of the light field projection array to generate a modulation driving flow.

[0039] Specifically, the light field source data in two-dimensional image format is reconstructed into a three-dimensional source data tensor containing channel, height, and width dimensions to meet the requirements of high-dimensional matrix operations; the standardized source data tensor and the dimensionality-reduced core tensor are multiplied and fused element-wise to obtain a feature fusion matrix that fuses the original light field information and the core features of optical path transmission.

[0040] The feature fusion matrix is ​​modally weighted and fused with the spatial modulation feature basis matrices of each dimension to integrate multi-dimensional optical modulation features and obtain a complete multi-modal tensor. Finally, the multi-modal tensor is expanded row by row to convert the high-dimensional tensor data into a one-dimensional sequence. The sequence is then reassembled in strict accordance with the physical pixel arrangement order of the light field projection array to generate a modulation drive stream that can directly drive the hardware, thus realizing the accurate mapping of light field image data to hardware electrical signal drive sequence.

[0041] In some embodiments, the light field projection array is mounted on a mechanical rotating platform; the generation of the modulation drive flow based on the light field source data and the spatial modulation feature basis matrix further includes: In the process of generating the modulation drive stream, a one-dimensional time modulation sequence is introduced; based on the refresh frequency parameters of the mechanical rotating platform, the modulation drive stream is divided into multiple sub-field drive sequences according to the time dimension; the output timing of each sub-field drive sequence is controlled according to the one-dimensional time modulation sequence.

[0042] Specifically, the duration of a single-frame complete imaging cycle is calculated based on the real-time refresh frequency of the mechanical rotating platform. A one-dimensional time modulation sequence is constructed based on the imaging cycle, and the complete cycle is evenly divided into multiple time intervals, ensuring that the sum of the durations of all intervals is consistent with the single-frame imaging cycle. According to the divided time nodes, the complete modulation drive flow is divided into a corresponding number of sub-field drive sequences. The output time and duration of each sub-field sequence are precisely controlled by the time modulation sequence, achieving precise spatiotemporal synchronization between the mechanical scanning motion and the light field modulation output, and completely eliminating imaging ghosting and image misalignment problems caused by rotational motion.

[0043] In some embodiments, generating a modulation drive stream based on optical field source data and a spatial modulation feature basis matrix further includes: Acquire relative spatial offset data collected by pose sensors arranged on the light field projection array; convert the relative spatial offset data into an affine transformation matrix; and correct the displacement deviation of the pixel coordinate mapping value of the modulation drive flow based on the affine transformation matrix.

[0044] Specifically, the horizontal, vertical, and rotational angle deviations of the light field projection array are collected in real time by a pose sensor. A three-dimensional affine transformation matrix is ​​constructed based on these spatial offset parameters, which can simultaneously adapt to both translational and rotational spatial deviation corrections. Matrix operations are performed between the original pixel mapping coordinates in the modulation drive stream and the affine transformation matrix to obtain the corrected, accurate pixel coordinates. This provides real-time compensation for imaging deviations caused by equipment installation errors, operational vibrations, and positional shifts, significantly improving the imaging stability of the system during long-term operation.

[0045] In some embodiments, generating the modulation drive flow based on the optical field source data and the spatial modulation feature basis matrix further includes: Obtain the optical energy distribution map corresponding to each surface region of the freeform surface; generate an energy adjustment matrix based on the optical energy distribution map; adjust the voltage driving signal of the pixel channel in the low-efficiency mapping region in the modulation drive flow based on the energy adjustment matrix.

[0046] Specifically, optical detection equipment is used to collect light transmission and reflection efficiencies in all regions of the freeform surface to obtain complete optical energy distribution data. The average optical efficiency of the entire surface is calculated, and a pixel-level energy adjustment matrix is ​​constructed based on the global average efficiency and the actual optical efficiency of each local region. For low-loss regions where the optical efficiency is lower than the global average, the voltage drive amplitude of the corresponding pixel channel is increased through weighted calculation of the energy matrix to compensate for light energy attenuation. For high-efficiency regions, the original drive signal is maintained, ultimately achieving uniform light energy output across the entire field of view and solving the problem of uneven brightness in the image.

[0047] In some embodiments, the process of generating the feature fusion matrix based on the core tensor and the source data matrix further includes: Obtain the point spread function matrix of the freeform surface optical system at each depth plane in the target image space; perform a two-dimensional Fourier transform on the point spread function matrix to generate the optical transfer function frequency domain matrix; perform inverse frequency domain filtering based on the core tensor and the optical transfer function frequency domain matrix to generate a filtered correction tensor; use the filtered correction tensor to replace the original core tensor and the source data matrix to generate a feature fusion matrix.

[0048] Specifically, point spread functions at different depth planes in the target image space are collected to characterize the spatial domain imaging blurring characteristics of the optical system. A two-dimensional Fourier transform is used to convert the spatial domain point spread function into the frequency domain optical transfer function, quantifying the imaging attenuation characteristics of different frequency components of the system. To compensate for diffraction and blurring distortion in the optical system, an inverse frequency domain filtering method is used to correct the core tensor. The core correction calculation formula is as follows:

[0049] in: For filtering correction tensors; For the original core tensor; The frequency domain matrix of the optical transfer function; It is a very small constant used to avoid the denominator being zero and to ensure the stability of the operation.

[0050] Replacing the original core tensor with the corrected filter correction tensor in subsequent feature fusion calculations can effectively compensate for high-frequency detail loss in the optical system, correct imaging blur and distortion defects, and significantly improve the clarity and detail reproduction of 3D imaging.

[0051] In some embodiments, the modulation light field is output by the light field projection array according to the modulation driving flow. The modulation light field is reflected or transmitted through a freeform surface and converges at a specified physical spatial location to form a spatial three-dimensional pixel.

[0052] Specifically, each pixel in the light field projection array outputs a modulated beam of light corresponding to its brightness and color based on the precise voltage driving signal of the modulation driving flow. After the beam undergoes high-precision deflection and optical modulation via a freeform surface, it converges precisely at the corresponding coordinate position in the target's three-dimensional image space, strictly following the optical path mapping relationship of the four-dimensional ray transmission tensor. This forms a discrete, uniform, precisely positioned, and color-accurate spatial three-dimensional pixel array, ultimately achieving high-quality, distortion-free freeform surface three-dimensional optical imaging.

[0053] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning, characterized in that, include: The raw images of Dendrobium production process are acquired by an industrial image acquisition device, and the raw images are preprocessed by parallel preprocessing operation based on FPGA hardware to obtain preprocessed Dendrobium images. The preprocessed Dendrobium images are input into a preset lightweight unsupervised reinforcement learning model; Deep feature learning is performed on preprocessed Dendrobium images to extract inherent features of Dendrobium texture, color, and morphology, and output standardized Dendrobium feature maps. Anomaly features are explored in the feature map of Dendrobium, and an intrinsic reward signal with dual fusion is adaptively generated to characterize the defect and abnormal state of Dendrobium image; By combining the Dendrobium feature map with the intrinsic reward signal, the system autonomously explores suspicious defect areas in the image and outputs refined detection action commands. By associating detection action commands with intrinsic reward signals, the system achieves intelligent identification and pixel-level precise positioning of Dendrobium defects, and outputs the final Dendrobium defect detection results.

2. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 1, characterized in that, The parallel preprocessing operation performed on the original image based on FPGA hardware includes: The original Dendrobium images were subjected to 3×3 window mid-range filtering to remove salt-and-pepper noise and transmission noise. The RGB color image is converted into a single-channel grayscale image by using a grayscale weighted mapping formula, thus completing the grayscale normalization process. Dendrobium outline boundaries are identified based on row and column grayscale projection curves, and the region of interest is adaptively cropped to remove invalid background areas. The image size is unified by bilinear interpolation algorithm and normalized to a preset standard input size image, which is suitable for the model and FPGA parallel inference architecture.

3. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 1, characterized in that, The output standardized Dendrobium feature map includes: Receive preprocessed standard Dendrobium images and convert the image data into a four-dimensional tensor; By removing redundant convolutional layers and the four-level deep separable convolutional structure of the SE attention module, the shallow texture features and deep morphological features of Dendrobium are extracted step by step. Based on the self-supervised SimCLR contrastive learning architecture, the pre-training was completed using a large number of unlabeled normal Dendrobium images to learn the distribution of inherent features of normal Dendrobium. The final generated Dendrobium feature map is output in two equally divided paths to achieve feature reuse.

4. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 1, characterized in that, The process of exploring abnormal features in the Dendrobium feature map and adaptively generating a dual-fusion intrinsic reward signal includes: Dendrobium feature maps are processed in parallel by a fixed target network and a learnable prediction network. The Euclidean distance between the output features of the two networks is calculated to generate a normalized RND novelty reward. A lightweight autoencoder is used to perform encoding compression and feature reconstruction on the feature map of Dendrobium. The mean square error of the original features and the reconstructed features is calculated to generate a normalized reconstruction error reward. Extract the feature dispersion of the current feature map, and dynamically adjust the weighted fusion weight of the RND novelty reward and the reconstruction error reward based on the feature dispersion; By merging the dual-path rewards through a weighted summation method, a final adaptive intrinsic reward signal is generated and output separately for defect detection strategy optimization and defect status discrimination.

5. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 4, characterized in that, The structural configuration of fixed-target networks, learnable prediction networks, and lightweight autoencoders includes: The fixed target network is a 3-layer fully connected network with fixed weight parameters after initialization. The learnable prediction network adopts the same 3-layer fully connected network with parameters updated iteratively in real time. The lightweight autoencoder is symmetrically composed of two fully connected encoding layers and two fully connected decoding layers, retaining only the core feature reconstruction operation to reduce computational overhead; Both the novelty reward and the reconstruction error reward of the generated RND are processed by Min-Max normalization and mapped to a unified interval to ensure the accuracy of reward fusion.

6. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 1, characterized in that, The method combines the Dendrobium feature map with the intrinsic reward signal to autonomously explore suspicious defect areas in the image and output refined detection action instructions, including: The feature map of Dendrobium is evenly divided into multiple feature blocks, and each feature block is used as an independent detection state to achieve fine detection of local areas of the image. By inputting state features into a lightweight strategy network, it outputs the probability distribution of four types of detection actions: focusing, zooming, moving, and judgment. A lightweight value network that shares input features with the policy network outputs a long-term revenue estimate of the current detection state. Based on the improved pruning-type PPO algorithm, the advantage function and policy gradient are calculated by combining the intrinsic reward signal, and the parameters of the policy network and value network are iteratively updated. Receive defect detection feedback results, dynamically calibrate the strategy exploration rate and reward fusion weight, and achieve end-to-end adaptive optimization.

7. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 6, characterized in that, The iterative update of the policy network and value network parameters includes: By using multi-layer feature mapping and Softmax activation to output a standardized action probability distribution, the action with the highest probability is selected as the optimal detection action. By using linear activation to output state reward values ​​and combining them with time-series discount factors to calculate state advantages, policy gradient optimization can be assisted. A gradient pruning mechanism is introduced during training to limit the range of policy gradient fluctuations, and the SGD optimizer is used to complete the stable iteration of network parameters.

8. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 1, characterized in that, The associated detection action command and intrinsic reward signal complete the intelligent identification and pixel-level precise positioning of Dendrobium defects, including: By integrating the intrinsic reward signal and the probability of detection action, the defect confidence is calculated for each feature block, and the defect region is determined by combining the dynamic adaptive threshold. Based on the detection action trajectory of focusing and magnification, a mapping relationship between feature block coordinates and original image coordinates is established, and the defect bounding box and defect area are output. Identify and classify five types of Dendrobium defects: mold, insect infestation, cracks, damage, and deformity, and output detection results including defect type, location, and confidence level; The detection results are fed back into the reinforcement learning iteration process to achieve online adaptive iterative optimization of the model.

9. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 1, characterized in that, It also includes lightweight model deployment steps: The convolution and fully connected operations of the model are decomposed into logical units and quantized and compressed to reduce the number of model parameters and computational cost. Adapted to FPGA hardware parallel pipeline architecture, it accelerates the entire process of image preprocessing, feature extraction, reward calculation, policy reasoning, and defect detection in parallel. By optimizing hardware computing power consumption through 16-bit fixed-point quantization, low-latency and low-power industrial real-time inference deployment can be achieved.

10. The lightweight defect detection method for Dendrobium officinale based on FPGA and reinforcement learning according to claim 1, characterized in that, It also includes the model training step: Unlabeled normal Dendrobium images were divided into training, validation and test sets according to proportions, and model pre-training and iterative training were completed using only normal samples. During the pre-training phase, the feature encoding parameters are initialized through self-supervised contrastive learning, and the general features of normal Dendrobium are learned. The core training phase relies on the inherent reward closed-loop optimization mechanism to iteratively update network parameters and monitor the model convergence state through the validation set to suppress overfitting. The convergence verification phase determines model convergence based on the stability of the reward distribution and the sample misclassification rate, and the defect detection performance of the test set is quantified.

Images