Optical crystal detection methods, equipment, and media based on improved YOLOv11 for sensing light transmission properties.

Abstract

This invention relates to an optical crystal detection method, device, and medium based on an improved YOLOv11 for sensing light transmission properties. First, a crystal image dataset containing both transmitted and reflected light conditions is acquired and segmented. Second, the YOLOv11 network is improved by introducing a windmill-shaped convolutional module into the feature extraction network to enhance the expression of crystal boundary direction and texture features. A local region self-attention module is introduced in the feature fusion stage to adaptively model local feature relationships, thereby improving the ability to distinguish weakly transparent crystals from the background. Then, the improved model is trained and validated using a training set to obtain the optimal detection model. Finally, this model is used to detect actual crystal images, outputting target location and category information. Compared with existing technologies, this invention effectively solves the problem of insufficient sensing of crystal light transmission properties under complex lighting conditions, improving detection accuracy and robustness.

Description

Technical Field

[0001] This invention relates to the field of intelligent optical crystal detection and computer vision technology, and in particular to an optical crystal detection method, device, and medium based on improved YOLOv11 for sensing light transmission properties. Background Technology

[0002] Optical crystal materials are core materials in fields such as infrared imaging, nonlinear optical devices, laser systems, and optical communication. Their light transmittance directly affects the performance and reliability of optical devices. During the growth and fabrication process, crystals are prone to defects due to factors such as process fluctuations, impurity introduction, and internal structural inhomogeneity, leading to a decrease in light transmittance. Therefore, after fabrication, crystal samples typically require effective testing and screening to distinguish samples with different light transmittance, ensuring consistency between material quality and device performance.

[0003] Early crystal screening relied primarily on imaging techniques such as optical microscopy, X-ray diffraction (XRD), and electron microscopy, with researchers manually judging crystals based on boundary integrity, transmittance, and morphological regularity. While this method is intuitive and can be combined with experience for analysis, it also has significant drawbacks: evaluation criteria depend on human experience, are highly subjective and difficult to quantify; it is inefficient and labor-intensive when dealing with high-throughput samples; and in low-contrast or complex backgrounds, the human eye's ability to distinguish between samples is limited, leading to false positives or false negatives. With the increasing demands for efficiency and consistency in the study of chalcogenide and other crystals, traditional manual screening methods are no longer adequate for practical needs.

[0004] In recent years, deep learning has been widely applied to crystal structure analysis and property prediction, gradually driving the transformation of crystal image analysis from relying on manual prior knowledge to a data-driven model. For example, in crystal structure recognition, atomic-scale image analysis, and automatic analysis of multiphase regions, methods based on convolutional neural networks have made significant progress, and related research has further extended to crystal target detection in microscopic environments. Meanwhile, single-stage target detection networks, represented by the YOLO series, have also attracted widespread attention in industrial vision tasks due to their efficient and compact structure, with several studies applying them to scenarios such as insulator defect detection and semiconductor grain localization. While these works demonstrate the potential of the YOLO framework in industrial inspection, its initial design was mostly aimed at natural scenes or general targets, focusing on the extraction of geometric morphology and texture features, without specifically modeling the differences in the light transmission properties of optical crystals.

[0005] Specifically, while CN120563418A enhances the ability to capture small defects through module replacement, its feature fusion mechanism still relies on a fixed receptive field. Under weak lighting and low contrast conditions in crystal imaging, it is difficult to effectively distinguish between transmission differences and background noise. Although CN116740177A uses a lightweight network to reduce computational overhead, its attention mechanism is not adaptable enough to optical response aliasing, which can easily lead to blurred boundaries and missed detections of targets under complex light field changes. More importantly, none of the above methods model the continuous gradient changes in crystal transmission characteristics. Their isotropic convolutional structures are unable to capture the directional features of transmission differences, resulting in insufficient detection stability in microscopic imaging scenarios.

[0006] In summary, most current deep learning-based methods still have limitations in the task of screening optical transmittance. On the one hand, existing research mostly focuses on geometric shape or structural recognition, with insufficient direct modeling and feature learning of differences in optical transmittance. On the other hand, these methods generally rely on images acquired under single illumination conditions, which limits their generalization ability under low light, low contrast, or complex light field changes, and are prone to missing targets due to optical response aliasing. Furthermore, isotropic convolution, commonly used in general detection models, is difficult to effectively capture the gradient of crystal boundary orientation and differences in transmittance, leading to frequent false positives and false negatives under noise and reflection interference.

[0007] Therefore, the core issue at present is: how to break through the limitations of traditional manual screening and existing general detection models, and develop an automated screening method that can stably and accurately identify the light transmission characteristics of crystals, so as to meet the actual needs of batch and high consistency testing of optical crystal materials. Summary of the Invention

[0008] This invention provides an optical crystal detection method, device, and medium based on improved YOLOv11 for sensing light transmission properties, in order to solve the problems in the existing technology, such as the high dependence of the crystal screening process on human experience, the easy omission of weakly transparent crystals under microscopic imaging conditions, and the insufficient detection accuracy under complex background conditions.

[0009] The objective of this invention can be achieved through the following technical solutions: The first aspect of this invention provides an optical crystal detection method based on an improved YOLOv11 for sensing light transmission properties, comprising the following steps: Acquire an optical crystal microscopy image dataset and divide the optical crystal image dataset into a training set and a validation set. The optical crystal image dataset includes crystal images acquired under transmitted light conditions and reflected light conditions. The YOLOv11 network structure is improved to construct an optical crystal detection model for light transmission property perception. This includes: introducing a windmill-shaped convolution module into the feature extraction network to enhance the expression of crystal boundary orientation structure and fine-grained texture features; and introducing a local region self-attention module into the feature fusion network stage to improve the ability to distinguish weakly transparent crystals from the background by adaptively modeling the feature relationships within local spatial regions. The training set is input into the optical crystal detection model for training to obtain the optimal detection model that meets the preset accuracy requirements. The performance of the optimal detection model is evaluated using the validation set. After meeting the preset accuracy requirements, the actual acquired crystal image is detected by the optimal detection model, and the position and category information of the crystal target are output.

[0010] Further, an optical crystal microscopy image dataset is acquired, and the optical crystal image dataset is divided into a training set and a validation set. The specific process includes: The transmitted light image is selected as the annotation reference image to annotate the crystal target and obtain annotation information including bounding box parameters and category information; The annotation information is mapped to the reflected light image corresponding to the same crystal through geometric transformation or direct mapping. The labeled image dataset was formatted to meet the input requirements of YOLO series networks and randomly divided into training and validation sets.

[0011] Furthermore, a windmill-shaped convolutional module is introduced into the feature extraction network to enhance the expressive power of crystal boundary orientation structure and fine-grained texture features. The specific process includes: In the feature extraction network, the input feature map is fed into the windmill-shaped convolution module, which processes the feature by setting multiple parallel convolution branches. Each branch uses an asymmetric filling method and combines a strip convolution kernel to extract boundary response features in a specific direction. Batch normalization and activation function processing are performed sequentially on the output of each branch to obtain feature maps in each direction; The output features of each branch are concatenated along the channel dimension to form a concatenated feature; By integrating the concatenated features through fusion convolution, a feature map with enhanced orientation is output, thereby improving the model's ability to perceive the directional information of crystal edges without significantly increasing computational complexity.

[0012] Furthermore, a local region self-attention module is introduced in the feature fusion stage to improve the distinguishing ability between the weakly transparent crystal and the background by adaptively modeling the feature relationships within the local spatial region. The specific process includes: In the feature fusion stage, the input feature map is divided into multiple local windows, each containing a feature region of a fixed size; Perform linear mapping on the feature sequences within each window to construct query vector, key vector, and value vector respectively; Self-attention computation is performed within a local window. Attention weights are obtained by taking the dot product of the query vector and the key vector, and then normalized using the softmax function. Finally, the attention output is obtained by weighted summation with the value vector. The attention output is remapped back to the spatial location and fused with the original features through residual connections. Then, it is combined with a convolutional feedforward network for feature enhancement, thereby strengthening the feature aggregation effect in weak contrast regions and effectively suppressing background noise interference.

[0013] Further, the training set is input into the optical crystal detection model for training to obtain an optimal detection model that meets the preset accuracy requirements. The performance of the optimal detection model is then evaluated using a validation set. The specific process includes: Configure the hyperparameters required for model training, including setting the number of training epochs, batch size, and initial learning rate. Select AdamW as the optimizer and configure the momentum and weight decay parameters. The training set images are preprocessed, and multiple training images are randomly stitched together using Mosaic data augmentation technology. At the same time, random flipping and brightness perturbation operations are performed, and all input images are uniformly scaled to a preset resolution. Then, the preprocessed training set is input into the optical crystal detection model, and the automatic mixed precision training mode is enabled to perform forward and backward propagation calculations, dynamically adjust the learning rate, and save intermediate model parameters. After each round of training, the validation set is used to evaluate the performance, and the mean precision, recall and average precision are calculated. By comparing each indicator with the preset precision threshold, the model parameters with the highest mean average precision on the validation set are selected as the optimal detection model.

[0014] Furthermore, after meeting the preset accuracy requirements, the actual acquired crystal image is detected using the optimal detection model, and the location and category information of the crystal target are output. The specific process includes: Load the optimal detection model parameters that have been evaluated on the validation set and meet the preset accuracy requirements, and initialize the model weights and network structure; Input the actual acquired crystal microscopic images into the model. The crystal microscopic images include grayscale images acquired under transmitted light conditions and color images acquired under reflected light conditions. The input image is processed by an improved feature extraction network, in which the windmill-shaped convolution module extracts the directional features of the crystal boundary, and the local region self-attention module enhances the feature discrimination ability of weak contrast regions. The feature fusion network fuses the multi-scale features extracted by the backbone network, and the detection head network predicts the crystal target information based on the fused features. The detection head network predicts the bounding box coordinates, class probability and confidence score of the crystal target. The model output is parsed, and a non-maximum suppression algorithm is used to remove redundant detection boxes with high overlap. Detection results with confidence scores higher than a set threshold are retained, and the location bounding box information and corresponding category label of each crystal target are output.

[0015] Furthermore, after outputting the location and category information of the crystal target, a light transmission property screening step is included, the specific process of which includes: Based on the crystal target location information output by the optimal detection model, the optical response features of the corresponding region in the transmitted light image and the reflected light image are extracted, including gray value distribution, texture contrast, and edge sharpness index. The ratio of the average light transmission intensity of each crystal target under transmitted light conditions to the surface reflection intensity under reflected light conditions is calculated and normalized in combination with the geometric morphology parameters of the crystal target to obtain the light transmission attribute score. Based on the preset light transmittance performance judgment threshold, crystal targets with light transmittance attribute scores higher than the threshold are classified as high light transmittance potential samples, while those with scores lower than the threshold are classified as low light transmittance potential samples. Based on the classification results, a screening report is generated, outputting the light transmission attribute category and corresponding confidence level of each crystal target, thus completing the automated light transmission performance screening.

[0016] Furthermore, when detecting the actual acquired crystal image using the optimal detection model, a multi-source image information fusion step is also included. The specific process includes: Images of the same crystal sample acquired under transmitted light and reflected light conditions are registered, and spatial displacement errors are eliminated by feature point matching or template alignment algorithms to ensure the consistency of the position of the crystal target in the two images. The registered transmitted light image and reflected light image are respectively input into the feature extraction network of the optimal detection model. The windmill-shaped convolution module extracts the direction-sensitive boundary features, and the local region self-attention module enhances the representation ability of weak contrast regions. Next, in the feature fusion stage, the feature maps of the transmitted light image and the reflected light image are stitched together at the channel level, and the optical response information under different lighting conditions is integrated through a weighted fusion algorithm, where the weights are dynamically adjusted based on the confidence scores of the feature maps; The detection head predicts the location and category information of the crystal target by using the fused multi-source features.

[0017] A second aspect of the present invention provides an optical crystal detection device for sensing light transmission properties, comprising: An image acquisition device is used to acquire crystal images under transmitted light and reflected light conditions. The image acquisition device includes at least a microscope, a switchable transmitted light source, a reflected light source, and an image sensor. An image preprocessing module, connected to the image acquisition device, is used to perform registration and format standardization processing on the acquired transmitted light image and reflected light image; A data storage device is used to store preprocessed image datasets, parameters of the improved YOLOv11 optical crystal detection model, and detection results; The model processor, connected to the data storage and image preprocessing module, is used to execute the optical crystal detection method based on the improved YOLOv11 for sensing light transmission properties as described in any of the above. The result output interface is connected to the model processor and is used to output the location information, category information, and light transmission property screening results of the crystal target.

[0018] A third aspect of the present invention provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the optical crystal detection method based on improved YOLOv11 for sensing light transmission properties as described above.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention provides an optical crystal detection method based on an improved YOLOv11 model, oriented towards the perception of light transmission properties. By specifically improving the YOLOv11 network structure, it achieves automated, high-precision detection of weakly transparent, small-scale optical crystals under microscopic imaging conditions. By introducing a windmill-shaped convolution module in the feature extraction stage, the model's ability to express crystal boundary direction features and fine-grained texture information is enhanced, effectively improving the discriminability of crystals under complex backgrounds and low-contrast conditions compared to existing general detection models. Furthermore, by introducing a local region self-attention module in the feature fusion stage, the ability to model features in areas with significant differences in light transmission is strengthened, reducing the interference of background noise on the detection results.

[0020] 2) The improved YOLOv11 detection model is trained on a crystal image dataset under multiple illumination conditions. The trained optimal detection model is then used to detect actual acquired crystal microscopic images, stably outputting crystal position and category information, thus achieving automated screening of crystal samples. Compared with existing crystal screening methods that rely on human experience, this invention maintains a relatively lightweight model structure and low computational complexity while ensuring improved detection accuracy, making it suitable for engineering deployment and improving the consistency, efficiency, and reliability of the crystal screening process. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall architecture of the intelligent optical crystal detection system for sensing light transmission properties as described in this invention. Figure 2 This is a schematic diagram of optical crystal images acquired under different lighting conditions, including grayscale images of the crystal acquired under transmitted light illumination and color images of the crystal acquired under reflected light illumination. Figure 3 This is a schematic flowchart of the optical crystal detection method based on improved YOLOv11 for sensing light transmission properties as described in this invention. Figure 4 This is a schematic diagram of the functional structure of the orientation-sensitive feature extraction unit in an embodiment of the present invention; Figure 5 This is a schematic diagram of the functional structure of the local region optical difference modeling unit in an embodiment of the present invention.

[0022] Figure 6 This is a schematic diagram of the characteristic response of the optical crystal detection model to the crystal region in an embodiment of the present invention; Figure 7 This is a schematic diagram of the results based on the crystal detection method in an embodiment of the present invention. Detailed Implementation

[0023] In specific implementation, this invention acquires an optical crystal microscopy image dataset and divides the dataset into a training set and a validation set. The specific process includes: The transmitted light image is selected as the annotation reference image to annotate the crystal target and obtain annotation information including bounding box parameters and category information; The annotation information is mapped to the reflected light image corresponding to the same crystal through geometric transformation or direct mapping. The labeled image dataset was formatted to meet the input requirements of YOLO series networks and randomly divided into training and validation sets.

[0024] In specific implementation, this invention introduces a windmill-shaped convolutional module into the feature extraction network to enhance the expressive power of crystal boundary orientation structure and fine-grained texture features. The specific process includes: In the feature extraction network, the input feature map is fed into the windmill-shaped convolution module, which processes the feature by setting multiple parallel convolution branches. Each branch uses an asymmetric filling method and combines a strip convolution kernel to extract boundary response features in a specific direction. Batch normalization and activation function processing are performed sequentially on the output of each branch to obtain feature maps in each direction; The output features of each branch are concatenated along the channel dimension to form a fused feature; By integrating the concatenated features through fusion convolution, a feature map with enhanced orientation is output, thereby improving the model's ability to perceive the directional information of crystal edges without significantly increasing computational complexity.

[0025] In specific implementation, this invention introduces a local region self-attention module in the feature fusion stage to improve the distinguishing ability between the weakly transparent crystal and the background through adaptive modeling of feature relationships within the local spatial region. The specific process includes: In the feature fusion stage, the input feature map is divided into multiple local windows, each containing a feature region of a fixed size; Perform linear mapping on the feature sequences within each window to construct query vector, key vector, and value vector respectively; Self-attention computation is performed within a local window. Attention weights are obtained by taking the dot product of the query vector and the key vector, and then normalized using the softmax function. Finally, the attention output is obtained by weighted summation with the value vector. The attention output is remapped back to the spatial location and fused with the original features through residual connections. Then, it is combined with a convolutional feedforward network for feature enhancement, thereby strengthening the feature aggregation effect in weak contrast regions and effectively suppressing background noise interference.

[0026] In specific implementation, the training set is input into the optical crystal detection model for training to obtain an optimal detection model that meets the preset accuracy requirements. The performance of the optimal detection model is then evaluated using a validation set. The specific process includes: Configure the hyperparameters required for model training, including setting the number of training epochs, batch size, and initial learning rate. Select AdamW as the optimizer and configure the momentum and weight decay parameters. The training set images are preprocessed, and multiple training images are randomly stitched together using Mosaic data augmentation technology. At the same time, random flipping and brightness perturbation operations are performed, and all input images are uniformly scaled to a preset resolution. Then, the preprocessed training set is input into the optical crystal detection model, and the automatic mixed precision training mode is enabled to perform forward and backward propagation calculations, dynamically adjust the learning rate, and save intermediate model parameters. After each round of training, the validation set is used to evaluate the performance, and the mean precision, recall and average precision are calculated. By comparing each indicator with the preset precision threshold, the model parameters with the highest mean average precision on the validation set are selected as the optimal detection model.

[0027] In specific implementation, after meeting the preset accuracy requirements, the present invention uses an optimal detection model to detect the actually acquired crystal image and outputs the position and category information of the crystal target. The specific process includes: Load the optimal detection model parameters that have been evaluated on the validation set and meet the preset accuracy requirements, and initialize the model weights and network structure; Input the actual acquired crystal microscopic images into the model. The crystal microscopic images include grayscale images acquired under transmitted light conditions and color images acquired under reflected light conditions. The input image is processed by an improved feature extraction network, in which the windmill-shaped convolution module extracts the directional features of the crystal boundary, and the local region self-attention module enhances the feature discrimination ability of weak contrast regions. The feature fusion network fuses multi-scale features, and the detection head network predicts the bounding box coordinates, class probability, and confidence score of the crystal target. The model output is parsed, and a non-maximum suppression algorithm is used to remove redundant detection boxes with high overlap. Detection results with confidence scores higher than a set threshold are retained, and the location bounding box information and corresponding category label of each crystal target are output.

[0028] In specific implementation, after outputting the location and category information of the crystal target, this invention further includes a light transmission property screening step, the specific process of which includes: Based on the crystal target location information output by the optimal detection model, the optical response features of the corresponding region in the transmitted light image and the reflected light image are extracted, including gray value distribution, texture contrast, and edge sharpness index. The ratio of the average light transmission intensity of each crystal target under transmitted light conditions to the surface reflection intensity under reflected light conditions is calculated and normalized in combination with the geometric morphology parameters of the crystal target to obtain the light transmission attribute score. Based on the preset light transmittance performance judgment threshold, crystal targets with light transmittance attribute scores higher than the threshold are classified as high light transmittance potential samples, while those with scores lower than the threshold are classified as low light transmittance potential samples. Based on the classification results, a screening report is generated, outputting the light transmission attribute category and corresponding confidence level of each crystal target, thus completing the automated light transmission performance screening.

[0029] In specific implementation, when detecting the actually acquired crystal image using the optimal detection model, this invention also includes a multi-source image information fusion step, the specific process of which includes: Images of the same crystal sample acquired under transmitted light and reflected light conditions are registered, and spatial displacement errors are eliminated by feature point matching or template alignment algorithms to ensure the consistency of the position of the crystal target in the two images. The registered transmitted light image and reflected light image are respectively input into the feature extraction network of the optimal detection model. The windmill-shaped convolution module extracts the direction-sensitive boundary features, and the local region self-attention module enhances the representation ability of weak contrast regions. Next, in the feature fusion stage, the feature maps of the transmitted light image and the reflected light image are stitched together at the channel level, and the optical response information under different lighting conditions is integrated through a weighted fusion algorithm, where the weights are dynamically adjusted based on the confidence scores of the feature maps; The detection head predicts the location and category information of the crystal target by using the fused multi-source features.

[0030] Where there is no conflict, the various embodiments of the present invention and the technical features thereof can be combined with each other. The following is in conjunction with... Figures 1 to 7 The following further illustrates the specific implementation process of the optical crystal detection method based on the improved YOLOv11 for sensing light transmission properties, as described in this invention. The accompanying drawings are only for illustrating some embodiments of the invention and do not constitute a limitation on the scope of protection of this invention. Equivalent substitutions or modifications made by those skilled in the art without departing from the technical concept of this invention should fall within the scope of protection of this invention.

[0031] Unless otherwise specified in this technical solution, the component model, material name, connection structure, circuit structure, control method, algorithm, and other features are considered to be common technical features disclosed in the prior art.

[0032] Example 1 The flowchart of the optical crystal detection method based on the improved YOLOv11 for sensing light transmission properties in this embodiment is shown below. Figure 3 As shown, this method sequentially includes crystal image acquisition under multiple illumination conditions, cross-illumination annotation mapping, crystal detection dataset construction, crystal detection model construction and training, and crystal target detection and screening. Each step is executed sequentially in a data flow manner. By introducing improved structures targeting crystal transparency properties in the feature extraction and feature fusion stages, automated detection and screening of weakly transparent, small-scale optical crystals is achieved. The method includes at least the following steps.

[0033] S101: Crystal image acquisition under multi-light conditions In a multi-source optical imaging device with both transmitted and reflected illumination structures, the same optical crystal sample to be tested is imaged under both transmitted and reflected illumination conditions to acquire a transmitted crystal image. Image with reflected light crystal .

[0034] like Figure 2 As shown, Figure 2This is a schematic diagram of optical crystal images acquired under different lighting conditions, including a grayscale image of the crystal acquired under transmitted light illumination and a color image of the crystal acquired under reflected light illumination. In this embodiment, the same crystal sample is imaged by switching between upper and lower light sources. The lower light source is used to acquire the grayscale image of the crystal under transmitted light illumination, and the upper light source is used to acquire the color image of the crystal under reflected light illumination.

[0035] By imaging the same crystal sample under different lighting conditions, we can obtain information on the internal light transmission characteristics and overall contour structure of the crystal, as well as the surface morphology and reflection characteristics of the crystal. This provides a complementary optical characterization data foundation for subsequent crystal detection aimed at sensing light transmission properties.

[0036] S102: Selection of Annotation Baseline and Cross-Lighting Annotation Mapping Selecting transmitted light crystal images Using the reference image for annotation, the crystal targets within it are annotated to obtain annotation information. The annotation information includes at least the bounding box parameters and category information of the crystal target.

[0037] To ensure spatial consistency of annotation results for the same crystal under different lighting conditions, the annotation information will be... Images mapped onto the same crystal under reflected light illumination conditions In the middle, obtain the corresponding annotation information. This allows for the construction of crystal detection sample pairs that are consistent across lighting conditions.

[0038] In one possible implementation, the bounding box can be represented as a set of four corner points. By estimating the geometric transformation relationship between the transmitted light image and the reflected light image, the coordinates of the four corner points are transformed, and the corresponding bounding boxes in the reflected light image are generated based on the transformed corner points. When the field of view is consistent under the two illumination conditions and the relative displacement is negligible, the same bounding box parameters can be used directly or only translation compensation can be performed.

[0039] S103: Constructing a crystal detection dataset for sensing light transmission properties Crystal images acquired under various lighting conditions and their corresponding annotation information are compiled into a crystal detection dataset, which is then divided into a training set and a validation set for subsequent training and performance evaluation of the intelligent crystal detection model. The annotation format preferably meets the input requirements of the YOLO series of object detection networks.

[0040] S104: Constructing a Crystal Intelligent Detection Model like Figure 1As shown, the crystal intelligent detection model in this embodiment is built based on a single-stage target detection network, preferably a YOLOv11 network structure, which includes a backbone network, a feature fusion network, and a detection head.

[0041] To address the common characteristics of optical crystal targets under microscopic imaging conditions, such as small scale, blurred boundaries, indistinct light transmission differences, and complex backgrounds, the following two functional units are introduced into the YOLOv11 network to enhance the model's ability to perceive crystal structure features and differences in light transmission properties.

[0042] (1) Direction-sensitive feature extraction based on windmill-shaped convolution PConv like Figure 4 The diagram shown illustrates the functional structure of the orientation-sensitive feature extraction unit in this embodiment of the invention. This unit processes the input features in parallel through multiple feature extraction branches with different orientation response characteristics, and forms orientation-enhanced features through feature fusion.

[0043] Specifically, the orientation-sensitive feature extraction unit enhances the orientation perception capability of the crystal boundary through a windmill-shaped convolution (PConv) structure. The input feature map is processed in parallel into 4 convolutional branches, each branch using asymmetric padding and bar convolutional kernels (such as 1×3 or 3×1) to extract boundary responses in specific directions (such as horizontal, vertical, and diagonal).

[0044] After batch normalization and SiLU activation function processing, the outputs of each branch are concatenated along the channel dimension to form a multi-directional feature aggregation.

[0045] Finally, the features are integrated and spliced ​​through 1×1 convolution to output a feature map with enhanced orientation.

[0046] In the backbone network of the crystal intelligent detection model, a pinwheel-shaped convolution (PConv) is introduced to replace some conventional convolutional layers, thereby enhancing the ability to express the directional features of crystal boundaries and structures. Let the input feature map be: PConv processes input features by setting up multiple parallel convolutional branches with different directional response characteristics. Each branch uses asymmetric padding and combines a strip convolutional kernel (1×3 or 3×1) to extract the boundary response in a specific direction. Its branch output can be represented as: in, This represents the asymmetric fill operation corresponding to the i-th direction. For the corresponding convolution kernel parameters, BN is the batch normalization operation. The activation function is preferably SiLU.

[0047] The output features of each directional branch are concatenated along the channel dimension to obtain the fused feature: Then, by integrating the multi-directional features through fusion convolution, the output features of PConv are obtained: Through the above structure, PConv enhances the model's ability to express crystal boundary orientation structure and fine-grained texture features without significantly increasing computational complexity, thereby improving the discriminability of weak contrast crystals under complex background conditions.

[0048] (2) Local optical difference modeling based on local region self-attention LRSA like Figure 5 The diagram shown illustrates the functional structure of the local region optical difference modeling unit in this embodiment of the invention. This unit performs local region segmentation of the input features during the feature fusion stage, and adaptively models the optical response features of the crystal region and the background region within the local region.

[0049] The local region optical difference modeling unit enhances the feature discrimination capability of weak contrast regions through local window self-attention (LRSA). The specific process includes: The input feature map is divided into local windows of fixed size (e.g., p×p), and the feature sequence within each window is linearly mapped into query (Q), key (K), and value (V) vectors.

[0050] Self-attention weights are calculated within a window to highlight the optical differences between the crystal and the background.

[0051] The attention output is fused with the original features through residual connections and then enhanced by a convolutional feedforward network, effectively suppressing background noise. This module adaptively focuses on local regions with significant differences in light transmittance, improving the robustness of small crystal detection.

[0052] In the feature fusion network of the crystal intelligent detection model, Local-Region Self-Attention (LRSA) is introduced to model the optical response differences between the crystal region and the background region within a local imaging area. Let the input fusion features be: The feature map is divided into multiple local windows, each with a size of (p×p), and the features within each window are flattened into a token sequence. A linear mapping is performed on the token sequence to construct a query vector (Q), a key vector (K), and a value vector (V): Perform self-attention calculations within each local window: in For the attention dimension, the attention output is remapped back to its corresponding spatial location and fused with the original features via residual connections. To further enhance the feature aggregation effect in local weak contrast regions, the LRSA module can also introduce a convolutional feedforward network for feature enhancement, ultimately obtaining fused features after modeling local optical differences.

[0053] In this way, the model can adaptively focus on the light transmission difference information within the crystal region in a local range and effectively suppress the interference of background noise on the detection results.

[0054] S105: Model Training, Crystal Detection, and Transmittance Potential Screening The training set is input into the above-mentioned crystal intelligent detection model for training to obtain the optimal model parameters that meet the preset accuracy requirements. After evaluating the performance of the model using the validation set, the optimal model is applied to the actual acquired crystal images to output the location information and confidence information of the crystal target.

[0055] Based on the detection results and in conjunction with preset light transmittance performance criteria, the system enables automatic detection and screening of optical crystal targets with high light transmittance potential. The light transmittance performance criteria can be determined comprehensively based on the imaging response characteristics of the crystal target under transmitted and reflected light conditions.

[0056] like Figure 6 The diagram shown is a schematic of the characteristic response of the optical crystal detection model to the crystal region in an embodiment of the present invention. It can be seen that after introducing the orientation-sensitive feature extraction and local region optical difference modeling mechanism, the characteristic response of the model in the crystal region is significantly enhanced and the crystal boundary is clearer.

[0057] like Figure 7 The diagram shown is a schematic diagram of the crystal detection result based on the detection method in an embodiment of the present invention. The method can accurately output the position information and corresponding confidence information of the crystal target, thereby realizing the automatic detection and screening of optical crystal targets.

[0058] in: mAP – Mean Precision GFLOPs refers to the number of floating-point operations that can be performed per second, measured in billions. This is a metric for hardware performance, especially the ability of a GPU or CPU to process graphics and deep learning models. Params(M) — Number of model parameters This embodiment was conducted under unified hardware and software conditions. The experimental environment included Windows 10, Python 3.9, PyTorc 2.8.0, and CUDA 12.8. The hardware platform used an Intel® Core™ i5-14600K processor and an NVIDIA GeForce RTX 5070. To ensure fair and comparable results, all models used consistent data partitioning, parameter initialization, and training strategies, and a fixed random seed. The model used the same hyperparameter settings as the baseline model: 200 training epochs, batch size 16, initial learning rate 0.01, optimizer AdamW, momentum 0.937, and weight decay 5×10⁻⁶. -4 During training, data augmentation techniques such as Mosaic, random flipping, and brightness perturbation were introduced, and AMP was enabled to improve training efficiency. All input images were uniformly set to 640×640 resolution.

[0059] The trained model can then be deployed and run according to the crystal screening requirements (images, video streams, etc.) to achieve real-time detection and screening.

[0060] In this embodiment, to systematically evaluate the model's performance in crystal detection and light transmittance screening tasks, precision, recall, and mean average precision (mAP) are selected as evaluation metrics.

[0061] Precision represents the proportion of the actual crystal target in the model's prediction results, and its calculation formula is as follows: Where TP represents the number of correctly detected crystal targets, and FP represents the number of falsely detected amorphous regions. Higher precision indicates that the model can effectively distinguish crystals from background noise.

[0062] Recall represents the proportion of true crystal targets identified by the model, and its calculation formula is as follows: FN represents the number of missed samples. A higher recall indicates that the model has a stronger ability to detect small crystals.

[0063] The mean accuracy (mAP) is used to comprehensively reflect the overall performance of the model under different detection thresholds, and its calculation formula is as follows: in For the number of categories, For the first Average accuracy. The performance comparison results of various existing detection models with the method of this embodiment are shown in Table 1. The results show that the method described in this embodiment exhibits superior technical performance in terms of crystal detection accuracy, recall capability, and overall performance, while also possessing advantages in terms of model parameter scale and computational complexity for engineering deployment.

[0064] In addition, this example conducts ablation experiments based on YOLOv11, and introduces single modules and their combinations for comparative analysis. The experimental results are shown in Table 1. After adding PConv alone, mAP@0.5 increases from 0.736 to 0.755, indicating that this convolution helps to enhance the expression of crystal boundaries and fine-grained texture features. When adding LRSA alone, mAP@0.5 increases to 0.760, indicating that this attention mechanism can effectively enhance feature aggregation and region discrimination under weak contrast conditions. When both are introduced synergistically, Precision, Recall, and mAP@0.5 reach 0.815, 0.756, and 0.788, respectively, with an overall performance improvement of approximately 5.2% compared to the baseline, showing a stable and significant synergistic gain.

[0065] Table 1 Ablation Experiment Results Furthermore, although the number of parameters and computational complexity increased slightly to 2.91 M and 7.7 GFLOPs respectively, the overhead was completely within an acceptable range given the significant improvement in accuracy. This indicates that the model achieved an effective balance between accuracy and efficiency and has good performance for engineering deployment.

[0066] To test the performance improvement of the method for crystal detection in this invention, relevant indicators were calculated for both existing detection models and the improved network model of this invention. The comparison results of relevant indicators between existing technologies and the method of this invention are shown in Table 2. The improved YOLOv11 model achieves higher crystal recognition accuracy than the Faster-RCNN algorithm and is also superior to other YOLO algorithms, enabling more accurate detection and location of crystals. The improved YOLOv11 model achieves a better balance between parameter quantity, computational cost, and model size, making it easier for equipment deployment and application. The method of this invention is more feasible and superior to existing technologies.

[0067] Table 2. Comparison of relevant indicators between existing technologies and the methods of the embodiments of the present invention Example 2 This embodiment provides an optical crystal testing device for sensing light transmission properties, including: An image acquisition device is used to acquire crystal images under transmitted light and reflected light conditions. The image acquisition device includes at least a microscope, a switchable transmitted light source, a reflected light source, and an image sensor. An image preprocessing module, connected to the image acquisition device, is used to perform registration and format standardization processing on the acquired transmitted light image and reflected light image; A data storage device is used to store preprocessed image datasets, parameters of the improved YOLOv11 optical crystal detection model, and detection results; The model processor, connected to the data storage and image preprocessing module, is used to execute the optical crystal detection method based on the improved YOLOv11 for sensing light transmission properties as described in any of the above. The result output interface is connected to the model processor and is used to output the location information, category information, and light transmission property screening results of the crystal target.

[0068] In practical implementation, this equipment achieves automated detection and screening of optical crystals through multi-module collaboration. First, the image acquisition device integrates switchable transmitted and reflected light sources (such as LED arrays) on a microscope platform, coupled with a high-resolution image sensor (such as CMOS or CCD), ensuring the capture of the crystal's internal light transmission characteristics under transmitted light conditions and the acquisition of surface morphology information under reflected light conditions. To eliminate displacement errors caused by illumination switching, the image preprocessing module uses a feature point matching algorithm based on SIFT or ORB to perform real-time registration of the dual-source images and standardizes the image format through bilinear interpolation (e.g., unifying it to 640×640 pixel RGB or grayscale images). The data storage uses a solid-state drive (SSD) or cloud storage system, storing not only the preprocessed image dataset but also caching the parameters of the improved YOLOv11 model (such as the weight file of the windmill-shaped convolution module), supporting rapid loading and updates. The model processor is the core component, typically equipped with a GPU (such as NVIDIA series) to accelerate computation. Its software layer implements the detection method based on the PyTorch or TensorFlow framework: In the feature extraction stage, the windmill-shaped convolution module extracts the directional features of the crystal boundary through parallel convolution branches; in the feature fusion stage, the local region self-attention module dynamically enhances the saliency of weak contrast regions. The result output interface is designed as a USB or Ethernet interface, which can be connected to a monitor or industrial PLC system to output the crystal position, category, and light transmission property score in real time, and generate a structured report (such as JSON format).

[0069] The deployment of this device requires optimization of power consumption and real-time performance. For example, the image acquisition unit can reduce light source switching latency through trigger-based control; the model processor adopts a multi-threaded mechanism to process image acquisition and model inference tasks in parallel; the data storage uses a partitioning strategy to separate training data from runtime cache, improving response speed. Furthermore, the device can integrate environmental sensors (such as temperature and humidity sensors) to compensate for imaging conditions, ensuring robust operation in complex laboratory environments. (Overall architecture reference) Figure 1 The system design enables full automation of the process from image acquisition to result filtering.

[0070] Example 3 An embodiment provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, performs the optical crystal detection method based on improved YOLOv11 for sensing light transmission properties as described above. The storage medium can be an electronic medium, magnetic medium, optical medium, electromagnetic medium, infrared medium, or semiconductor system or propagation medium. The storage medium may also include semiconductor or solid-state memory, magnetic tape, removable computer disk, random access memory (RAM), read-only memory (ROM), hard disk, and optical disc. Optical discs may include optical disc-read-only memory (CD-ROM), optical disc-read / write (CD-RW), and DVD.

[0071] In practice, when the medium is executed by a computer processor (such as an Intel Core i5 or ARM architecture chip), the pre-trained parameters of the improved YOLOv11 model are first loaded (e.g., via Pickle or HDF5 format), and the network structure of the windmill-shaped convolutional module and the local region self-attention module is initialized.

[0072] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.

Claims

1. An optical crystal detection method based on improved YOLOv11 for sensing light transmission properties, characterized in that, Includes the following steps: Acquire an optical crystal microscopy image dataset and divide the optical crystal image dataset into a training set and a validation set. The optical crystal image dataset includes crystal images acquired under transmitted light conditions and reflected light conditions. The YOLOv11 network structure is improved to construct an optical crystal detection model for light transmission property perception. This includes: introducing a windmill-shaped convolution module into the feature extraction network to enhance the expressive ability of crystal boundary orientation structure and fine-grained texture features; and introducing a local region self-attention module in the feature fusion stage to improve the ability to distinguish weakly transparent crystals from the background by adaptively modeling the feature relationship within the local spatial region. The training set is input into the optical crystal detection model for training to obtain the optimal detection model that meets the preset accuracy requirements. The performance of the optimal detection model is evaluated using the validation set. After meeting the preset accuracy requirements, the actual acquired crystal image is detected by the optimal detection model, and the position and category information of the crystal target are output.

2. The optical crystal detection method based on improved YOLOv11 for sensing light transmission properties according to claim 1, characterized in that, Acquire an optical crystal microscopy image dataset and divide the dataset into a training set and a validation set. The specific process includes: The transmitted light image is selected as the annotation reference image to annotate the crystal target and obtain annotation information including bounding box parameters and category information; The annotation information is mapped to the reflected light image corresponding to the same crystal through geometric transformation or direct mapping. The labeled image dataset was formatted to meet the input requirements of YOLO series networks and randomly divided into training and validation sets.

3. The optical crystal detection method based on improved YOLOv11 for sensing light transmission properties according to claim 1, characterized in that, A windmill-shaped convolutional module is introduced into the feature extraction network to enhance the expressive power of crystal boundary orientation structure and fine-grained texture features. The specific process includes: In the feature extraction network, the input feature map is fed into the windmill-shaped convolution module, which processes the feature by setting multiple parallel convolution branches. Each branch uses an asymmetric filling method and combines a strip convolution kernel to extract boundary response features in a specific direction. Batch normalization and activation function processing are performed sequentially on the output of each branch to obtain feature maps in each direction; The output features of each branch are concatenated along the channel dimension to form a concatenated feature; By integrating the concatenated features through fusion convolution, a feature map with enhanced orientation is output, thereby improving the model's ability to perceive the directional information of crystal edges without significantly increasing computational complexity.

4. The optical crystal detection method based on improved YOLOv11 for sensing light transmission properties according to claim 1, characterized in that, In the feature fusion network stage, a local region self-attention module is introduced to improve the distinguishing ability between the weakly transparent crystal and the background by adaptively modeling the feature relationships within the local spatial region. The specific process includes: In the feature fusion stage, the input feature map is divided into multiple local windows, each containing a feature region of a fixed size; Perform linear mapping on the feature sequences within each window to construct query vector, key vector, and value vector respectively; Self-attention computation is performed within a local window. Attention weights are obtained by taking the dot product of the query vector and the key vector, and then normalized using the softmax function. Finally, the attention output is obtained by weighted summation with the value vector. The attention output is remapped back to the spatial location and fused with the original features through residual connections, and then combined with a convolutional feedforward network for feature enhancement.

5. The optical crystal detection method based on improved YOLOv11 for sensing light transmission properties according to claim 1, characterized in that, The training set is input into the optical crystal detection model for training to obtain the optimal detection model that meets the preset accuracy requirements. The performance of the optimal detection model is then evaluated using a validation set. The specific process includes: Configure the hyperparameters required for model training, including setting the number of training epochs, batch size, and initial learning rate. Select AdamW as the optimizer and configure the momentum and weight decay parameters. The training set images are preprocessed, and multiple training images are randomly stitched together using Mosaic data augmentation technology. At the same time, random flipping and brightness perturbation operations are performed, and all input images are uniformly scaled to a preset resolution. Then, the preprocessed training set is input into the optical crystal detection model, and the automatic mixed precision training mode is enabled to perform forward and backward propagation calculations, dynamically adjust the learning rate, and save intermediate model parameters. After each round of training, the validation set is used to evaluate the performance, and the mean precision, recall and average precision are calculated. By comparing each indicator with the preset precision threshold, the model parameters with the highest mean average precision on the validation set are selected as the optimal detection model.

6. The optical crystal detection method based on improved YOLOv11 for sensing light transmission properties according to claim 1, characterized in that, After meeting the preset accuracy requirements, the actual acquired crystal image is detected using the optimal detection model, and the location and category information of the crystal target are output. The specific process includes: Load the optimal detection model parameters that have been evaluated on the validation set and meet the preset accuracy requirements, and initialize the model weights and network structure; Input the actual acquired crystal microscopic images into the model. The crystal microscopic images include grayscale images acquired under transmitted light conditions and color images acquired under reflected light conditions. The input image is processed by an improved feature extraction network, in which the windmill-shaped convolution module extracts the directional features of the crystal boundary, and the local region self-attention module enhances the feature discrimination ability of weak contrast regions. The feature fusion network fuses the multi-scale features extracted by the backbone network, and the detection head network predicts the crystal target information based on the fused features. Specifically, the detection head network predicts the bounding box coordinates, class probability, and confidence score of the crystal target. The model output is parsed, and a non-maximum suppression algorithm is used to remove redundant detection boxes with high overlap. Detection results with confidence scores higher than a set threshold are retained, and the location bounding box information and corresponding category label of each crystal target are output.

7. The optical crystal detection method based on improved YOLOv11 for sensing light transmission properties according to claim 1, characterized in that, After outputting the location and category information of the crystal target, a further step is included: a light transmission property screening step. The specific process includes: Based on the crystal target location information output by the optimal detection model, the optical response features of the corresponding region in the transmitted light image and the reflected light image are extracted, including gray value distribution, texture contrast, and edge sharpness index. The ratio of the average light transmission intensity to the surface reflection intensity under transmitted light conditions for each crystal target is calculated and normalized in combination with the geometric morphology parameters of the crystal target to obtain the light transmission attribute score. Based on the preset light transmittance performance judgment threshold, crystal targets with light transmittance attribute scores higher than the threshold are classified as high light transmittance potential samples, while those with scores lower than the threshold are classified as low light transmittance potential samples. Based on the classification results, a screening report is generated, outputting the light transmission attribute category and corresponding confidence level of each crystal target, thus completing the automated light transmission performance screening.

8. The optical crystal detection method based on improved YOLOv11 for sensing light transmission properties according to claim 1, characterized in that, When detecting the actual acquired crystal images using the optimal detection model, a multi-source image information fusion step is also included. The specific process includes: Images of the same crystal sample acquired under transmitted light and reflected light conditions are registered, and spatial displacement errors are eliminated by feature point matching or template alignment algorithms to ensure the consistency of the position of the crystal target in the two images. The registered transmitted light image and reflected light image are respectively input into the feature extraction network of the optimal detection model. The windmill-shaped convolution module extracts the direction-sensitive boundary features, and the local region self-attention module enhances the representation ability of weak contrast regions. Next, in the feature fusion stage, the feature maps of the transmitted light image and the reflected light image are stitched together at the channel level, and the optical response information under different lighting conditions is integrated through a weighted fusion algorithm, where the weights are dynamically adjusted based on the confidence scores of the feature maps; The detection head predicts the location and category information of the crystal target by using the fused multi-source features.

9. An optical crystal testing device for sensing light transmission properties, characterized in that, include: An image acquisition device is used to acquire crystal images under transmitted light and reflected light conditions. The image acquisition device includes at least a microscope, a switchable transmitted light source, a reflected light source, and an image sensor. An image preprocessing module, connected to the image acquisition device, is used to perform registration and format standardization processing on the acquired transmitted light image and reflected light image; A data storage device is used to store preprocessed image datasets, parameters of the improved YOLOv11 optical crystal detection model, and detection results; A model processor, connected to the data storage and image preprocessing module, is used to execute the optical crystal detection method based on improved YOLOv11 for sensing light transmission properties as described in any one of claims 1 to 8. The result output interface is connected to the model processor and is used to output the location information, category information, and light transmission property screening results of the crystal target.

10. A storage medium containing computer-executable instructions, characterized in that, When executed by a computer processor, the storage medium of the computer-executable instructions is used to perform the optical crystal detection method based on the improved YOLOv11 for sensing light transmission properties as described in any one of claims 1 to 8.

Images