High PPI VR display module display foreign matter detection method, medium and equipment

Abstract

The invention relates to the technical field of machine vision and industrial detection, in particular to a high PPI VR display module display foreign matter detection method, medium and equipment, and the method comprises the following steps: data set acquisition: adopting multiple pure color pictures as test display content; labeling a data set; model training; image acquisition; foreign matter detection; foreign matter size calculation; and counting the number of foreign matters in a partitioned manner: partitioning the display region, counting the number of the foreign matters in each partition, and judging whether the display module is a good product or not according to a set threshold value. According to the invention, the efficiency and precision of foreign matter detection are obviously improved; a dimension specification measuring and judging method is provided for display module products with foreign matters, products which do not affect display performance and reliability are judged as good products, misjudgment is avoided, and benefits are improved; and the number of the foreign matters in each region is counted by using partition statistics, so that the product quality is ensured while the efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision and industrial inspection, and in particular to a method, medium, and device for detecting foreign objects in a high-PPI VR display module. Background Art

[0002] With the popularization and application of virtual reality (VR) technology, the demand for high-PPI (Pixels Per Inch) display modules is increasing continuously. Different from conventional display modules, high-PPI VR display modules have remarkable characteristics such as small pixel size, high pixel density, and fast response speed. Due to the extremely high pixel density of high-PPI display modules, the number of pixels per inch contained therein far exceeds that of traditional display modules. This characteristic makes it difficult for traditional foreign object detection methods for displays to meet the requirements for detecting foreign objects with finer dimensions during the detection process, which may lead to defective products flowing into the market. Therefore, there is an urgent need for an efficient detection method to solve the current situation. Summary of the Invention

[0003] Aiming at the problems of the prior art, the present invention provides a method, medium, and device for detecting foreign objects in a high-PPI VR display module. The design is ingenious, and a detection method for foreign objects in a high-PPI VR display module that integrates YOLO, attention mechanism, and Swin-Transformer is proposed, which significantly improves the efficiency and accuracy of foreign object detection; a method for measuring and determining the size specifications of display module products with foreign objects is proposed. Products that do not affect display performance and reliability are determined as qualified products, avoiding misjudgment and improving efficiency; partition statistics are used to separately count the number of foreign objects in each region, and the number of foreign objects is controlled by region according to requirements differences, improving efficiency while ensuring product quality.

[0004] To solve the above technical problems, the present invention adopts the following technical solutions:

[0005] The present invention provides a method for detecting foreign objects in a high-PPI VR display module, which includes the following steps:

[0006] Step S110, data set collection: A variety of pure color pictures are used as test display contents, and these pure color pictures will be displayed on the screen of the same display module with different background colors;

[0007] Step S120, data set annotation: After the data set collection is completed, it is necessary to classify and label the foreign objects in the collected images with region bounding boxes;

[0008] Step S130, Model Training: Use a deep learning model that includes Backbone, Neck, and Head. This deep learning model adopts the single-stage structure of YOLO. Among them, Backbone uses the outputs of Stage 2 to Stage 5 of the ResNet-50 residual network to extract the basic features of the input image. Before the features are passed to Neck, a convolutional network is used for deep feature fusion. The CBAM attention mechanism is used to highlight important features and eliminate redundant features. The features weighted by the CBAM attention mechanism will be passed into Neck. In Neck, the feature extraction method of traditional YOLO is adopted, and the Swin-Transformer feature extraction branch is introduced. Finally, the output features of Neck will be passed to the Detect detection network, and a threshold algorithm is used to filter out the detection results that meet the threshold range.

[0009] Step S210, Image Acquisition: Use an industrial camera to acquire the frame images displayed in real time by the VR display module.

[0010] Step S220, Foreign Object Detection: Pass the acquired image into the trained model for foreign object detection.

[0011] Step S230, Foreign Object Size Calculation: Calculate the actual size represented by each pixel in the image. By comparing with the actual pixel number of the foreign object, determine whether the foreign object size meets the specifications.

[0012] Step S240, Partition and Count the Number of Foreign Objects: Partition the display area, count the number of foreign objects in each partition, and determine whether the display module is a good product according to the set threshold.

[0013] Among them, the frame images in the image acquisition in Step S210 include images under five pure color frames of red, green, blue, white, gray, and black.

[0014] Among them, the results of the foreign object size calculation and the partition and count of the number of foreign objects are jointly used to determine whether the display module is a good product.

[0015] Among them, the steps of the foreign object size calculation in Step S230 include:

[0016] Step S231, Image Pixel Size Characterization Based on the Calibration Plate: Use the known calibration plate grid size and camera calibration parameters to calculate the ratio of pixel to actual size conversion, and calculate the actual size corresponding to the pixel length through the distance of the detected corner points in the image: Among them, L is the actual size of a grid on the calibration plate, and N is the number of pixels of the calibration plate grid in the acquired image.

[0017] Step S232, Calculation of the foreign object size based on the detection result: According to the number n of pixel points where the foreign object is located calculated from the detection result, calculate the size of the foreign object. The calculation method is as follows: A = S * n; Compare the calculated foreign object size A with the Pixel size x of the display module, and x is set according to the characteristics of different products; The determination principle is as follows:

[0018] For a display module that meets the specifications, the number of foreign objects is statistically analyzed by partition, and it is determined whether the number of foreign objects in each partition is within the specifications.

[0019] Among them, the step of statistically analyzing the number of foreign objects by partition in step S240 is as follows:

[0020] Step S241, Use the OTSU algorithm to segment the display area from other areas to obtain the display area; After that, use the moments algorithm of Opencv to solve the moment M of the contour, and the calculation method of the center coordinates is as follows: After obtaining the center coordinate point O(c x , c y ), respectively set R1 and R2 as the region radii of partition 1 and partition 2; R1 and R2 are set with different radii according to the requirements of different products and partitions are added;

[0021] Step S242, Foreign object quantity statistics: After completing the partition, calculate the Euclidean distance D n of the location of each foreign object. If D n is less than R1, the foreign object belongs to partition 1. If D n is greater than R1 and less than R2, it belongs to partition 2. If D n is greater than R2, it belongs to partition 3; Count the number of foreign objects in the three partitions. The number of foreign objects in partition 1 is T1, the number of foreign objects in partition 2 is T2, and the number of foreign objects in partition 3 is T3;

[0022] Step S243, Specification determination based on the number of foreign objects: For a display module where the number of foreign objects in each partition is less than the set threshold, it is determined as a good product. Otherwise, it is determined as a defective product.

[0023] The present invention also provides a computer storage medium. The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the above-mentioned method for detecting foreign objects in a high-PPI VR display module.

[0024] The present invention also provides an electronic device. Among them, the electronic device includes: a processor; and a memory arranged to store computer-executable instructions, and when the executable instructions are executed, the processor executes the above-mentioned method for detecting foreign objects in a high-PPI VR display module.

[0025] Advantages of the present invention:

[0026] The present invention is ingeniously designed, and proposes a detection method for foreign objects in a high-PPI VR display module that integrates YOLO, attention mechanism, and Swin-Transformer, significantly improving the efficiency and accuracy of foreign object detection; for display module products with foreign objects, a method for measuring and determining size specifications is proposed, and products that do not affect display performance and reliability are determined as good products, avoiding misjudgment and improving efficiency; using partition statistics to separately count the number of foreign objects in each area, and controlling the number of foreign objects according to requirements differences in different partitions, improving efficiency while ensuring product quality; the present invention can achieve efficient and accurate detection of foreign objects in a high-PPI VR display module, improving production efficiency and quality control level; at the same time, according to the control requirements of near-eye display products, a method for measuring and determining foreign object size specifications and a method for determining the number of foreign objects by partition statistics are invented; through the present invention, efficient and accurate detection of foreign objects in a high-PPI VR display module is achieved, improving the efficiency and benefits of the industry. Description of the Drawings

[0027] Figure 1 It is a flowchart of a method for detecting foreign objects in a high-PPI VR display module according to the present invention.

[0028] Figure 2 It is a logical structure diagram of a method for detecting foreign objects in a high-PPI VR display module according to the present invention.

[0029] Figure 3 It is a partition principle diagram of partition determination according to the present invention. Detailed Embodiments

[0030] For the convenience of understanding by those skilled in the art, the present invention will be further described below in conjunction with embodiments and drawings. The content mentioned in the embodiments is not a limitation to the present invention. The present invention will be described in detail below with reference to the drawings.

[0031] Embodiment 1

[0032] Embodiment 1 of the present application provides a method for detecting foreign objects in a high-PPI VR display module, as Figures 1 to 3 shown, which includes the following steps:

[0033] Step S110, Dataset Collection: Use multiple pure-color images as test display content. These pure-color images will be displayed on the screen of the same display module with different background colors; in order to effectively identify and highlight the characteristics of these foreign objects, use multiple pure-color images as test display content. These pure-color images will be displayed on the screen of the same display module with different background colors, thereby enhancing the visibility of foreign objects; by applying five different pure-color images of red, green, blue, white, gray, and black, foreign objects can present obvious contrast effects under these backgrounds, effectively eliminating background interference. By comparing the performance of foreign objects under different pure-color backgrounds, the subsequent detection model can better identify various foreign objects.

[0034] Step S120, Dataset Annotation: After the dataset collection is completed, it is necessary to classify and label the regions of the foreign objects in the collected images; in terms of foreign object classification, according to the characteristics of different foreign objects, they are classified into dust, bubbles, stains, and other foreign objects that may affect the display effect.

[0035] Step S130, Model Training: Use a deep learning model that includes Backbone, Neck, and Head. This deep learning model adopts the single-stage structure of YOLO; among them, Backbone uses the outputs of Stage 2 to Stage 5 of the ResNet-50 residual network to extract the basic features of the input image; since each Stage concatenates the features of the previous Stage, before the features are passed to Neck, a convolutional network is used for deep feature fusion; the CBAM attention mechanism is used to weight the fused features, reduce the weights of redundant and useless features, and enhance the weights of important features to highlight key features. The features weighted by the CBAM attention mechanism will be passed into Neck. In Neck, the feature extraction method of traditional YOLO is adopted, and a Swin-Transformer feature extraction branch is introduced; the fusion of these two branches forms the C3STR structure, thereby further enhancing the feature extraction ability; finally, the output features of Neck will be passed to the Detect detection network, and a threshold algorithm is used to filter out the detection results that meet the threshold range; to ensure the accuracy of the detection results, a threshold algorithm is used to filter out the detection results that meet the threshold range; with the cooperation of this series of steps, this model can effectively perform foreign object detection and improve the accuracy and robustness of the detection.

[0036] Step S210, Image Acquisition: Use an industrial camera to acquire the real-time display image of the VR display module. The image in the image acquisition in Step S210 includes images under five pure-color images of red, green, blue, white, gray, and black.

[0037] Step S220, foreign object detection: Transmit the collected image into the trained model for foreign object detection;

[0038] Step S230, foreign object size calculation: Calculate the actual size represented by each pixel in the image, and determine whether the foreign object size meets the specifications by comparing with the actual pixel quantity of the foreign object;

[0039] Step S240, partition and count the number of foreign objects: Partition the display area, count the number of foreign objects in each partition, and determine whether the display module is a good product according to the set threshold.

[0040] Specifically, the embodiment of the present application is ingeniously designed, and a detection method for foreign objects displayed on a high-PPI VR display module integrating YOLO, attention mechanism, and Swin-Transformer is proposed, which significantly improves the efficiency and accuracy of foreign object detection; a size specification measurement and determination method is proposed for the display module products with foreign objects, and the products that do not affect the display performance and reliability are determined as good products to avoid misjudgment and improve efficiency; the number of foreign objects in each area is statistically counted by using partition statistics, and the number of foreign objects is controlled in different partitions according to the requirement differences, which improves the efficiency and ensures the quality of the products at the same time; the embodiment of the present application can realize efficient and accurate detection of foreign objects displayed on a high-PPI VR display module, and improve the production efficiency and quality control level; at the same time, according to the control requirements of near-eye display products, a method for measuring and determining the size specification of foreign objects and a method for determining the number of foreign objects by partition statistics are invented; through the present invention, efficient and accurate detection of foreign objects displayed on a high-PPI VR display module is realized, and the efficiency and benefits of the industry are improved.

[0041] In the embodiment of the present application, the results of the foreign object size calculation and the partition and count of the number of foreign objects are jointly used to determine whether the display module is a good product.

[0042] In the embodiment of the present application, the specific training steps of the model training are as follows:

[0043] S131: Training strategy: During the model training process, by selecting appropriate hyperparameters, such as learning rate, batch size, etc.; through multiple rounds of training iterations, the model will gradually learn how to extract features, classify, and identify different foreign objects; in this process, it is also very important to adopt techniques such as cross-validation and early stopping, which can not only prevent the model from overfitting, but also improve its reliability in actual application scenarios.

[0044] S132: Training and Evaluation: To evaluate the training effect, the performance of the model on the validation set and the test set will be regularly tested, and key metrics such as accuracy, recall, and F1-score will be observed; these metrics will reflect the performance of the model, and necessary adjustments will be made according to the evaluation results to further optimize the performance of the model; after completing the model training, detailed testing and verification will be carried out to ensure that the model can operate effectively in the actual production environment; this includes evaluating the inference speed of the model to ensure that it can meet the real-time detection requirements in practical applications; at the same time, the stability and robustness of the model are also aspects that need to be focused on to prevent performance degradation under different environmental conditions.

[0045] In the embodiments of the present application, the steps of calculating the foreign object size in step S230 include: Step S231, Characterizing the pixel size of the calibration board image: Using the known grid size of the calibration board and the camera calibration parameters, calculate the ratio of pixel to actual size conversion, and calculate the actual size corresponding to the pixel length through the distance of the detected corner points in the image: Wherein, L is the actual size of a grid of the calibration board, and N is the number of pixels of the calibration board grid in the collected image;

[0046] Step S232, Calculating the foreign object size based on the detection result: According to the number n of pixel points where the foreign object is located calculated from the detection result, calculate the size of the foreign object, and the calculation method is as follows: A = S * n; Compare the calculated foreign object size A with the Pixel size x of the display module, and x is set according to the characteristics of different products; The determination principle is as follows:

[0047] For the display module that meets the specifications, the number of foreign objects in each partition is statistically analyzed to determine whether the number of foreign objects in each partition is within the specifications.

[0048] In the embodiments of the present application, the steps of statistically analyzing the number of foreign objects in step S240 are as follows:

[0049] Step S241, Using the OTSU algorithm to segment the display area from other areas to obtain the display area; then, use the moments algorithm of Opencv to solve the moment M of the contour, and calculate the center coordinates in the following way: After obtaining the center coordinate point O(c x ,c y ), respectively set R1 and R2 as the region radii of partition 1 and partition 2; R1 and R2 are set with different radii according to the requirements of different products and additional partitions are added;

[0050] Step S242, Counting the number of foreign objects: After completing the partition division, calculate the Euclidean distance D n of the location of each foreign object, if D nIf it is less than R1, the foreign object belongs to Zone 1. If D n is greater than R1 and less than R2, it belongs to Zone 2. If D n is greater than R2, it belongs to Zone 3; count the number of foreign objects in the three zones. The number of foreign objects in Zone 1 is T1, the number of foreign objects in Zone 2 is T2, and the number of foreign objects in Zone 3 is T3;

[0051] Step S243, Specification determination based on the number of foreign objects: For a display module where the number of foreign objects in each zone is less than the set threshold, it is determined as a good product; otherwise, it is determined as a defective product.

[0052] Embodiment 2

[0053] Embodiment 2 of the present application provides a computer storage medium. The computer storage medium stores computer instructions, which are used to execute the method for detecting foreign objects in a high-PPI VR display module when the computer instructions are called.

[0054] Embodiment 3

[0055] Embodiment 3 of the present application provides an electronic device. The electronic device includes: a processor; and a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute the method for detecting foreign objects in a high-PPI VR display module.

[0056] As described above, it is only a preferred embodiment of the present invention and does not impose any form of limitation on the present invention. Although the present invention is disclosed above in a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art, without departing from the scope of the technical solution of the present invention, when making some changes or modifications using the above-disclosed technical content as equivalent change equivalent embodiments, but as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change and modification made to the above embodiments according to the technical means of the present invention all fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting display foreign objects in a high-PPI VR display module, characterized in that, Including the following steps: Step S110, dataset collection: Use multiple pure-color images as the test display content, and these pure-color images will be displayed on the screen of the same display module with different background colors; Step S120, dataset annotation: After the dataset collection is completed, it is necessary to classify and label the foreign objects in the collected images with bounding boxes; Step S130, model training: Use a deep learning model including Backbone, Neck, and Head. This deep learning model adopts the single-stage structure of YOLO; among them, Backbone uses the outputs of Stage 2 to Stage 5 of the ResNet-50 residual network to extract the basic features of the input image; before the features are passed to Neck, a convolutional network is used for deep feature fusion; the CBAM attention mechanism is used to highlight important features and eliminate redundant features. The features weighted by the CBAM attention mechanism will be passed into Neck. In Neck, the feature extraction method of traditional YOLO is adopted, and the Swin-Transformer feature extraction branch is introduced; Finally, the output features of Neck will be passed to the Detect detection network, and a threshold algorithm is used to filter out the detection results that meet the threshold range; Step S210, image acquisition: Use an industrial camera to acquire the frame images of the VR display module in real-time display; Step S220, foreign object detection: Pass the collected images into the trained model for foreign object detection; Step S230, foreign object size calculation: Calculate the actual size represented by each pixel in the image. By comparing with the actual pixel number of the foreign object, determine whether the foreign object size meets the specifications; Step S240, partition and count the number of foreign objects: Partition the display area, count the number of foreign objects in each partition, and determine whether the display module is a good product according to the set threshold.

2. The foreign object detection method for a high-PPI VR display module according to claim 1, wherein: The frame images in the image acquisition in step S210 include images under five pure-color images of red, green, blue, white, gray, and black.

3. A method for detecting foreign objects in a high-PPI VR display module according to claim 1, characterized in that: The results of the foreign object size calculation and the partition and count of the number of foreign objects are jointly used to determine whether the display module is a good product.

4. A method for detecting foreign objects in the display of a high-PPI VR display module according to claim 1, characterized in that, The steps of the foreign object size calculation in step S230 include: Step S231: Characterize the pixel size of the calibration board image: Using the known grid size of the calibration board and the camera calibration parameters, calculate the ratio of pixel-to-actual size conversion, and calculate the actual size corresponding to the pixel length through the distance of the detected corner points in the image: where L is the actual size of a grid of the calibration board, and N is the number of pixels of the calibration board grid in the acquired image; Step S232, foreign object size calculation based on the detection result: According to the number n of pixel points where the foreign object is located calculated from the detection result, calculate the size of the foreign object. The calculation method is as follows: A = S * n; Compare the calculated foreign object size A with the Pixel size x of the display module, and x is set according to the characteristics of different products; The determination principle is as follows: For the display modules that meet the specifications, the foreign object quantity is counted by partition, and it is determined whether the foreign object quantity in each partition is within the specifications.

5. A method for detecting foreign objects in the display of a high-PPI VR display module according to claim 1, characterized in that, The steps of the partition and count of the number of foreign objects in step S240 are: Step S241: Use the OTSU algorithm to segment the display area from other areas to obtain the display area. Then, use the moments algorithm in Opencv to solve the moments M of the contour, and calculate the center coordinates as follows: After obtaining the center coordinate point O(c x ,c y ), set R1 and R2 as the regional radii of partition 1 and partition 2 respectively; R1 and R2 are set with different radii according to the requirements of different products and partitions are added; Step S242, foreign object quantity statistics: After completing the partition division, calculate the Euclidean distance D of the location of each foreign object n , if D n is less than R1, the foreign object belongs to partition 1. If D n is greater than R1 and less than R2, it belongs to partition 2. If D n is greater than R2, it belongs to partition 3; Count the number of foreign objects in the three partitions. The number of foreign objects in partition 1 is T1, the number of foreign objects in partition 2 is T2, and the number of foreign objects in partition 3 is T3; Step S243, specification determination based on the number of foreign objects: A display module with the number of foreign objects in each partition less than the set threshold is determined to be a good product, otherwise, it is determined to be a defective product.

6. A computer storage medium storing computer instructions that, when called, are used to execute a method for detecting foreign objects in a high-PPI VR display module as described in any one of claims 1-5.

7. An electronic device, wherein, The electronic device includes: a processor; and a memory arranged to store computer-executable instructions which, when executed, cause the processor to execute a method for detecting foreign objects in a high-PPI VR display module as described in any one of claims 1-5.

Images