XR glasses detection device and method

Abstract

The present invention discloses a detection device and method for XR glasses, which belongs to the field of image processing technology. The method comprises the following steps: projecting a white field image in the XR glasses and collecting a brightness image; preprocessing the brightness image, extracting the ROI area of ​​the XR glasses display area in the brightness image, and obtaining the ROI area image; according to the brightness image, generating a simulated Mura image training set through a data simulation module and training a segmentation detection model; inputting the ROI area image into the segmentation detection model, and the segmentation detection model outputs the segmented detection result. Compared with the prior art, the benefit of the present invention lies in that a data simulation method for interlacing and overlapping single Mura is proposed, which can more effectively simulate the complex situations in actual detection. The segmentation detection model trained by this method can significantly improve the detection accuracy of Mura defects in the display quality of XR glasses, making the detection results more accurate and reliable, and having higher precision.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a detection device and method for XR glasses. Background Art

[0002] High-quality user experience is inseparable from the strict control of the various performances of XR glasses. During the manufacturing and actual use process, affected by various factors such as the characteristics of electronic components, the precision of production processes, and the control of the working environment, XR glasses will inevitably encounter various defects, among which the uneven brightness on the display interface is particularly prominent, and Mura is one of them. Mura refers to the obvious brightness or color difference between the red and green colors perceived by the user under the same light source and the same background color. This unevenness seriously undermines the overall beauty and consistency of the picture. The Mura defect in the display screen of XR glasses is not only directly related to the evaluation of other key optical properties of the glasses, such as contrast, color saturation, etc., but also directly affects the user's viewing experience and immersion. In severe cases, it may even cause health problems such as visual fatigue and dizziness, which will have an adverse effect on the long-term use of users. Therefore, it is particularly important to conduct efficient and accurate detection of Mura defects in the display quality of XR glasses. On the one hand, this helps manufacturers objectively evaluate the brightness uniformity of glasses and provide data support for product optimization; on the other hand, through strict quality control during the production process, products with Mura defects can be effectively eliminated before they are launched on the market, ensuring that every pair of XR glasses can provide an excellent user experience and avoid damage to the brand image and user trust due to product quality issues.

[0003] For the current Mura detection of XR glasses display quality, most of its detection methods are derived from the Mura detection of display screens. The two have similarities to a certain extent, but there are still differences in the specific implementation. The detection methods of Mura are mainly divided into traditional image processing algorithms and image algorithms based on deep learning. Traditional image algorithms are mainly based on two ideas: background reconstruction and feature expression. The background reconstruction idea is implemented by combining curve fitting, matrix decomposition and threshold segmentation. The image algorithm based on deep learning is mainly based on supervised learning segmentation detection, but the defect data is extremely scarce at present, and the supervised learning model is difficult to drive. In addition, the Mura data simulations currently available on the Internet basically expand the defect data set by cutting out the screen, and this method only changes the low-level features such as the location and background of the defect. The generalization ability of its detection model is limited, and it is only suitable for simple Mura defect scenes. For complex, overlapping, and varied Mura defects, its detection effect is poor. In the relevant patents, Chinese patent document CN105913419A discloses a TFT-LCD Mura defect detection method based on ICA learning and multi-channel fusion. It first uses the unsupervised learning method of independent component analysis to generate the gray domain image basis and S domain image basis of the defect-free image, and then divides the image to be tested into small blocks and projects them onto the trained image basis for background reconstruction. Finally, the differential image is fused to achieve Mura detection. This solution has a high false detection rate for complex backgrounds and is easily affected by noise. It is implemented based on the feature expression idea in the form of gray feature statistics, frequency domain feature statistics or local feature detection.

[0004] From the above, it can be seen that the relevant technology does not provide any technical inspiration on how to test real XR defect images. Summary of the invention

[0005] 1. Technical problems to be solved

[0006] In view of the problem of how to improve the accuracy of detecting Mura defects in the display quality of XR glasses in the prior art, the present invention provides a detection device and method for XR glasses, which can improve the accuracy of the detection results of Mura in the display quality of XR glasses by training a segmentation detection model, combining the data simulation of single Mura interspersed and overlapped with the image segmentation network.

[0007] 2. Technical solution

[0008] The purpose of the present invention is achieved through the following technical solutions.

[0009] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0010] Some embodiments of the present application propose a detection device and method for XR glasses to solve the technical problems mentioned in the above background technology section.

[0011] As a first aspect of the present application, some embodiments of the present application provide a method for detecting XR glasses, comprising the following steps: projecting a white field image into the XR glasses and acquiring a brightness image; preprocessing the brightness image, extracting the ROI area of ​​the XR glasses display area in the brightness image, and obtaining an ROI area image; based on the brightness image, generating a simulated Mura image training set through a data simulation module and training a segmentation detection model; inputting the ROI area image into the segmentation detection model, and the segmentation detection model outputs the segmented detection result.

[0012] Furthermore, when projecting a white field image, the resolution of the white field image is consistent with the display resolution of the XR glasses, and the white field image completely covers the display area of ​​the XR glasses.

[0013] Furthermore, preprocessing includes filtering, denoising, cropping and scaling the brightness image.

[0014] Furthermore, the process of cropping the brightness image is specifically as follows: according to the display area and resolution of the XR glasses, the range of the ROI area is determined, and the ROI area is cropped out on the denoised brightness image.

[0015] Furthermore, the process of scaling the brightness image is specifically as follows: scaling the ROI region image to a square image of a fixed scale, where the number of pixels on any side of the square image ranges from 32 to 256.

[0016] Furthermore, the simulation process for generating a simulated mura image training set is specifically as follows: randomly generating a number of curves in a base image, and randomly selecting a number of points on each curve at a fixed interval as the center position of a single mura defect; determining the mura type of the overlapping area of ​​a single mura according to a contrast threshold interval, and fusing the single mura with the brightness background base image to generate a simulated mura image and a mask position corresponding to the single mura; and obtaining a simulated mura image training set by repeatedly executing the simulation process.

[0017] Furthermore, brightness images obtained after projecting white field images are collected from a number of XR glasses, and a brightness image is randomly selected as the base image.

[0018] Furthermore, based on the generation of simulated mura defect images, the mask position corresponding to a single mura The expression is:

[0019] ;

[0020] ( , ) is the pixel coordinate, is the base weight sampler, is the contrast threshold.

[0021] Furthermore, the basic weight sampler The expression is:

[0022] ;

[0023] is the overall weighting coefficient, which represents the brightness amplitude and is used to control the brightness of a single mura; is the weight base value, ranging from [1, 3]; is a constant compensation term, with a value range of [0, 2]; ( , ) represents the peak coordinate of the basic weight sampler; , and are all hyperparameters.

[0024] As a second aspect of the present application, some embodiments of the present application provide a detection device for XR glasses, characterized in that it includes an acquisition module: projecting a white field image in the XR glasses and acquiring a brightness image; a preprocessing module: preprocessing the brightness image, extracting the ROI area of ​​the XR glasses display area in the brightness image, and obtaining the ROI area image; a training module: based on the brightness image, generating a simulated Mura image training set through a data simulation module and training a segmentation detection model; a detection module: inputting the ROI area image into the segmentation detection model, and the segmentation detection model outputs the segmented detection result.

[0025] 3. Beneficial effects

[0026] Compared with the prior art, the advantages of the present invention are: a detection device and method for XR glasses of the present invention propose a data simulation method for single Mura interlacing and overlapping, which can more effectively simulate the complex situations in actual detection. The segmentation detection model trained by this method can significantly improve the detection accuracy of Mura defects in the display quality of XR glasses, making the detection results more accurate and reliable, with higher precision; the detection device and method of the present invention are not only suitable for specific types of XR glasses, but can also be flexibly applied to a variety of different types of XR glasses products. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 1 is a flow chart of a method for detecting XR glasses according to an embodiment of the present invention;

[0028] Figure 2 1 is a flow chart of a method for detecting XR glasses in another embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of Mura defect decomposition of a real image of XR glasses in one embodiment of the present invention;

[0030] Figure 4 Schematic diagram of the XR glasses image Mura simulation process in one embodiment of the present invention;

[0031] Figure 5 Schematic diagram of basic weights of pixels based on a two-dimensional image in one embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0033] Combination Figures 1 to 5 , a method for detecting XR glasses of the present invention comprises the following steps:

[0034] Project a white field image in the XR glasses and collect brightness images;

[0035] Preprocess the brightness image, extract the ROI area of ​​the XR glasses display area in the brightness image, and obtain the ROI area image;

[0036] Based on the brightness image, a simulated mura image training set is generated through the data simulation module and a segmentation detection model is trained;

[0037] The ROI area image is input into the segmentation detection model, and the segmentation detection model outputs the segmented detection result.

[0038] In a specific embodiment, the specific process of the detection method of XR glasses is as follows:

[0039] S1. Image acquisition

[0040] A white field image matching the resolution of the XR glasses is projected into the glasses, and then the brightness image is captured by an imaging brightness meter.

[0041] The brightness image is the image data obtained after the white field image is collected by an imaging brightness meter. The brightness image reflects the brightness data of the white field image. The white field image exists physically, that is, the input image; and the brightness image is the output data, which is the result after collection. More precisely, the white field image serves as the benchmark for subsequent image collection. Its resolution is consistent with the display resolution of the XR glasses, and the white field image completely covers the display area of ​​the XR glasses; the brightness image is the brightness information of the white field image collected by a high-precision imaging brightness meter. The brightness image contains detailed data of the XR glasses display interface at different positions and different brightnesses.

[0042] Specifically, a white field image that matches the resolution of the XR glasses is projected; the brightness information of the white field image is then collected through an imaging luminance meter to ensure that detailed data of the glasses' display interface at different positions and different brightnesses can be obtained to obtain a white field brightness image.

[0043] In a specific embodiment, the process of collecting images is as follows:

[0044] Before capturing images, first calibrate the display system of the XR glasses to ensure the accuracy and consistency of the display effect. The calibration process includes adjusting parameters such as display brightness, color balance, and contrast. By calibrating the display system of the XR glasses, you can ensure that the display brightness, color balance, and contrast of the XR glasses are in the optimal state before capturing white field brightness images, thereby reducing the acquisition errors caused by inconsistent display system parameters.

[0045] Prepare a white field image that matches the resolution of the XR glasses. A white field image is a completely white image, that is, an image with consistent grayscale values ​​for each pixel, which serves as a benchmark for subsequent image acquisition. A white field image that matches the resolution of the XR glasses means that the resolution of the white field image is consistent with the display resolution of the XR glasses. This ensures that the white field image can completely cover the display area of ​​the XR glasses, and the theoretical brightness values ​​at different positions remain consistent, ensuring that comprehensive and accurate brightness data is obtained during the acquisition process. A white field image that completely covers the display area can ensure that the acquired brightness data is representative and can truly reflect the overall display performance of the XR glasses.

[0046] An imaging luminance meter is placed at a position capable of capturing the entire image on the display screen of the XR glasses, the imaging luminance meter is started and luminance data of the white field image is collected; the collected luminance information is recorded in a computer or other storage device to obtain a luminance image having luminance data of the white field image.

[0047] Specifically, the brightness information includes brightness value, position information (such as pixel coordinates) and additional information (such as timestamp and device identification, etc.).

[0048] S2. Preprocess the collected brightness image

[0049] The collected brightness image is preprocessed to extract the ROI area of ​​the XR glasses display area in the brightness image.

[0050] Specifically, the brightness image is filtered, denoised, and cropped to extract the ROI area of ​​the XR glasses display area and remove the invalid background area at the edge. Since there are many types of XR glasses and the resolution size distribution is not concentrated, the ROI area of ​​the image is scaled to a square image of a fixed scale with any resolution to obtain an ROI area image to adapt to Mura detection of various XR glasses.

[0051] Through filtering and denoising processing, noise interference in brightness images, such as Gaussian noise and salt and pepper noise, can be effectively reduced, which helps to improve the clarity and contrast of the image; at the same time, the details and edge information of the image are maintained, and information loss caused by excessive smoothing is avoided, which can ensure the integrity and authenticity of the image.

[0052] Preprocessing includes filtering, denoising and cropping the brightness image to extract the ROI (Region of Interest) of the image and remove the invalid background area at the edge.

[0053] Due to the wide variety of XR glasses with different resolutions, in order to unify the processing standards, the ROI area image is scaled to a fixed-scale square image of any resolution. A fixed-scale square image of any resolution means scaling the ROI area brightness image to N×N, where N represents the number of pixels on one side of the image, and the number of pixels on any side of the square image ranges from 32 to 256, that is, the range of N×N is 32×32 to 256×256. This ensures that different XR glasses can use the same algorithm and parameters when performing Mura detection, thereby improving the versatility and practicality of the algorithm.

[0054] In a specific embodiment, the preprocessing process is as follows:

[0055] Filtering and denoising:

[0056] Perform noise analysis on the acquired brightness image to determine the type and degree of noise; select the corresponding filtering algorithm for denoising based on the noise type and degree of the brightness image.

[0057] Specifically, for Gaussian noise, a Gaussian filter is used. The Gaussian filter replaces the value of each pixel with the weighted average of its surrounding pixels by weighted averaging, thereby smoothing the image and reducing Gaussian noise. For salt and pepper noise, a median filter is used. The median filter calculates the median of the pixel values ​​around each pixel and assigns the median to the current pixel, thereby effectively removing salt and pepper noise.

[0058] When applying the filtering algorithm, the filter parameters (such as window size and standard deviation, etc.) are adjusted according to the severity of the noise and the detail requirements of the image, so as to ensure that the details and edge information of the white field image are maintained while removing the noise, and avoid information loss due to excessive smoothing.

[0059] Through filtering and denoising processing, the noise interference in the image can be effectively reduced, the image quality can be improved, more accurate image data can be provided for subsequent processing steps, and the accuracy and reliability of the algorithm can be improved; at the same time, selecting appropriate filtering algorithms and parameters according to the type and degree of noise can maintain the details and edge information of the image and avoid information loss caused by excessive smoothing.

[0060] Crop ROI area:

[0061] According to the display area and resolution of the XR glasses, the range of the ROI area is determined, and the ROI area is cropped out on the denoised brightness image.

[0062] Specifically, a geometric transformation is performed on the denoised brightness image to ensure that the ROI area completely corresponds to the display area of ​​the XR glasses.

[0063] Use an image cropping tool or algorithm to crop out the ROI region. During the cropping process, ensure that the ROI region includes all display information while avoiding any background area.

[0064] Cropping out the ROI area of ​​the XR glasses display area in the brightness image can remove the invalid background area at the edge, thereby reducing the amount of calculation for subsequent processing and improving processing efficiency. At the same time, ensuring that the ROI area completely corresponds to the display area of ​​the XR glasses can ensure the accuracy and reliability of subsequent analysis.

[0065] Image Scaling:

[0066] The ROI area image is uniformly scaled to a fixed scale of N×N.

[0067] Specifically, during the image scaling process, the aspect ratio of the image is maintained to avoid image deformation or distortion.

[0068] More specifically, image scaling uses algorithms such as bilinear interpolation and bicubic interpolation for scaling. The bilinear interpolation algorithm estimates the value of new pixels by calculating linear interpolation in two directions, which is suitable for most image scaling scenarios; the bicubic interpolation algorithm estimates the value of new pixels by calculating cubic interpolation in four directions, which can provide a smoother image scaling effect, but the amount of calculation is relatively large.

[0069] Scaling the ROI area image to a fixed scale can eliminate the impact of different XR glasses’ resolution differences on subsequent processing. This improves the algorithm’s versatility and practicality by unifying the processing standards. At the same time, the image’s aspect ratio is maintained to avoid image deformation or distortion, ensuring that the scaled image is still representative.

[0070] S3. Training and testing

[0071] A simulated Mura image training set is generated through a data simulation module and a segmentation detection model is trained; the ROI area image is input into the segmentation detection model, and the segmentation detection model outputs the segmented detection result.

[0072] Specifically, the segmentation detection model is trained, and a simulation data set is obtained through a data simulation module for training and debugging; the preprocessed ROI area image is input into the segmentation detection model, and the segmentation detection model outputs the detection result after segmentation.

[0073] The data simulation module simulates overlapping Mura defect scenes based on the characteristics of the real Mura defect images of XR glasses and generates a simulation data set. The segmentation detection model is based on deep learning technology and is trained and optimized through the simulation data set generated by the data simulation module. The segmentation detection model can accurately identify Mura defects in the image and output the detection results, including the number, size, center position, confidence level and segmentation effect diagram of Mura.

[0074] By generating a simulation data set through the data simulation module, we can overcome the limitations of the number and quality of real images and generate a large number of rich and diverse Mura defect images. These simulation images can truly reflect the characteristics and distribution patterns of Mura defects, providing strong data support for model training. At the same time, the generation of simulation data sets can also reduce the dependence on real images and improve the efficiency and flexibility of model training. By training the segmentation detection model for Mura defect detection, we can use the powerful feature extraction and classification capabilities of deep learning technology to improve the accuracy and robustness of detection. At the same time, the segmentation detection model can also output detailed detection results, including the number, size, center position, confidence, and segmentation effect diagram of Mura, providing a strong basis for subsequent analysis and repair.

[0075] Specifically, after the segmentation detection model completes the segmentation, the detection results output include: the number, size, center position, confidence level and segmentation effect diagram of the mura.

[0076] like Figure 3 As shown, the process of constructing a data simulation module is: according to the real Mura defect image of XR glasses, the data simulation module is composed of a uniform background area, a dark area, and a bright area.

[0077] Specifically, in the real Mura defect image of XR glasses, the brightness of the background area is determined. The dark area refers to the area darker than the background area, and the bright area refers to the area brighter than the background area. The dark area and the bright area are formed by the random overlap of several single Mura of the same type, and the characteristics of a single Mura are mainly determined by brightness, contrast, shape, size, position and edge blur.

[0078] like Figure 4 As shown, the data simulation steps in the data simulation module include: randomly generating a number of curves in the base image, and randomly selecting a number of points on each curve at a fixed interval as the center position of a single Mura defect; determining the Mura type of the overlapping area according to the contrast threshold interval, and generating a corresponding mask label; and obtaining a simulated Mura image training set by repeatedly executing the above simulation program.

[0079] In a specific embodiment, the data simulation process of the data simulation module is as follows:

[0080] Acquire the basement image:

[0081] Collect luminance images of uniform scale (such as Z*Z) from several XR glasses to ensure that the collected luminance images cover XR glasses of different styles and resolutions;

[0082] A brightness image is randomly selected as the base image for subsequent Mura defect simulation.

[0083] Generate Mura defects:

[0084] Several curves are randomly generated on the substrate image, and several points are randomly selected at fixed intervals on each curve, and these points are used as the center positions of single Mura defects.

[0085] Specifically, the curve includes a straight line, a curve or a broken line, which is used to simulate the distribution path of the Mura defect.

[0086] Generate the brightness distribution of a single mura using the 2D coordinates and weight sampler w;

[0087] Specifically, according to the brightness amplitude , weight base value , constant compensation term B and quadratic term coefficient , , , determine the brightness, shape and location of individual mura.

[0088] M groups of offsets and weighting coefficients are randomly generated to generate multiple different basic weighters for fusion to ensure that the brightness value and shape of a single Mura area are different.

[0089] Based on the contrast threshold range (determined according to IEC standards and the background brightness average), the overlapping areas of a single mura are judged as dark mura, background, or light mura.

[0090] Generate a training set of simulated mura images:

[0091] All generated single Mura are fused with the brightness background image to obtain a simulated Mura defect image. According to the generation process of the simulated Mura defect image, mask labels corresponding to all single Mura are generated for subsequent segmentation detection model training.

[0092] The above simulation program is repeatedly executed to generate multiple simulated Mura images, which eventually constitute a simulation data set. The simulation program is repeatedly executed until a preset number of simulated Mura images is reached.

[0093] Specifically, the types of mura include bright mura and dark mura. Several points are randomly selected on each curve at intervals, and the selected points are used as the center positions of single mura defects. The single mura defects generated by each curve belong to the same category; the overlapping parts between single mura are determined as dark mura, background or bright mura according to the contrast threshold range.

[0094] Compared with the method of using cutout screen replacement, the mura simulation image generated in this way not only simulates the low-level features of mura defects such as the location and background, but also can simulate complex overlapping and changing mura defects.

[0095] According to the IEC (International Electrotechnical Commission) standard on the relationship between contrast difference and area of ​​Mura, the contrast difference of the overlapping area is obtained. Since the background brightness base image selected by the XR glasses image is known, the background brightness pixel mean can be calculated. According to the contrast difference, the pixel area to be defined and the background brightness mean, the contrast threshold interval T (approximately ±11 brightness pixel difference) can be determined.

[0096] Through the data simulation module, we can simulate a variety of overlapping Mura defect scenes based on the characteristics of real Mura defect images of XR glasses and generate rich simulation data sets. These simulation data sets can truly reflect the diversity of Mura defects and provide strong data support for the training of segmentation detection models.

[0097] like Figure 4 As shown in Figure 1, the generation process of a single Mura includes: first, according to the two-dimensional coordinates, a basic weight sampler of a pixel point is implemented. , Sampler The expression is:

[0098]

[0099] in, is the contrast threshold, The value range of is the contrast threshold interval T; It is the overall weighting coefficient, which indicates the brightness amplitude and is used to control the brightness of a single Mura.

[0100] is the weight base value, ranging from [1, 3]; is a constant compensation term, and its value range is [0, 2];

[0101] ( , ) is the pixel coordinate; ( , ) is the peak coordinate of the basic weight sampler, which can also be used as the center point coordinate of a single Mura area to control the position of a single Mura in a two-dimensional image;

[0102] , , are quadratic term coefficients, all of which are hyperparameters, and their values ​​represent the roundness of the two-dimensional shape of the weight region; , The greater the difference, the more elliptical the two-dimensional shape of the weight region. Used to adjust the tilt degree of the two-dimensional shape;

[0103] , , and brightness amplitude The value range of is determined by statistically analyzing the Mura defect features in the Mura images of XR glasses.

[0104] Specifically, the brightness amplitude Determined by the difference between the minimum brightness value, the maximum brightness value and the background brightness value of the Mura image, , The maximum coordinate of the range occupied by Mura ( , ) Scope determination; Can be converted to an angle, with a value range of [0, Π].

[0105] After the base weighter is generated, m groups of offsets are randomly generated , and weighting coefficients :

[0106] The parameter selection can be randomly selected in a certain range according to uniform distribution. The parameter range of the offset is 15~30 pixels; the weighting coefficient range is 0.6~0.94. The purpose of generating m groups of offsets and weighting coefficients is to generate multiple different basic weighters for fusion, on the one hand to ensure that the brightness value of a single Mura area is randomly different, and on the other hand to ensure that the shape of a single Mura area is different.

[0107] In a specific embodiment, the randomly generated Group offset , and weighting coefficients , expressed as follows:

[0108]

[0109] According to the offset and weighting coefficient, we get Group parameters, expressed as follows:

[0110]

[0111] according to Group parameters, generate Base Weight Sampler , expressed as follows:

[0112]

[0113] Based on the threshold The pixel weight coefficients are intercepted from the base weight sampler; Figure 5 As shown, it is a schematic diagram of the basic weight after fusion, and the peak value is calculated ( , ) coordinate mean as the center position of a single Mura; calculate In the group parameters ( , ) direction ( , ) is expressed as:

[0114]

[0115] like Figure 5 As shown in the figure, the horizontal axis represents the Y pixel position, the vertical axis represents the normalized grayscale intensity, and the vertical axis represents the X pixel position. The unit is pixel position; the brighter the color, the closer the value is to 1. The colors in the figure are only used to distinguish the areas and have no special meaning. The weight interception threshold is determined according to the value of the vertical axis. ;when The smaller the weight coefficient is, the closer it is to 0, and the better the transition of a single Mura edge is in the image. The larger the value, the stronger the layering between the single Mura edge and the background. Obviously, the more natural the edge transition is, the better. Generally, it is set to 0.0001. After the threshold is cut, the mask of a single Mura and the weight coefficient of the area can be obtained. Finally, all the single Mura areas are fused with the background brightness base image to obtain the simulated Mura defect image.

[0116] Mask location for a single mura The expression is:

[0117]

[0118] The expression for calculating the brightness pixel value of a single Mura area is:

[0119]

[0120] in, Indicates the brightness of the background area; Indicates the type of Mura of the background brightness. The value of is [-1, 1]. A value of -1 indicates dark Mura relative to the background brightness. A value of 1 indicates bright mura relative to the background brightness.

[0121] The segmentation detection model is based on the basic idea of ​​instance segmentation, and follows the encoding-decoding structure, focusing on the abstract expression of the shape, edge, texture and brightness distribution features of the input image. The input image is first encoded, and the encoder trunk performs four-layer downsampling to achieve feature extraction at different scales. At the same time, traditional image processing methods are used to extract basic texture features and statistical grayscale distribution feature vectors, and then merged with feature maps of different scales. The decoder part is responsible for restoring these feature maps to segmentation results of the same scale as the input image.

[0122] Specifically, the encoder of the segmentation detection model adopts a four-layer downsampling structure to achieve feature extraction at different scales.

[0123] The segmentation detection model uses traditional image processing methods to extract basic texture features and statistical grayscale distribution feature vectors; the extracted texture features and grayscale distribution feature vectors are fused with feature maps of different scales to enhance the model's ability to understand image features and achieve feature fusion; the fused feature image is restored to a segmentation result of the same scale as the input image through the decoder.

[0124] In a specific embodiment, the process of training the segmentation detection model is as follows:

[0125] The segmentation detection model performs data preprocessing on the simulation data set;

[0126] Use the simulated data set after data preprocessing to train the segmentation detection model, and optimize the model performance by adjusting hyperparameters such as learning rate and batch size;

[0127] Loss functions such as cross entropy loss and Dice loss are used to evaluate the difference between the model output and the true label and guide the update of the model parameters.

[0128] Specifically, data preprocessing includes data enhancement and normalization; data enhancement includes rotating, scaling, and flipping the simulation data set.

[0129] Using the generated simulation data set to train the segmentation detection model can enable the model to learn the characteristics and distribution patterns of Mura defects. Through continuous iteration and optimization, the segmentation detection model can gradually improve the accuracy and robustness of Mura defect identification, providing reliable model support for subsequent detection tasks.

[0130] In a specific embodiment, after the segmentation detection model is trained, the preprocessed ROI area image is input into the trained segmentation detection model; the segmentation detection model outputs the segmented detection results, including the number, size, center position, confidence and segmentation effect diagram of Mura.

[0131] Based on the output results of the segmentation detection model, the distribution, type and severity of Mura defects in the image can be intuitively understood.

[0132] By inputting the preprocessed ROI area image into the segmentation detection model, the image can be segmented and detected using the features and distribution rules learned by the model. The segmentation detection model can accurately identify the mura defects in the image and output detailed detection results, including the number, size, center position, confidence level, and segmentation effect diagram of the mura.

[0133] The test results output by the segmentation test model can clearly show the location, shape and quantity of Mura defects, providing a strong basis for subsequent fault analysis and repair. At the same time, the output of confidence can also reflect the reliability of the test results, helping users better understand and evaluate the test results.

[0134] Compared with the traditional method, which only outputs simple detection results such as the location and number of Mura defects, but lacks detailed morphology and confidence information, the segmented detection results output by this solution can provide more detailed and comprehensive Mura defect information, including morphology, size, center position, confidence, and segmentation effect diagram. This information can be used to better understand and evaluate the monitoring results, providing a more accurate basis for subsequent analysis and repair.

[0135] The method for XR glasses of the present invention proposes a data simulation method for single Mura interlacing and overlapping, which can more effectively simulate the complex situations in actual detection. The segmentation detection model trained by this method can significantly improve the detection accuracy of Mura defects in the display quality of XR glasses, making the detection results more accurate and reliable.

[0136] Specifically for the description of the device, the corresponding hardware implementation can be performed by referring to the corresponding software and method content in this embodiment; the corresponding device and equipment can be constructed;

[0137] For example, a device for detecting XR glasses includes a collection module: projecting a white field image in the XR glasses and collecting a brightness image;

[0138] Preprocessing module: preprocesses the brightness image, extracts the ROI area of ​​the XR glasses display area in the brightness image, and obtains the ROI area image;

[0139] Training module: Based on the brightness image, the data simulation module generates a simulated mura image training set and trains the segmentation detection model;

[0140] Detection module: The ROI area image is input into the segmentation detection model, and the segmentation detection model outputs the segmented detection result.

[0141] The device designed for detecting XR glasses may be a hardware device including one or more processors and a memory, wherein the memory stores a program and is configured to execute the following steps by the one or more processors:

[0142] Project a white field image in the XR glasses and collect brightness images;

[0143] Preprocess the brightness image, extract the ROI area of ​​the XR glasses display area in the brightness image, and obtain the ROI area image;

[0144] Based on the brightness image, a simulated mura image training set is generated through the data simulation module and a segmentation detection model is trained;

[0145] The ROI area image is input into the segmentation detection model, and the segmentation detection model outputs the segmented detection result.

[0146] The storage medium designed for the detection of XR glasses may be a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for calculating the brightness uniformity of the XR glasses is implemented.

[0147] The detection device and method of the present invention are not only applicable to specific types of XR glasses, but can also be flexibly applied to a variety of different types of XR glasses products. By adopting the detection device and method of the present invention, the image segmentation accuracy of Mura defects reached more than 97%, and the average intersection over union (IoU) reached more than 96%. These performance indicators fully demonstrate the significant advantages of the present invention in Mura defect detection, providing strong technical support for improving the display quality of XR glasses.

[0148] How to realize the device specifically can be realized in the existing technology, and no further detailed explanation will be given again. The following explains the possibility of corresponding realization in principle.

[0149] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many improvements to method flows today can be considered direct improvements to hardware circuit structures.

[0150] Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logic function is determined by the user's programming of the device. Designers program themselves to "integrate" a digital system on a PLD, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when writing program development, and the original code before compilation must also be written in a specific programming language, which is called Hardware Description Language (HDL), and there is not only one HDL, but many types, such as Verilog. Those skilled in the art should also be aware that it is easy to obtain a hardware circuit that implements the logical method flow by simply performing some logic programming on the method flow using the above-mentioned hardware description languages ​​and programming it into an integrated circuit.

[0151] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include but are not limited to the following microcontrollers: ATMEL AT89S52, microchip pic16c57. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0152] The systems, devices, modules or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices. For the convenience of description, the above devices are described separately by function into various units. Of course, when implementing this specification, the functions of each unit can be implemented in the same or more software and / or hardware.

[0153] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that instructions executed by the processor of a computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps of the functions specified in the blocks or multiple blocks. In a typical configuration, the computing device includes one or more processors (CPU), input / output interfaces, network interfaces and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0155] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0156] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0157] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0159] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A method for detecting XR glasses, comprising the following steps: Project a white field image in the XR glasses and collect brightness images; Preprocess the brightness image, extract the ROI area of ​​the XR glasses display area in the brightness image, and obtain the ROI area image; Based on the brightness image, a simulated mura image training set is generated through the data simulation module and a segmentation detection model is trained; The ROI area image is input into the segmentation detection model, and the segmentation detection model outputs the segmented detection result; The simulation process of generating the simulated mura image training set is as follows: randomly generate several curves in the base image, and randomly select several points on each curve at fixed intervals as the center position of a single mura defect; determine the mura type of the overlapping area of ​​a single mura according to the contrast threshold interval, and fuse the single mura with the brightness background base image to generate a simulated mura image and the mask position corresponding to the single mura; By repeatedly executing the simulation process, a simulated Mura image training set is obtained.

2. The method for detecting XR glasses according to claim 1, characterized in that: When projecting a white field image, the resolution of the white field image is consistent with the display resolution of the XR glasses, and the white field image completely covers the display area of ​​the XR glasses.

3. The method for detecting XR glasses according to claim 1, characterized in that: Preprocessing includes filtering, denoising, cropping, and scaling the brightness image.

4. The method for detecting XR glasses according to claim 3, characterized in that: The specific process of cropping the brightness image is as follows: according to the display area and resolution of the XR glasses, the range of the ROI area is determined, and the ROI area is cropped out on the denoised brightness image.

5. The method for detecting XR glasses according to claim 3, characterized in that: The process of scaling the brightness image is specifically as follows: scaling the ROI region image to a square image of a fixed scale, where the number of pixels on any side of the square image ranges from 32 to 256.

6. The method for detecting XR glasses according to claim 1, characterized in that: The brightness images obtained after projecting the white field image are collected from several XR glasses, and a brightness image is randomly selected as the base image.

7. The method for detecting XR glasses according to claim 1, characterized in that: Based on the generation of simulated mura defect images, the mask position corresponding to a single mura The expression is: ； ( , ) is the pixel coordinate, is the base weight sampler, is the contrast threshold.

8. The method for detecting XR glasses according to claim 7, characterized in that: Base Weight Sampler The expression is: ； is the overall weighting coefficient, which represents the brightness amplitude and is used to control the brightness of a single mura; is the weight base value, ranging from [1, 3]; is a constant compensation term, with a value range of [0, 2]; ( , ) represents the peak coordinate of the basic weight sampler; , and are all hyperparameters.

9. A detection device for XR glasses, characterized in that: include: Acquisition module: projects white field images in XR glasses and acquires brightness images; Preprocessing module: preprocesses the brightness image, extracts the ROI area of ​​the XR glasses display area in the brightness image, and obtains the ROI area image; Training module: Based on the brightness image, the data simulation module generates a simulated mura image training set and trains the segmentation detection model; Several curves are randomly generated in the base image, and several points are randomly selected at fixed intervals on each curve as the center position of a single Mura defect; the Mura type of the overlapping area of ​​a single Mura is determined according to the contrast threshold interval, and the single Mura is fused with the brightness background base image to generate a simulated Mura image and the mask position corresponding to the single Mura; By repeatedly executing the simulation process, a simulated Mura image training set is obtained; Detection module: The ROI area image is input into the segmentation detection model, and the segmentation detection model outputs the segmented detection result.

Images