XR glasses brightness uniformity processing device and method

Abstract

The present invention discloses a brightness uniformity processing device and method for XR glasses, which belongs to the field of image processing technology. The following steps are included: obtaining a brightness image and performing preprocessing; building and training an image segmentation network, inputting the brightness image into the image segmentation network for segmentation, and obtaining the feature vector of the brightness area; calculating the distance between the feature vectors of the brightness area, obtaining the brightness difference of the brightness area, and calculating the brightness uniformity result. Compared with the prior art, the advantages of the present invention are: it can accurately calculate the brightness uniformity of the display area of ​​the XR glasses, and obtain a more objective and reliable brightness uniformity evaluation result; and it has strong stability and anti-interference ability, reducing the calculation error caused by insufficient or uneven distribution of sampling points; it can also resist external interference such as light changes and equipment vibration to ensure stable results in complex environments.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a device and method for processing brightness uniformity of XR glasses. Background Art

[0002] At present, the brightness uniformity calculation method for XR glasses display quality basically follows the brightness uniformity test method of display panels, which mainly includes 9-point method, 23-point method and 135-point method. The sample data is collected point by point through optical equipment. As customers have higher and higher requirements for brightness uniformity testing, they even require the calculation of the brightness uniformity of the entire field of view. The brightness distribution of the entire field of view can be obtained by an imaging brightness meter, which can greatly save data collection time. However, whether it is the 9-point method or based on the full-field sample points, the brightness uniformity calculation method is basically expressed in terms of relative brightness difference, that is, the maximum brightness value in the sample points (9 points or full-field points) minus the average brightness value and then divided by the average brightness value. For the 9-point method, although the position of the 9-point Although it is a fixed selection, in an image composed of a structure with large brightness differences, whether a certain brightness area falls at the 9 o'clock position will directly affect the final calculation result, and when the number of samples is too small and the individual differences are large, the evaluation result is somewhat one-sided; if it is based on full-field sample points, on the one hand, it will be disturbed by noise, and on the other hand, relying solely on a maximum brightness value or a brightness mean to characterize the brightness characteristics of the bright area or the overall average brightness level is obviously accidental. For example, there are several maximum brightness points and an area with the maximum brightness point. For the human eye, these two brightness uniformities are obviously different, but the calculation results of the method are basically the same, so this method does not take into account the impact of pixel areas in different brightness areas.

[0003] In the related art, for example, Chinese patent document CN115885237A provides a head-mounted display image and foveal area brightness calculation, where an image that can be displayed at a head-mounted display (HMD) is downsampled to image pixels representing the image. This solution calculates the illuminance of an image pixel from the color value of the image pixel to determine the image brightness, calculates the foveal area of ​​the image from the eye tracking information of the user of the HMD, and the foveal area of ​​the image is downsampled to a foveal pixel representing the foveal area, and the illuminance of the foveal pixel is calculated from the color value of the foveal pixel to determine the foveal area brightness. However, this solution does not provide any technical inspiration on how to consider the negative impact of different brightness areas on the accuracy of the calculation results of the brightness uniformity of XR glasses. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In response to the problem of how to improve the accuracy of XR glasses brightness uniformity calculation results in the prior art, the present invention provides an XR glasses brightness uniformity processing device and method, which can characterize the differences in different brightness areas by calculating the distance between the bright or dark area vector and the background area vector, and multiply the pixel area weight of the uneven area to consider the influence of the pixel area on the result, thereby improving the accuracy of the XR glasses brightness uniformity calculation results.

[0006] 2. Technical solution

[0007] The purpose of the present invention is achieved through the following technical solutions.

[0008] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0009] Some embodiments of the present application propose an XR glasses brightness uniformity processing device and method to solve the technical problems mentioned in the above background technology section.

[0010] As the first aspect of the present application, some embodiments of the present application provide a method for processing brightness uniformity of XR glasses, including the following steps: obtaining a brightness image and performing preprocessing; building and training an image segmentation network, inputting the brightness image into the image segmentation network for segmentation, and obtaining feature vectors of the brightness area; calculating the distance between the feature vectors of the brightness area, obtaining the brightness difference of the brightness area, and calculating the brightness uniformity result.

[0011] Furthermore, the specific process of preprocessing is: generate and display the white field image of the XR glasses; use an imaging brightness meter to collect brightness images of the XR glasses and extract, position correct and scale the ROI area of ​​the collected brightness image.

[0012] Furthermore, the extracted ROI area is an ROI area that completely covers the display area of ​​the XR glasses.

[0013] Furthermore, position correction includes image registration and ROI region alignment.

[0014] Furthermore, the specific process of scaling is: scaling the ROI region brightness image scale to N×N; removing the surrounding information of the ROI region, and adjusting the size of the ROI region to match the expected display area.

[0015] Furthermore, the input of the image segmentation network is an image of size N×N with 1 channel; the output of the constructed image segmentation network is an image of size N×N with 3 channels.

[0016] Furthermore, the feature vector of the brightness area obtained is the segmentation result of the bright area, background area and dark area generated by the image segmentation network extracting the brightness image features.

[0017] Furthermore, the specific process of training the image segmentation network is: making a training set and inputting it into the image segmentation network; calculating the segmentation result through the image segmentation network; calculating the loss of the segmentation result and updating the image segmentation network; iterating the training until the performance of the image segmentation network reaches a preset result or number of iterations.

[0018] Furthermore, the brightness uniformity value is calculated The expression is:

[0019] ;

[0020] in, Represents the brightness non-uniformity value, and the expression is:

[0021] ;

[0022] = + ;

[0023] in, Represents the distance between the bright area feature vector and the background area feature vector, Represents the distance between the dark area feature vector and the background area feature vector; represents the pixel area of ​​the bright area, represents the pixel area of ​​the dark region, Represents the sum of pixel areas in all brightness regions, Represents the pixel area of ​​the background area.

[0024] As a second aspect of the present application, some embodiments of the present application provide an XR glasses brightness uniformity processing device, including an acquisition module: acquiring a brightness image and preprocessing it to obtain a brightness image; a segmentation module: building and training an image segmentation network, inputting the brightness image into the image segmentation network for segmentation, and obtaining a feature vector expression of the brightness area; a calculation module: calculating the distance between the feature vectors of the brightness area, obtaining the brightness difference of the brightness area, and calculating the brightness uniformity result.

[0025] 3. Beneficial effects

[0026] Compared with the prior art, the advantages of the present invention are: an XR glasses brightness uniformity processing device and method of the present invention can accurately calculate the brightness uniformity of the XR glasses display area, and the obtained numerical results are highly consistent with the standard deviation change trend of the brightness image, ensuring the accuracy of the calculation results, and being able to obtain more objective and reliable brightness uniformity evaluation results; compared with the traditional nine-point sampling method, the processing method of the present invention shows stronger stability and anti-interference ability in brightness uniformity calculation, and can effectively reduce the calculation errors caused by insufficient or uneven distribution of sampling points, thereby improving the accuracy of brightness uniformity evaluation; in addition, the processing method of the present invention can also resist various external interference factors such as light changes and equipment vibrations, ensuring that stable calculation results can be obtained even in complex environments; and the numerical results fit the standard deviation change trend of the brightness image and conform to the intuitive perception of the human eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 1 is a flow chart of a method for calculating brightness uniformity of XR glasses in one embodiment of the present invention;

[0028] Figure 2 FIG. 4 is a schematic diagram of an image segmentation network in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0030] like Figure 1 to Figure 2 As shown, a method for processing brightness uniformity of XR glasses of the present invention comprises the following steps:

[0031] Acquire brightness image and perform preprocessing;

[0032] Build and train an image segmentation network, input the brightness image into the image segmentation network for segmentation, and obtain the feature vector of the brightness area;

[0033] The distance between the eigenvectors of the brightness area is calculated to obtain the brightness difference of the brightness area, and the brightness uniformity result is calculated.

[0034] In a specific embodiment, the specific process of the XR glasses brightness uniformity processing method includes:

[0035] S1. Obtain brightness image and preprocess it:

[0036] Acquire a brightness image; extract the region of interest from the brightness image using ROI region extraction technology; use position correction technology to adjust the extracted ROI region. Obtain a brightness image of N×N size through the steps of brightness image acquisition and preprocessing.

[0037] In a specific embodiment, the process of brightness image acquisition and preprocessing is as follows:

[0038] S101. Generate and display a white field image of a scale corresponding to the XR glasses:

[0039] Determine the display parameters of the XR glasses and obtain the display pixel resolution of the XR glasses; generate a white field image of the corresponding scale of the XR glasses and load the generated white field image into the display system of the XR glasses. The size and resolution of the white field image match the display area of ​​the XR glasses to ensure the accuracy of the test.

[0040] Among them, the display parameters of XR glasses include resolution, field of view, screen size, etc. The display parameters of XR glasses determine the scale and resolution of the white field image.

[0041] The process of generating a white field map can be to use image editing software or a special test pattern generation tool to create a completely white image, which is used as a white field map for subsequent brightness uniformity testing.

[0042] Image editing software includes Photoshop and GIMP.

[0043] Specifically, a white field image refers to an 8-bit image with a grayscale value of 255 for all pixels. An 8-bit image means that each pixel value of the image is represented by an 8-bit binary number, and its value range is usually 0 to 255 (for unsigned integers); a grayscale value of 255 means that in a grayscale image, the pixel value reaches the maximum, i.e. 255, representing white.

[0044] The process of loading the generated white field map into the display system of the XR glasses usually involves transferring the image file to the storage device of the XR glasses, and then running the corresponding display software or application on the XR glasses to display the image. The storage device includes an internal memory or an external SD card.

[0045] Specifically, before the white field image is displayed, the display parameters of the XR glasses are set, including brightness, contrast, and color.

[0046] S102. Use an imaging brightness meter to collect brightness images of XR glasses:

[0047] Before acquisition, set the acquisition parameters according to the display parameters of the XR glasses and the performance of the imaging brightness meter.

[0048] Specifically, the XR glasses are placed on a stable bracket to ensure that the display area faces the lens of the imaging brightness meter. The acquisition parameters include exposure time, gain, and white balance.

[0049] An imaging photometer is a metering instrument that measures light and color based on the principle of imaging. It consists of a detector that matches vision (or color vision), an optical system, and a signal output processing system that is proportional to brightness (or tristimulus values). The main features of an imaging photometer include the use of an image sensor and a V-Lambda filter, with a matrix of millions of detector points, which can capture the entire scene and measure the brightness of each point simultaneously, thereby achieving high-resolution and high-sensitivity measurements of multiple points quickly in one operation.

[0050] Specifically, adjust the lens focal length and position of the imaging brightness meter to ensure that the imaging brightness meter can clearly capture the display area of ​​the XR glasses. Set the collection environment light to be uniform to avoid interference from direct sunlight or other strong light sources. The sensor size, field of view, resolution, and brightness range of the imaging brightness meter should match the display parameters of the XR glasses to avoid measurement errors caused by device mismatch.

[0051] During the acquisition process, the acquisition mode of the imaging luminance meter is set to continuous acquisition or manual trigger acquisition; the brightness image is started to be acquired according to the preset acquisition parameters and number of images; the acquired brightness image is saved to a computer or other storage device for subsequent analysis and processing.

[0052] S103, extracting, position correcting and scaling the ROI area of ​​the acquired brightness image:

[0053] First, according to the size and shape of the display area of ​​the XR glasses, the ROI area is determined, extracted, position corrected, and saved.

[0054] Specifically, it is determined that the ROI area that needs to be analyzed is an ROI area that completely covers the display area of ​​the XR glasses, and the ROI area does not contain background information.

[0055] The process of ROI region extraction is to automatically extract the ROI region according to the image content through image processing software or algorithms. For example, color threshold, edge detection and other methods can be used to identify and extract the display area of ​​XR glasses as the ROI region.

[0056] Position correction includes image registration and ROI area alignment.

[0057] Image registration makes the acquired brightness images consistent in position through methods such as feature point matching and image transformation. When there is position deviation or inconsistent rotation angle between the acquired brightness images, image registration is performed to correct the deviation.

[0058] After image registration, ROI region alignment is performed.

[0059] Specifically, the offset and rotation angle of each ROI region relative to the reference coordinate system are calculated, and they are adjusted to the correct position to achieve ROI region alignment. The correct position is to ensure that the ROI regions in all acquired brightness images are aligned to the position under the same reference coordinate system.

[0060] ROI extraction and position correction are two important steps in image processing. ROI (Region of Interest) extraction refers to extracting the part of interest from the image, while position correction is to make necessary adjustments to the extracted ROI area to ensure its accuracy and stability. More precisely, ROI extraction involves image processing techniques such as color space conversion (from BGR to HSV), threshold segmentation (creating a mask by setting a color range), and bitwise operations (using bitwise_and to extract specific parts of the image). For example, by setting the color range in the HSV color space, the area of ​​a specific color in the image can be extracted. This process can be implemented through image processing libraries such as OpenCV. Position correction involves matching or Blob analysis to determine the reference points and reference angles, and generating an affine transformation matrix for position adjustment. This process includes determining the reference source, selecting the parameters for generating the affine matrix, and applying this matrix to adjust the position of the ROI to ensure that it accurately frames the measurement area.

[0061] Specifically, after the ROI area is extracted and the position is corrected, the ROI area is saved as a separate image file or data format. Before any operation in S103 is performed, the original data and image files are backed up so that the original data can be restored in case of problems, avoiding data loss or damage, and ensuring the accuracy and reliability of subsequent brightness uniformity calculations.

[0062] Finally, the ROI area brightness image scale is scaled to a square image of any resolution. By scaling the ROI area brightness image scale, irrelevant information around the ROI area is removed, and the size of the ROI area is adjusted to match the expected display area.

[0063] A square image of arbitrary resolution means scaling the ROI area brightness image to N×N, where N represents the number of pixels on one side of the image and the range of N×N is 32×32 to 256×256.

[0064] Specifically, scaling refers to the scaling of image pixels. For XR glasses, the uneven brightness of the displayed image quality is more obvious in the white field base mode. Therefore, first display the white field image of the corresponding resolution in the XR glasses, and then use the imaging brightness meter to collect the brightness image of the glasses. If the field of view is large, it is necessary to remove the invalid edge area, extract the region of interest ROI and perform correction. Since the resolution of different XR glasses varies greatly, they are uniformly scaled to an N×N image scale.

[0065] ‌‌Combined with the above steps, the acquired brightness image can be processed as follows: ‌First, the region of interest is extracted from the brightness image through ROI region extraction technology, such as color segmentation or threshold processing. ‌Then, using position correction technology, the extracted ROI region is adjusted as necessary to ensure that its position in the image is accurate. ‌Such a processing flow is crucial to improving the accuracy and efficiency of image analysis, especially when it is necessary to extract specific targets from complex backgrounds or perform precise measurements. Through the steps of brightness image acquisition and preprocessing, images with invalid areas removed and N×N image scales of different resolutions smoothed are obtained. Among them, the preprocessing process smoothes individual differences and unifies the input of the network.

[0066] S2. Get the feature vectors of different brightness areas:

[0067] Build an image segmentation network, input the obtained brightness image into the image segmentation network for segmentation, and obtain the feature vector of the brightness area.

[0068] Specifically, after the brightness image is acquired and the N×N scale brightness image obtained in the preprocessing is input into the image segmentation network, the brightness image is segmented into three different brightness areas: bright, background and dark.

[0069] Brightness image segmentation aims to divide an image into different regions according to brightness. In this embodiment, the brightness image is segmented into three different brightness regions: bright, background, and dark. The brightness image segmentation process is the result of the output of the trained image segmentation network.

[0070] Specifically, the background area is a reference brightness area in the image, which usually represents the average brightness level of most areas in the image. In the brightness image of XR glasses, the background area may correspond to the border or background environment around the display area.

[0071] The bright area is an area with higher brightness relative to the background area. In the display area of ​​the XR glasses, the bright area may correspond to the part that displays high-brightness content.

[0072] The dark area is an area with lower brightness relative to the background area. In the display area of ​​XR glasses, the dark area may correspond to a portion displaying low-brightness content or in shadow.

[0073] In order to achieve brightness image segmentation, it is necessary to train the image segmentation network. The training of the image segmentation network is carried out through the following steps:

[0074] Prepare training data: Collect a large number of brightness images and manually or automatically mark the boundaries of bright areas, background areas, and dark areas. These labeled data will serve as the training target of the network.

[0075] Build an image segmentation network: The image segmentation network includes convolutional layers, pooling layers, upsampling layers, etc., which are used to extract image features and generate segmentation results.

[0076] Training process: create a training set and input it into the image segmentation network; calculate the segmentation result through the image segmentation network; calculate the loss of the segmentation result and update the image segmentation network; iterate the training until the performance of the image segmentation network reaches the preset result or number of iterations.

[0077] After training, the brightness image is input into the segmentation network to obtain the segmentation results of bright areas, background areas, and dark areas. These results are represented in the form of mask images, where each area is assigned a different label or color. Furthermore, the pixel area of ​​each area in the mask image can be calculated by counting the pixels in each area.

[0078] More specifically, according to the image segmentation network, the input images are segmented to obtain their mask images (the designated area is 1, and other areas are 0), and then the mask images are matrix-multiplied with the input images to achieve effective separation of different brightness areas, that is, three N×N regional brightness images are obtained, which are then flattened into 1×(N×N) vectors and input into the fully connected network for feature extraction and dimensionality reduction, thereby obtaining feature vectors of N-dimensional different brightness areas.

[0079] A mask image is usually used to identify or highlight a specific area in an image. In this embodiment, the mask image is used to identify the bright area, background area, and dark area in the brightness image. Each pixel value in the mask image is either 1 or 0. Among them, 1 means that the pixel belongs to the specified area (such as a bright area, background area, or dark area), and 0 means that the pixel does not belong to the area. For example, if there is a mask image of a bright area, then all pixel values ​​belonging to the bright area in the image will be set to 1, and the pixel values ​​of other areas will be set to 0. In this way, the bright area in the brightness image can be easily identified through the mask image.

[0080] Matrix dot multiplication is an element-by-element multiplication operation, which requires that the dimensions of the two matrices are the same. In this embodiment, the mask image is matrix dot multiplied with the input image to achieve effective separation of different brightness areas. Specifically, a mask image can be generated for each brightness area (bright area, background area, and dark area). Then, each mask image is matrix dot multiplied with the input image. In this way, each dot product result will only retain the pixel values ​​of the corresponding brightness area, while the pixel values ​​of other areas will be set to 0. In this way, three N×N scale regional brightness images can be obtained, corresponding to the bright area, background area, and dark area respectively. These images only contain the pixel values ​​of their respective areas, thereby achieving effective separation of different brightness areas.

[0081] Flattening an N×N scale regional brightness image into a 1×(N×N) vector means converting an N×N matrix into a one-dimensional vector. In this one-dimensional vector, all elements of the original matrix are arranged in a certain order (such as row priority or column priority).

[0082] In this embodiment, the three separated regional brightness images of N×N size are flattened into 1×(N×N) vectors respectively. In this way, each regional brightness image is converted into a one-dimensional vector containing N×N elements.

[0083] These vectors are then input into a fully connected network for feature extraction and dimensionality reduction. The fully connected network learns to extract useful feature information from these vectors and converts them into lower-dimensional feature vectors (such as N-dimensional feature vectors). These feature vectors can be used for subsequent classification, recognition, or regression tasks.

[0084] In this module, the key is to determine the image segmentation network and the fully connected network.

[0085] In a specific embodiment, the image data may be collected by collecting 6,000 white field images with different brightness and with various brightness regions having different shapes and scales.

[0086] In a specific embodiment, since the brightness value of the normal white field brightness image of the XR glasses basically fluctuates within a small range, the background brightness area of ​​the image is basically determined. Therefore, the brightness image can be divided into three parts: bright, background, and dark areas, thereby obtaining a data set consisting of the brightness image and segmentation labels;

[0087] Image segmentation network such as Figure 2 As shown, the image input is N×N, the number of channels is 1, the output is N×N, the number of channels is 3, and the image segmentation network can be trained according to the data set.

[0088] The fully connected network is composed of ordinary fully connected layers. The input is a 1×(N×N)-dimensional vector, and the output is three N-dimensional feature vectors. The N×N brightness image obtains three N-dimensional feature vectors after passing through the fully connected network. The segmentation network inference result can also obtain three N-dimensional vectors after passing through the fully connected network. The fully connected network can be trained by minimizing the sum of the distances between the corresponding feature vectors.

[0089] In a specific embodiment, the process of obtaining the feature vector expression of different brightness areas is as follows:

[0090] S201. Build an image segmentation network:

[0091] Build an image segmentation network whose input is an N×N image with 1 channel and whose output is an N×N image with 3 channels.

[0092] Among them, a channel number of 1 represents a grayscale image or a single-channel color image; a channel number of 3 represents the segmentation results of the bright area, background area, and dark area, respectively.

[0093] Specifically, in order to train the image segmentation network, a labeled dataset is established, in which each image corresponds to a segmentation label of three channels, marking the bright area, background area and dark area respectively.

[0094] like Figure 2 As shown in the figure, the image segmentation network includes convolutional layers, pooling layers, and upsampling layers, which are used to extract image features and generate segmentation results.

[0095] The convolution layer can deeply extract feature information from the image; the pooling layer helps to reduce the dimension of the data and improve the efficiency of calculation; the upsampling layer is responsible for restoring the feature map to the size of the original image, thereby ensuring that a segmentation result consistent with the size of the input image can be generated.

[0096] S202, segment and label different brightness areas and create training sets:

[0097] Data preparation: 6,000 white field images with different brightness and varying shapes and scales in each brightness area were collected.

[0098] Labeling: Label each image and generate segmentation labels for three channels, corresponding to bright areas, background areas, and dark areas.

[0099] Training set preparation: Combine the labeled images and labels into a training set for training the image segmentation network.

[0100] S203, training image segmentation network:

[0101] Input training data: Input the images and labels in the training set into the image segmentation network.

[0102] Forward propagation: Perform forward propagation through the image segmentation network to calculate the segmentation results.

[0103] Specifically, the image is feature extracted through structures such as convolutional layers and pooling layers to form a feature map; the feature map is gradually restored to the size of the original image through upsampling layers or other decoding structures, and the segmentation result is generated.

[0104] The segmentation result is usually a feature map with the same size as the input image, but the number of channels is equal to the number of categories (3 in this example, corresponding to bright areas, background areas, and dark areas, respectively).

[0105] Loss calculation: Calculate the loss between the segmentation result and the true label, such as cross entropy loss or Dice loss.

[0106] Backpropagation: Backpropagate the loss to update the weights of the image segmentation network.

[0107] Iterative training: Repeat the above process until the performance of the image segmentation network reaches a preset result or number of iterations.

[0108] S204, perform image segmentation on the input image to obtain a segmentation mask image, and calculate the pixel area of ​​each part:

[0109] Input image: Input the brightness image of N×N scale to be processed into the trained image segmentation network.

[0110] Segmentation results: The image segmentation network outputs segmentation results of three channels, corresponding to bright areas, background areas, and dark areas.

[0111] Convert and calculate: Convert the segmentation result into a mask image, where the bright area, background area, and dark area are represented by 1, 0, and other values ​​(such as 2) respectively;

[0112] By counting the number of pixels in each region in the mask image, the pixel area of ​​each part is obtained.

[0113] S205. Build a fully connected network and perform training based on the inference results of the segmentation network:

[0114] First, build a fully connected network, the input is a 1×(N×N)-dimensional vector, and the output is an N-dimensional feature vector.

[0115] Among them, the input 1×(N×N)-dimensional vector is obtained by flattening (i.e., flattening into a one-dimensional array) the pixel values ​​of a specific area (such as a bright area, background area, or dark area) obtained by the image segmentation network.

[0116] Secondly, prepare the training data. Use the trained image segmentation network to infer the input image and obtain the segmentation result; select the area of ​​interest (such as bright area, background area or dark area); flatten the pixel values ​​of the selected area image into a one-dimensional array as the input vector of the fully connected network; form a training pair with the flattened input vector and the corresponding feature vector for training the fully connected network.

[0117] Finally, the composed training pairs are used to train a fully connected network, enabling it to extract an N-dimensional feature vector from the region image.

[0118] Through the above steps, a fully connected network can be systematically built and trained based on the inference results of the segmentation network to obtain the feature vector of the brightness area.

[0119] S206, based on the fully connected network reasoning, the feature vectors of the three brightness areas of bright, background and dark are obtained:

[0120] Input region image: First, use the trained image segmentation network to segment the input image and obtain the segmentation results of bright area, background area and dark area. The segmentation result is usually a two-dimensional image or matrix.

[0121] Secondly, the segmented bright area, background area, and dark area images are flattened into 1×(N×N)-dimensional vectors respectively. This process involves flattening the pixel values ​​of the two-dimensional image matrix into a one-dimensional array in a certain order (such as row-first or column-first).

[0122] Fully connected network reasoning: The flattened vectors of the bright area, background area, and dark area are used as inputs and input into the fully connected network in sequence.

[0123] The fully connected network calculates the output feature vector corresponding to each input vector through forward propagation. In this process, the input vector passes through each layer of the fully connected network, multiplied by the weight matrix and added with the bias term, and then passes through the activation function to obtain the input of the next layer.

[0124] Feature vector output: The fully connected network outputs three N-dimensional feature vectors, corresponding to the features of the bright area, background area, and dark area respectively.

[0125] S3. Brightness uniformity calculation:

[0126] The distance between the eigenvectors of the brightness area is calculated to obtain the brightness difference of the brightness area, and the brightness uniformity result is calculated.

[0127] Specifically, after the steps of acquiring and preprocessing the brightness image and acquiring the feature vector expression of different brightness areas, the feature vector expression of different brightness areas of the XR glasses can be obtained, and then the brightness uniformity calculation is further performed. By calculating the distance between the feature vector of the bright area and the feature vector of the background area, the brightness difference of the bright area relative to the background area and the brightness difference of the dark area relative to the background area are measured, and the brightness uniformity calculation is realized.

[0128] In a specific embodiment, the process of calculating brightness uniformity is as follows:

[0129] Calculate the distance between the bright area feature vector and the background area feature vector :

[0130] ;

[0131] Calculate the distance between the dark area feature vector and the background area feature vector :

[0132] ;

[0133] in, The first eigenvector of the bright area Quantity, The first eigenvector of the dark region Quantity, The first character vector of the background region components, and N is the dimension of the feature vector.

[0134] Then, the calculated vector distance and Normalized to between 0 and 1:

[0135] ;

[0136] ;

[0137] in, is the minimum of all possible distances, is the maximum value among all possible distances, and The value is set according to the actual application scenario.

[0138] Calculate the brightness non-uniformity value of the input image: multiply the distance sum by the sum of the pixel areas of the bright and dark areas to get the brightness non-uniformity value , the expression is as follows:

[0139]

[0140] in, represents the pixel area of ​​the bright area, represents the pixel area of ​​the dark region, Represents the sum of pixel areas of different brightness regions, = + , Represents the pixel area of ​​the background area.

[0141] The purpose of dividing by 2 is to make the order of magnitude of the calculation result consistent with a certain standard or reference method (such as the 9-point method) to facilitate comparison and analysis of the results. This is usually determined based on specific application scenarios and needs. The purpose of multiplying by the sum of the pixel areas of the bright and dark areas is to take the pixel area factor into account.

[0142] The brightness non-uniformity value depends on the relative brightness difference and the pixel area ratio of the non-uniform area. The larger the vector distance is, the greater the brightness difference with the background area is, and the stronger the non-uniformity is; the larger the pixel area ratio of the non-uniform area is, the stronger the non-uniformity is.

[0143] Calculate the brightness uniformity value of the input image :

[0144] ;

[0145] The brightness non-uniformity value is calculated to reflect the brightness difference between the bright and dark areas in the image relative to the background area and the pixel area ratio of these non-uniform areas. The larger the brightness non-uniformity value, the stronger the brightness non-uniformity.

[0146] In this embodiment, the Euclidean distance is used to calculate the distance between feature vectors, but other distance metrics (such as Manhattan distance, cosine similarity, etc.) can also be used for this purpose. The choice of distance metric depends on the specific application scenario and the nature of the feature vector.

[0147] In the process of calculating the brightness uniformity of the display area of ​​XR glasses, the numerical results obtained in this scheme are highly consistent with the standard deviation change trend of the brightness image, which ensures the accuracy of the calculation results and can obtain more objective and reliable brightness uniformity evaluation results; in addition, it can effectively reduce the calculation errors caused by insufficient or uneven distribution of sampling points, thereby improving the accuracy of brightness uniformity evaluation.

[0148] Specifically for the description of the device, the corresponding hardware implementation can be performed by referring to the corresponding software and method content in this embodiment; the corresponding device and equipment can be constructed;

[0149] For example, a brightness uniformity processing device for XR glasses includes an acquisition module: acquiring a brightness image and performing preprocessing to obtain a brightness image of N×N scale;

[0150] Segmentation module: Build and train an image segmentation network, input the obtained brightness image into the image segmentation network for segmentation, and obtain the feature vector expression of the brightness area;

[0151] Calculation module: calculates the distance between the characteristic vectors of the brightness area, obtains the brightness difference of the brightness area, and calculates the brightness uniformity result.

[0152] The device designed for calculating the brightness uniformity of XR glasses may be a hardware device, including one or more processors and a memory, wherein the memory stores a program and is configured to execute the following steps by the one or more processors:

[0153] Obtain a brightness image and perform preprocessing to obtain a brightness image of N×N scale;

[0154] Build and train an image segmentation network, input the obtained brightness image into the image segmentation network for segmentation, and obtain the feature vector of the brightness area;

[0155] The distance between the eigenvectors of the brightness area is calculated to obtain the brightness difference of the brightness area, and the brightness uniformity result is calculated.

[0156] The storage medium designed for XR glasses brightness uniformity calculation may be a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned XR glasses brightness uniformity processing method is implemented.

[0157] How to realize the device specifically can be realized in the existing technology, and no further detailed explanation will be given again. The following explains the possibility of corresponding realization in principle.

[0158] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many improvements to method flows today can be considered direct improvements to hardware circuit structures.

[0159] Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logic function is determined by the user's programming of the device. Designers program themselves to "integrate" a digital system on a PLD, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when writing program development, and the original code before compilation must also be written in a specific programming language, which is called Hardware Description Language (HDL), and there is not only one HDL, but many types, such as Verilog. Those skilled in the art should also be aware that it is easy to obtain a hardware circuit that implements the logical method flow by simply performing some logic programming on the method flow using the above-mentioned hardware description languages ​​and programming it into an integrated circuit.

[0160] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include but are not limited to the following microcontrollers: ATMEL AT89S52, microchip pic16c57. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0161] The systems, devices, modules or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices. For the convenience of description, the above devices are described separately by function into various units. Of course, when implementing this specification, the functions of each unit can be implemented in the same or more software and / or hardware.

[0162] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that instructions executed by the processor of a computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps of the functions specified in the blocks or multiple blocks. In a typical configuration, the computing device includes one or more processors (CPU), input / output interfaces, network interfaces and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0164] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0166] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0168] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A method for processing brightness uniformity of XR glasses, comprising the following steps: Obtain a brightness image and perform preprocessing to obtain a brightness image of N×N scale; Build and train an image segmentation network. The image input of the image segmentation network is N×N, the number of channels is 1, and the output is N×N, the number of channels is 3; input the acquired brightness image into the trained image segmentation network for image segmentation, and obtain the segmentation results of each brightness area; Separate each brightness region through matrix dot multiplication to obtain a regional brightness image of N×N scale, perform feature extraction and dimensionality reduction, and obtain the feature vector of the brightness region. The feature vector of the brightness region is the segmentation result of the bright region, background region, and dark region generated by extracting brightness image features through the image segmentation network. Calculate the distance between the characteristic vectors of the brightness area, obtain the brightness difference of the brightness area, and calculate the brightness uniformity result; Calculate brightness uniformity value The expression is: ； in, Represents the brightness non-uniformity value, and the expression is: ； = + ； in, Represents the distance between the bright area feature vector and the background area feature vector, Represents the distance between the dark area feature vector and the background area feature vector; represents the pixel area of ​​the bright area, represents the pixel area of ​​the dark region, Represents the sum of pixel areas in all brightness regions, Represents the pixel area of ​​the background area.

2. The XR glasses brightness uniformity processing method according to claim 1, characterized in that: The specific process of preprocessing is: generate and display the white field image of the XR glasses; use an imaging brightness meter to collect brightness images of the XR glasses and extract, position correct and scale the ROI area of ​​the collected brightness image.

3. The XR glasses brightness uniformity processing method according to claim 2, characterized in that: The extracted ROI area is an ROI area that completely covers the display area of ​​the XR glasses.

4. The XR glasses brightness uniformity processing method according to claim 2, characterized in that: Position correction includes image registration and ROI area alignment.

5. The XR glasses brightness uniformity processing method according to claim 2, characterized in that: The specific process of scaling is: scaling the ROI area brightness image to a square image of any resolution; removing the surrounding information of the ROI area, and adjusting the size of the ROI area to match the expected display area.

6. The XR glasses brightness uniformity processing method according to claim 1, characterized in that: The specific process of training the image segmentation network is: create a training set and input it into the image segmentation network; calculate the segmentation result through the image segmentation network; calculate the loss of the segmentation result and update the image segmentation network; iterate the training until the performance of the image segmentation network reaches the preset result or number of iterations.

7. A device for processing brightness uniformity of XR glasses, characterized in that: include, Acquisition module: acquires the brightness image and performs preprocessing to obtain a brightness image of N×N scale; Segmentation module: Build and train an image segmentation network. The image input of the image segmentation network is N×N, the number of channels is 1, and the output is N×N, the number of channels is 3. Input the acquired brightness image into the trained image segmentation network for image segmentation, and obtain the segmentation results of each brightness area. Each brightness region is separated by matrix dot multiplication to obtain a regional brightness image of N×N scale, and feature extraction and dimensionality reduction are performed to obtain the feature vector expression of the brightness region. The feature vector of the brightness region is the segmentation result of the bright region, background region and dark region generated by extracting the brightness image features through the image segmentation network; Calculation module: calculates the distance between the characteristic vectors of the brightness area, obtains the brightness difference of the brightness area, and calculates the brightness uniformity result; Calculate brightness uniformity value The expression is: ； in, Represents the brightness non-uniformity value, and the expression is: ； = + ； in, Represents the distance between the bright area feature vector and the background area feature vector, Represents the distance between the dark area feature vector and the background area feature vector; represents the pixel area of ​​the bright area, represents the pixel area of ​​the dark region, Represents the sum of pixel areas in all brightness regions, Represents the pixel area of ​​the background area.

Images