Image data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide an image data processing method and device, electronic equipment and storage medium, relating to the technical field of data processing, and used for reducing the occurrence of uneven brightness of displayed images. The method comprises: obtaining an initial gray scale value corresponding to a pixel region in which any pixel group in target image data is located; determining a target gray scale value corresponding to the pixel group based on a target gray scale value mapping model, in combination with a pixel group type and the initial gray scale value of the pixel group; wherein the target gray scale value mapping model is used to describe the correlation between the initial gray scale value and the target gray scale value of at least one pixel group type; and replacing the initial gray scale value of the pixel region in which the pixel group is located in the target image data with the target gray scale value, and displaying an image based on the target image data through a display screen.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to an image data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In order to improve the transmittance of a display screen, the touch emission lines (TX lines) that are originally located between pixels can be removed. This method can increase the transmittance of the display screen by more than 8%.

[0003] However, removing the TX line results in inconsistent widths of the black matrix (BM) occlusion area. Specifically, the BM occlusion area within a pixel group is narrower, while the BM occlusion area between pixel groups is wider. Areas with narrower BMs are brighter, and areas with wider BMs are less bright. This leads to uneven brightness in the displayed image, such as visible fine lines due to differences in brightness between different pixel groups.

[0004] Therefore, reducing uneven brightness in displayed images is a problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this disclosure is to provide an image data processing method, apparatus, electronic device, and storage medium for reducing uneven brightness in displayed images.

[0006] To achieve the above objectives, the embodiments of this disclosure provide the following technical solutions: On one hand, an image data processing method is provided. The method includes: for any pixel group in target image data, obtaining the initial grayscale value corresponding to the pixel region where the pixel group is located; determining the target grayscale value corresponding to the pixel group based on a target grayscale value mapping model, combined with the pixel group type and the initial grayscale value; replacing the initial grayscale value of the pixel region where the pixel group is located in the target image data with the target grayscale value; and displaying the image on a display screen based on the target image data. The target grayscale value mapping model is used to describe the correlation between the initial grayscale value and the target grayscale value for at least one pixel group type.

[0007] In the above embodiments, in order to achieve brightness uniformity, a target grayscale value mapping model is established to describe the correlation between the initial grayscale value and the target grayscale value of each pixel group type. In this way, when it is necessary to display the image corresponding to the target image data, the initial grayscale value corresponding to the pixel region where each pixel group is located in the target image data is first obtained, and then the target grayscale value corresponding to each pixel group is obtained by combining the target grayscale value mapping model. By replacing the initial grayscale value of the pixel region where each pixel group is located in the target image data with the corresponding target grayscale value, grayscale compensation is achieved. Finally, the image is displayed based on the grayscale compensated target image data, which reduces the occurrence of uneven display of the displayed image.

[0008] In some embodiments, the pixel regions of multiple pixel groups in the target image data are obtained by the following methods: Pixel group identification is performed on a first calibration image containing at least one bright spot to obtain the initial region corresponding to each of the multiple pixel groups in the first calibration image. Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, pixel group analysis is performed on the target image data to obtain the pixel region where each of the multiple pixel groups in the target image data is located. The first calibration image is acquired from a target viewing angle of the display screen when multiple bright spots arranged in a preset pattern on the display screen are lit.

[0009] In some embodiments, pixel group recognition is performed on a first calibration image containing at least one bright spot to obtain initial regions corresponding to each of the multiple pixel groups in the first calibration image, including: inputting the first calibration image into a segmentation model, performing grid segmentation on the first calibration image by combining the segmentation model with at least one bright spot to obtain grid segmentation results corresponding to the first calibration image; and dividing the first calibration image into initial regions corresponding to each of the multiple pixel groups based on the grid segmentation results.

[0010] In some embodiments, pixel group analysis is performed on the target image data based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image to obtain the pixel regions where each of the multiple pixel groups in the target image data is located, including: obtaining the pixel regions where the pixel groups corresponding to the target viewpoint are located in the target image data based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image; and obtaining the pixel regions where the pixel groups corresponding to other viewpoints are located in the target image data based on the pixel regions where the pixel groups corresponding to the target viewpoint are located, combined with a preset pixel group distribution method.

[0011] In some embodiments, the segmentation model is obtained by: performing multiple first iterations of training on the segmentation model based on a sample set to adjust the first parameter corresponding to bright spot detection in the segmentation model; and performing multiple second iterations of training on the segmentation model based on the sample set to adjust the target parameters in the segmentation model. The target parameters include the second parameter corresponding to mesh segmentation in the segmentation model and the aforementioned first parameter.

[0012] In some embodiments, the sample set includes multiple samples, each containing a sample calibration image. The calibration image of a target sample among the multiple samples contains at least one bright spot, and the number of bright spots in the calibration images of at least some of the target samples differs.

[0013] In some embodiments, each first iteration of training includes: selecting a first sample from the sample set, inputting the first sample labeling image in the first sample into the segmentation model, performing bright spot detection on the first sample labeling image through the segmentation model to obtain the corresponding first predicted bright spot detection result; obtaining a first loss representing the difference between the first predicted bright spot detection result and the actual bright spot detection result in the first sample, and adjusting the first parameter based on the first loss.

[0014] In some embodiments, each second iteration of training includes: A second sample is selected from the sample set, and the second sample calibration image in the second sample is input into the segmentation model. The segmentation model is used to detect bright spots in the second sample calibration image to obtain the corresponding second predicted bright spot detection result. The segmentation model is also used to perform grid segmentation on the second sample calibration image to obtain the corresponding predicted grid distribution. A second loss representing the difference between the second predicted bright spot detection result and the actual bright spot detection result in the second sample is obtained, as well as a third loss representing the difference between the predicted grid distribution and the actual grid distribution in the second sample. The target parameters are adjusted by combining the second loss and the third loss.

[0015] In some embodiments, the target grayscale value mapping model is obtained by: when multiple bright spots arranged in a preset pattern on the display screen are lit, multiple images are captured on the display screen to obtain multiple second calibration images; for any second calibration image, display parameters corresponding to multiple pixel groups in the second calibration image are obtained; based on the obtained multiple display parameters, at least one preset grayscale value mapping model is established; and the target grayscale value mapping model is selected from the at least one preset grayscale value mapping model. Each display parameter includes the actual brightness value and the actual grayscale value of the corresponding pixel group.

[0016] In some embodiments, establishing at least one preset grayscale value mapping model based on multiple obtained display parameters includes: for any preset viewing angle range among multiple preset viewing angle ranges, establishing a preset grayscale value mapping model corresponding to the preset viewing angle range based on the display parameters corresponding to the second calibration image of the preset viewing angle range among the multiple display parameters. Selecting a target grayscale value mapping model from at least one preset grayscale value mapping model includes: obtaining a preset viewing angle range that matches the viewing angle of the pixel group from the multiple preset viewing angle ranges, and selecting the preset grayscale value mapping model corresponding to the preset viewing angle range that matches the viewing angle as the target grayscale value mapping model.

[0017] In some embodiments, establishing at least one preset grayscale value mapping model based on multiple obtained display parameters includes: establishing a preset grayscale value mapping model based on multiple display parameters. Selecting a target grayscale value mapping model from at least one preset grayscale value mapping model includes: using one preset grayscale value mapping model as the target grayscale value mapping model.

[0018] In some embodiments, establishing a preset grayscale value mapping model corresponding to the preset viewing angle range based on the display parameters corresponding to the second calibration image of the preset viewing angle range among a plurality of display parameters includes: for any pixel group type, establishing a corresponding first brightness model based on the display parameters corresponding to the pixel group type in the second calibration image of the preset viewing angle range among a plurality of display parameters; based on the established plurality of first brightness models of the preset viewing angle range, using the actual grayscale value of a reference pixel group type as the initial grayscale value, and the actual grayscale value of at least one non-reference pixel group type under the same brightness as the corresponding target grayscale value, establishing a preset grayscale value mapping model corresponding to the preset viewing angle range. The aforementioned first brightness model is used to describe the correlation between the actual brightness value and the actual grayscale value in the pixel group type for the preset viewing angle range.

[0019] In some embodiments, a preset grayscale value mapping model is established based on multiple display parameters, including: for any pixel group type, establishing a corresponding second brightness model based on the display parameters corresponding to that pixel group type among the multiple display parameters; and based on the established multiple second brightness models, establishing a preset grayscale value mapping model with the actual grayscale value of a reference pixel group type as the initial grayscale value and the actual grayscale value of at least one non-reference pixel group type under the same brightness as the corresponding target grayscale value. The second brightness model is used to describe the correlation between the actual brightness value and the actual grayscale value in that pixel group type from a global viewing angle.

[0020] In some embodiments, before determining the target grayscale value corresponding to the pixel group, the method further includes: determining that the pixel group type of the pixel group is any non-reference pixel group type.

[0021] On the other hand, an image data processing apparatus is provided. The apparatus includes: an initial grayscale acquisition module, a target grayscale acquisition module, and a grayscale compensation module. The initial grayscale acquisition module is used to obtain the initial grayscale value corresponding to the pixel region where any pixel group is located in the target image data; wherein the target image data corresponds to the target viewing angle of the display screen. The target grayscale acquisition module is used to determine the target grayscale value corresponding to the pixel group based on a target grayscale value mapping model corresponding to the target image data, combined with the pixel group type and the initial grayscale value; wherein the target grayscale value mapping model is used to describe the correlation between the initial grayscale value and the target grayscale value of at least one pixel group type. The grayscale compensation module is used to replace the initial grayscale value of the pixel region where the pixel group is located in the target image data with the target grayscale value, and display the image on the display screen based on the target image data.

[0022] In another aspect, an electronic device is provided. The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the image data processing method as described in any of the above embodiments.

[0023] In another aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores computer program instructions that, when executed on a computer (e.g., an electronic device), cause the computer to perform the image data processing method as described in any of the above embodiments.

[0024] In another aspect, a computer program product is provided. The computer program product includes computer program instructions that, when executed on a computer (e.g., an electronic device), cause the computer to perform the image data processing method as described in any of the above embodiments.

[0025] In another aspect, a computer program is provided. When the computer program is executed on a computer (e.g., an electronic device), the computer program causes the computer to perform the image data processing method as described in any of the above embodiments.

[0026] The image data processing apparatus, electronic device, computer-readable storage medium, computer program product, and computer program described above have the same beneficial technical effects as the image data processing methods provided in some embodiments, and will not be repeated here. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in this disclosure, the accompanying drawings used in some embodiments of this disclosure will be briefly described below. Obviously, the drawings described below are merely drawings of some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings. Furthermore, the drawings described below can be regarded as schematic diagrams and are not intended to limit the actual process of the methods involved in the embodiments of this disclosure.

[0028] Figure 1 Pixel grouping map provided for embodiments of this disclosure; Figure 2 A pixel periodic distribution diagram in a display screen provided in an embodiment of this disclosure; Figure 3 This is a pixel distribution map within a pixel cycle provided in an embodiment of the present disclosure; Figure 4 A flowchart illustrating the first image data processing method provided in this disclosure embodiment; Figure 5 The first calibration image provided for embodiments of this disclosure; Figure 6 A pixel group distribution map in the first calibration image provided in this embodiment of the disclosure; Figure 7 This is a first preset viewing angle range division diagram provided in the embodiments of this disclosure; Figure 8 This is a second preset viewpoint range division diagram provided in the embodiments of this disclosure; Figure 9 This is a third preset viewpoint range division diagram provided in the embodiments of this disclosure; Figure 10 This is a structural block diagram of the image data processing apparatus provided in the embodiments of this disclosure; Figure 11 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0029] The technical solutions of this disclosure in some embodiments will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments provided in this disclosure are within the scope of protection of this disclosure.

[0030] Unless the context otherwise requires, throughout the specification and claims, the term "comprise" and its other forms, such as the third-person singular "comprises" and the present participle "comprising," are interpreted as open-ended and encompassing, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific example," or "some examples," etc., are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples.

[0031] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0032] "At least one of A, B and C" has the same meaning as "at least one of A, B or C", both including the following combinations of A, B and C: only A, only B, only C, combinations of A and B, combinations of A and C, combinations of B and C, and combinations of A, B and C.

[0033] As used herein, depending on the context, the term “if” may optionally be interpreted as meaning “when”, “in the event of”, “in response to determination”, or “in response to detection”. Similarly, depending on the context, the phrase “if it is determined that…” or “if [the stated condition or event] is detected” may optionally be interpreted as meaning “in the event of determination that…”, “in response to determination that…”, “when [the stated condition or event] is detected”, or “in response to the detection of [the stated condition or event]”.

[0034] In addition, the use of “based on” implies openness and inclusivity, because processes, steps, calculations or other actions “based on” one or more of the stated conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated.

[0035] In order to improve the transmittance of a display screen, the touch emission lines (TX lines) that are originally located between pixels can be removed. This method can increase the transmittance of the display screen by more than 8%.

[0036] However, removing the TX line changed the pixel spacing pattern, causing the originally uniform pixel groups to be redefined as periodically distributed pixel groups. See also... Figure 1 As shown, there are 1-pixel groups consisting of 1 pixel, 2-pixel groups consisting of 2 pixels, and 3-pixel groups consisting of 3 pixels.

[0037] The above pixel group distribution exhibits an inconsistent width of the BM occlusion area. Specifically, the BM occlusion area within a pixel group is narrower, while the BM occlusion area between pixel groups is wider. Areas with narrower BMs have higher brightness, while areas with wider BMs have lower brightness.

[0038] This can lead to uneven brightness in the images displayed on the screen. For example, due to differences in brightness between different pixel groups, fine lines that are visible to the naked eye may appear.

[0039] A display screen contains a large number of pixels, and these pixels form pixel groups that are periodically distributed. Taking a display screen with 2000×900 pixels as an example, each pixel cycle consists of 63×50 pixels. The periodic distribution can be found in [reference needed]. Figure 2 As shown.

[0040] For example, see the following line. Figure 3 As shown, 50 pixels constitute one pixel cycle. The pixels start with a group of 3 pixels (pixel 1 / 2 / 3), followed by a group of 1 pixels (pixel 4), then a group of 2 pixels (pixel 5 / 6), and so on, with groups of 1 and 2 pixels arranged in a cycle until the next cycle begins.

[0041] Uneven brightness mainly occurs near the 3-pixel group, while the 1-pixel group and 2-pixel group are regularly distributed in the loop, with relatively uniform brightness.

[0042] Therefore, reducing uneven brightness in displayed images is a problem that urgently needs to be solved.

[0043] In view of this, embodiments of the present disclosure provide an image data processing method, apparatus, electronic device, and storage medium for assessing the severity of defects. The method includes: for any pixel group in target image data, obtaining an initial grayscale value corresponding to the pixel region where the pixel group is located; determining a target grayscale value corresponding to the pixel group based on a target grayscale value mapping model, combined with the pixel group type and the initial grayscale value; wherein the target grayscale value mapping model is used to describe the correlation between the initial grayscale value and the target grayscale value of at least one pixel group type; replacing the initial grayscale value of the pixel region where the pixel group is located in the target image data with the target grayscale value, and displaying an image based on the target image data through a display screen.

[0044] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with reference to the accompanying drawings and specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0045] Figure 4 A flowchart of the first image data processing method provided in the embodiments of this disclosure is shown below. Figure 4 As shown, it includes the following steps: Step S401: For any pixel group in the target image data, obtain the initial grayscale value corresponding to the pixel region where the pixel group is located.

[0046] For displays, brightness is a grayscale-driven result; that is, the grayscale value of a single pixel determines its brightness. To solve the problem of uneven brightness, adjustments need to be made from the perspective of grayscale values.

[0047] Based on this, in this embodiment of the application, the initial grayscale value corresponding to the pixel region where the pixel group is located is first obtained for each pixel group in the target image data (the initial grayscale value corresponding to each pixel point contained in the pixel group), so as to make subsequent adjustments.

[0048] Step S402: Based on the target grayscale value mapping model, combined with the pixel group type and the initial grayscale value of the pixel group, determine the target grayscale value corresponding to the pixel group.

[0049] The target grayscale value mapping model is used to describe the relationship between the initial grayscale value and the target grayscale value of at least one pixel group type.

[0050] In this embodiment of the application, in order to achieve brightness uniformity, a target grayscale value mapping model is established to describe the correlation between the initial grayscale value and the target grayscale value of each of at least one pixel group type.

[0051] In this way, for a group of pixels, based on the target grayscale value mapping model, combined with the pixel group type and the initial grayscale value, it can be determined what target grayscale value the initial grayscale value should be adjusted to in order to achieve uniform brightness.

[0052] Step S403: Replace the initial grayscale value of the pixel region where the pixel group is located in the target image data with the target grayscale value, and display the image on the display screen based on the target image data.

[0053] In this embodiment, grayscale compensation is achieved by replacing the initial grayscale value of the pixel region where each pixel group is located in the target image data with the corresponding target grayscale value (the initial grayscale value of each pixel point contained in the pixel group is replaced with its corresponding target grayscale value). Finally, the image is displayed based on the grayscale-compensated target image data, which reduces the occurrence of uneven display of the displayed image.

[0054] In the above scheme, in order to achieve uniform brightness, a target grayscale value mapping model is established to describe the correlation between the initial grayscale value and the target grayscale value of each pixel group type. In this way, when it is necessary to display the image corresponding to the target image data, the initial grayscale value of the pixel region where each pixel group is located in the target image data is first obtained. Then, combined with the target grayscale value mapping model, the target grayscale value corresponding to each pixel group is obtained. By replacing the initial grayscale value of the pixel region where each pixel group is located in the target image data with the corresponding target grayscale value, grayscale compensation is achieved. Finally, the image is displayed based on the grayscale compensated target image data, which reduces the occurrence of uneven display of the displayed image.

[0055] In some embodiments, the pixel regions of multiple pixel groups in the target image data are obtained by the following methods: Pixel group identification is performed on a first calibration image containing at least one bright spot to obtain the initial region corresponding to each of the multiple pixel groups in the first calibration image; wherein, the first calibration image is obtained by capturing the display screen from a target viewing angle when multiple bright spots arranged in a preset manner on the display screen are lit. Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, pixel group analysis is performed on the target image data to obtain the pixel regions where each of the multiple pixel groups in the target image data is located.

[0056] As mentioned above, the pixel groups composed of the display screen pixels are periodically distributed. In practice, the pixel group division can be assisted by images captured by a CCD camera (an industrial-grade high-precision camera).

[0057] However, CCD cameras have a limited field of view and can only capture a portion of the pixels in a local view (such as 21×25 pixels). Therefore, it is difficult to determine which pixel group these pixels in the local view belong to when directly capturing images from the display screen.

[0058] Therefore, multiple bright spots can be arranged in a preset manner on the display screen, such as arranging one or more bright spots in each positioning cycle (each positioning cycle consists of 21×25 pixels). In this way, when the bright spots are lit, an image is captured on the display screen to obtain a first calibration image. Then, by combining at least one bright spot in the first calibration image, the initial regions corresponding to each of the multiple pixel groups in the first calibration image can be obtained, that is, which pixel group each pixel in the first calibration image belongs to.

[0059] See Figure 5 The first calibration image shown, combined with the four bright spots it contains, is used for pixel group recognition to obtain the following: Figure 6 The initial regions corresponding to the multiple pixel groups shown.

[0060] The positioning cycle (denoted as positioning cycle i) includes the 29th to 50th pixels of the j-th pixel cycle and the 1st to 3rd pixels of the (j+1)-th pixel cycle. It can be deduced that the positioning cycle i-1 includes the 4th to 28th pixels of the j-th pixel cycle. It can also be deduced that the positioning cycle i+1 includes the 4th to 28th pixels of the (j+1)-th pixel cycle.

[0061] The above Figure 5 and Figure 6 Taking setting one bright spot at each of the four corners of a positioning cycle as an example, in practice, other methods can also be used to set bright spots, such as setting only one bright spot in the middle position on the left side (the side not adjacent to the 3-pixel group).

[0062] Since a positioning cycle cannot contain a complete pixel cycle, such as Figure 5 and Figure 6 As shown, one pixel cycle spans two positioning cycles. Therefore, in order to distinguish between the two positioning cycles, the bright spots on the side adjacent to the 3-pixel group and the bright spots on the side not adjacent to the 3-pixel group can be displayed differently (such as different brightness of the bright spots or different number of bright spots). In this way, for the first calibration image corresponding to one positioning cycle, the initial area corresponding to each of the multiple pixel groups contained therein can be accurately obtained.

[0063] Since the first calibration image corresponds to the target viewpoint, while the entire display screen needs to display the global viewpoint, and the pixel groups composed of the display screen pixels are periodically distributed, pixel group analysis of the target image data based on the initial regions corresponding to the multiple pixel groups in the first calibration image can obtain the pixel regions where the multiple pixel groups in the target image data are located.

[0064] In some embodiments, the above-described pixel group analysis of the target image data can be achieved, but is not limited to, the following methods: Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, the pixel regions where the pixel groups corresponding to the target viewpoint are located in the target image data are obtained. Based on the pixel region where the pixel group corresponding to the target viewpoint is located, and combined with the preset pixel group distribution method, the pixel region where the pixel group corresponding to other viewpoints in the target image data is located is obtained.

[0065] Since the initial regions corresponding to the multiple pixel groups in the first calibration image are only local pixels under the target viewpoint, while the target image data contains pixels from the global viewpoint, we can first map the initial regions corresponding to the multiple pixel groups in the first calibration image to the pixel regions of the pixel groups under the same viewpoint (target viewpoint) in the target image data; then, based on the pixel regions of the pixel groups corresponding to the target viewpoint, combined with a preset pixel group distribution method, we can obtain the pixel regions of the pixel groups corresponding to other viewpoints in the target image data.

[0066] Or should we correspond to the target perspective? Figure 3 Taking the upper left corner of the thick outline shown as an example, by combining Figure 2 The periodic distribution shown, and Figure 3 The cyclical pattern shown can be used to obtain the preset pixel group distribution pattern, and then to deduce the pixel group to which each pixel belongs, that is, the pixel region corresponding to each pixel group.

[0067] In some embodiments, the pixel group identification of the first calibration image described above can be achieved, but is not limited to, the following methods: The first calibration image is input into the segmentation model. The segmentation model, combined with at least one bright spot, performs grid segmentation on the first calibration image to obtain the grid segmentation result corresponding to the first calibration image. Based on the grid segmentation results, the first calibration image is divided into initial regions corresponding to multiple pixel groups.

[0068] In this embodiment, a trained segmentation model is used to perform grid segmentation on the first calibration image by combining at least one bright spot. In this way, the first calibration image is divided into multiple grids (each grid corresponds to one pixel), and a grid segmentation result is obtained. Then, based on the grid segmentation result and combined with a preset pixel group distribution method, the first calibration image is divided into initial regions corresponding to multiple pixel groups.

[0069] In some embodiments, after obtaining the grid segmentation result, the grid segmentation result can be further corrected so that the pixel group region can be divided more accurately in the subsequent process.

[0070] For example, based on the above mesh segmentation results and the ideal mesh distribution, target correction parameters (such as geometric correction matrix, perspective transformation matrix, or a transformation matrix obtained by combining the two) are obtained, and the mesh segmentation results are corrected using the target correction parameters.

[0071] The following concrete example will illustrate how to perform pixel group identification on the first calibration image: First, each grid is preprocessed based on the grid segmentation results.

[0072] For example, in the mesh segmentation result, the boundary of each mesh is checked to see if it falls within the effective image range. If the first calibration image is in color mode, it is converted to grayscale mode for subsequent processing. For non-rectangular meshes, the mesh vertices are mapped to the local coordinate system through coordinate transformation, and the corresponding polygon mask is generated and filled to accurately cover the effective area of ​​the mesh.

[0073] Next, based on the position (row and column coordinates) of each preprocessed grid, the image region of each grid is extracted.

[0074] Then, pixel regions are extracted for each image region, and pixel groups are divided.

[0075] For example, to eliminate the interference of table grid lines on content extraction, a method combining morphological operations and masking is used: first, an erosion operation is applied to the grid area to weaken or remove the grid lines; then, a dilation operation is performed to restore the size of the grid content area; finally, a bitwise AND operation is performed between the processed image area and the grid mask to obtain the pixel area with clean content after removing the grid lines. Combined with a preset pixel group distribution method, the initial area corresponding to each pixel group can be determined.

[0076] This application does not limit the specific implementation of the segmentation model. For example, the segmentation model includes a main architecture, such as a Swing Window Transformer network (corresponding to basic parameters), and also includes multiple branches corresponding to functions.

[0077] In implementation, the segmentation model combines the bright spots in the first calibration image to perform grid segmentation. Therefore, the segmentation model needs to have both bright spot detection and grid segmentation functions. Thus, the segmentation model may include a first branch (corresponding to the first parameter) for bright spot detection and a second branch (corresponding to the second parameter) for grid segmentation.

[0078] In addition, for pixel-level tasks, in order to ensure the reliability of the segmentation model, a third branch (corresponding to the third parameter) can be set in the segmentation model corresponding to the overall confidence level.

[0079] Taking the main architecture using the Swing Transformer network as an example, after the image is input into the segmentation model, the segmentation model divides it into multiple image patches, extracts the initial features of each image patch, and injects positional encoding in the image space into each image patch, so that the segmentation model can explicitly perceive the relative position and layout relationship of each image patch in the image space. The initial features of multiple image patches are fused using the attention mechanism of the Swing Transformer network to obtain fused features. For each branch, the corresponding result is obtained based on the fused features.

[0080] Taking the three branches mentioned above as an example, the first branch outputs the location and probability of the bright spots in the image, which is used to capture local detail features; the second branch outputs the grid segmentation result of the image, which represents the global geometric layout; and the third branch outputs the confidence score, which is used to evaluate the reliability of the prediction result.

[0081] For example, the attention mechanism in the above-mentioned Swin Transformer network can be implemented in the following way: Layers 1-4: Local Window Attention.

[0082] In layers 1-4, feature maps are divided using fixed-size windows (7×7 pixels), and self-attention computation is performed within each window. This stage focuses on detecting local bright spots and extracting high-resolution details such as key points and corners. Information exchange between windows is handled by moving windows or global attention mechanisms in subsequent layers.

[0083] To further enhance the model's ability to model non-rigid deformations, deformable attention modules can be introduced in layers 1-4. By learning the spatial offset field, the position of the attention sampling points can be dynamically adjusted, effectively handling complex deformations and irregular structures.

[0084] Layers 5-8: Global Row-Column Attention.

[0085] In layers 5-8, attention calculations are performed along the row and column directions at the feature map level: all image patches within the same row interact to model horizontal structural relationships. All image patches within the same column interact to model vertical structural relationships. This stage achieves global structure awareness, providing row and column aligned features for grid parameter regression.

[0086] Layers 9-12: Grid Parameter Regression Attention.

[0087] A learnable grid start symbol ([GRID_ORIGIN]) is introduced as a query vector in layers 9-12 to aggregate the fused features of the global row and column attention output.

[0088] In the Swin Transformer network, image features are extracted at multiple scales through the hierarchical shift window attention mechanism described above, while effectively modeling the semantic relationship between local details and global structure.

[0089] In some embodiments, the segmentation model can be obtained in, but is not limited to, the following ways: The segmentation model is trained multiple times using the sample set to adjust the first parameter corresponding to the bright spot detection in the segmentation model. The segmentation model is trained multiple times using the sample set to adjust the target parameters in the segmentation model. The target parameters include the second parameter corresponding to the mesh segmentation in the segmentation model and the first parameter mentioned above.

[0090] Since the bright spot detection function is the foundation for implementing other functions, it is necessary to first adjust the first parameter corresponding to bright spot detection in the segmentation model based on the sample set, so that the segmentation model has a certain bright spot detection function. Then, the parameters of the segmentation model as a whole (e.g., including the above basic parameters, first parameter, second parameter and third parameter) are adjusted based on the sample set.

[0091] Specifically, in the first iteration training phase, the parameters of other functional branches are frozen, and only the first parameter is adjusted; in the second iteration training phase, the parameters of other functional branches are unfrozen, and some of the first parameters are frozen (in this phase, only some of the first parameters can be fine-tuned), and the overall parameters are adjusted.

[0092] In some embodiments, the sample set contains multiple samples, and each sample contains a sample calibration image; Among them, the sample calibration images of the target samples in multiple samples contain at least one bright spot, and the sample calibration images of at least some of the target samples contain different numbers of bright spots.

[0093] During implementation, due to issues such as sampling angle deviation, the first calibration image may not contain all the bright spots in a positioning cycle. Therefore, it is necessary to train the segmentation model to have a certain ability to fill in bright spots when they are missing.

[0094] Based on this, the target samples in the sample set need to cover the sample calibration images with different numbers of bright spots, and provide some non-target samples (as extreme degradation samples, so that the segmentation model can cope with the calibration images with blurred, overexposed or underexposed bright spots).

[0095] For example, the proportion of sample calibration images containing 4 clear bright spots is 60%; the proportion of sample calibration images containing 1 to 3 clear bright spots is 25%; and the proportion of non-target samples is 15%.

[0096] In some embodiments, each first iteration of training includes: The first sample is selected from the sample set, and the first sample labeling image in the first sample is input into the segmentation model. The segmentation model is used to perform bright spot detection on the first sample labeling image to obtain the corresponding first predicted bright spot detection result. A first loss is obtained to represent the difference between the first predicted bright spot detection result and the actual bright spot detection result in the first sample, and the first parameter is adjusted based on the first loss.

[0097] In the first iteration of training, only the first parameter of the first branch is adjusted. Therefore, the loss related to bright spot detection is mainly considered at this stage.

[0098] Based on this, the first sample calibration image from the first sample is input into the segmentation model. The segmentation model performs bright spot detection on the first sample calibration image to obtain the corresponding first predicted bright spot detection result. The closer the first predicted bright spot detection result is to the actual bright spot detection result in the first sample, the more accurate the prediction of the first branch is. Conversely, the greater the difference between the first predicted bright spot detection result and the actual bright spot detection result, the less accurate the prediction of the first branch is, and the more the first parameter needs to be adjusted. Therefore, a first loss representing the difference between the first predicted bright spot detection result and the actual bright spot detection result in the first sample is obtained, and the first parameter is adjusted based on the first loss.

[0099] For example, both the first predicted bright spot detection result and the actual bright spot detection result contain a bright spot label for each image block in the first sample labeled image (representing the probability of whether the corresponding image block contains a bright spot; the bright spot label in the first predicted bright spot detection result is the predicted bright spot label, and the bright spot detection result in the actual bright spot detection result is the labeled bright spot label). The first loss is obtained based on the difference between the predicted bright spot label and the labeled bright spot label for each image block.

[0100] The formula is expressed as: the first loss of the first sample calibration image l Where N represents the total number of image patches in any sample calibration image, BCE represents the cross-entropy loss, and p i,l Let y represent the predicted bright spot label of the i-th image patch. i,l The label represents the highlight of the i-th image patch.

[0101] In some embodiments, each second iteration of training includes: Select a second sample from the sample set and input the second sample labeling image from the second sample into the segmentation model. The segmentation model performs bright spot detection on the second sample labeling image to obtain the corresponding second predicted bright spot detection result. The second sample calibration image is segmented into grids using a segmentation model to obtain the corresponding predicted grid distribution. A second loss is obtained, representing the difference between the second predicted bright spot detection result and the actual bright spot detection result in the second sample, and a third loss is obtained, representing the difference between the predicted grid distribution and the actual grid distribution in the second sample; The target parameters are adjusted by combining the second and third losses.

[0102] In the second iteration of training, the global parameters need to be adjusted. Therefore, at this stage, we need to consider not only the loss related to bright spot detection, but also the loss related to grid segmentation.

[0103] Based on this, the second sample calibration image from the second sample is input into the segmentation model. The segmentation model performs bright spot detection on the second sample calibration image to obtain the corresponding second predicted bright spot detection result. The segmentation model then performs grid segmentation on the second sample calibration image to obtain the corresponding predicted grid distribution. The closer the second predicted bright spot detection result is to the actual bright spot detection result in the second sample, the more accurate the prediction of the first branch is; conversely, the greater the difference between the second predicted bright spot detection result and the actual bright spot detection result, the less accurate the prediction of the first branch is, and the more the target parameters need adjustment. Similarly, the closer the predicted grid distribution is to the actual grid distribution in the second sample, the more accurate the prediction of the second branch is; conversely, the greater the difference between the predicted grid distribution and the actual grid distribution, the less accurate the prediction of the second branch is, and the more the target parameters need adjustment. Therefore, a second loss representing the difference between the second predicted bright spot detection result and the corresponding actual bright spot detection result is obtained, and a third loss representing the difference between the predicted grid distribution and the corresponding actual grid distribution is obtained. The target parameters are adjusted based on the second and third losses; for example, the weighted sum of the second and third losses is used as the target loss, and the target parameters are adjusted based on the target loss.

[0104] For example, the second loss is calculated in a similar way to the first loss described above, and is expressed by the formula: the second loss of the second sample labeled image k Where N represents the total number of image patches in any sample calibration image, BCE represents the cross-entropy loss, and p i,k Let y represent the predicted bright spot label of the i-th image patch. i,k The label represents the highlight of the i-th image patch.

[0105] In practice, because the number of bright spots in the sample calibration image may be incomplete, the segmentation model cannot know in advance whether some bright spots correspond to the top left corner, bottom right corner, or a certain position in the middle of the actual grid. Therefore, if the predicted grid distribution is forced to correspond to the actual grid distribution in a specific order (pixel), the order will be disordered once a bright spot is missing, and the gradient will backpropagate, causing the model to crash. Therefore, the predicted bright spots can be projected to the nearest grid intersection (pixel corner). In this way, it only needs to automatically find the nearest grid intersection to calculate the distance. Thus, regardless of the order of the bright spots output by the segmentation model, as long as they all fall near the grid intersection, the loss is small.

[0106] For example, the third loss of the second sample labeled image k Where M is the predicted total number of bright spots, and S j Let G be the coordinates of the j-th predicted bright spot, and G be the actual grid distribution. G (S j In the actual grid distribution, ) is related to S j The latest grid. This indicates that the projection is onto the nearest grid.

[0107] Furthermore, by combining the weights corresponding to the second and third losses (the weights can be adjustable parameters or preset values), the two are weighted and summed to obtain the target loss.

[0108] The formula is expressed as: the target loss L of the second sample calibration image k 0,k =α1 L 2,k +α2 L 3,k Among them, L 2,k The second loss is used to calibrate the image k of the second sample, where α1 is the weight corresponding to the second loss, and L 3,k The third loss is used to calibrate the image k of the second sample, and α2 is the weight corresponding to the third loss.

[0109] In some embodiments, the target grayscale value mapping model is obtained in the following manner: First, when multiple bright spots arranged in a preset pattern on the aforementioned display screen are lit, multiple images are captured on the display screen to obtain multiple second calibration images.

[0110] As mentioned above, CCD cameras have a limited field of view and can only capture a portion of the pixels within a local view. Therefore, it is difficult to determine which pixel group a portion of the pixels in a local view belongs to when directly capturing an image from a display screen.

[0111] Based on this, during the calibration stage of the grayscale mapping model, it is also necessary to perform multiple image acquisitions on the display screen when multiple bright spots arranged in a preset manner are lit, to obtain multiple second calibration images, and to perform pixel group recognition by combining the bright spot images in each second calibration image.

[0112] The acquisition method of the second calibration image is similar to that of the first calibration image, except that the second calibration image is not limited to the above-mentioned target viewpoint.

[0113] The method for pixel group identification of the second calibration image can be the same as the method for pixel group identification of the first calibration image described above, and will not be repeated here.

[0114] However, the second calibration image is used in the subsequent process of establishing the preset grayscale value mapping model. Therefore, after performing pixel group recognition to obtain the initial regions corresponding to each of the multiple pixel groups in the second calibration image, the following steps can also be performed: For each pixel group, calculate its actual brightness value for the initial region, including but not limited to the average brightness, brightness variance, maximum and minimum values, and histogram distribution of multiple pixels within the pixel group.

[0115] Then, the actual brightness values ​​can be mapped to a data matrix according to the row and column organization structure of pixel groups, thereby generating a standard format CSV file to ensure the structural integrity and traceability of the output data for subsequent use.

[0116] Next, for any second calibration image, the display parameters corresponding to each of the multiple pixel groups in the second calibration image are obtained; wherein, each display parameter includes the actual brightness value and the actual grayscale value of the corresponding pixel group.

[0117] In practice, for each pixel group of each second calibration image, based on its initial region, the corresponding actual brightness value (actual display effect) and actual grayscale value (actual driving parameters) can be obtained.

[0118] Then, based on the obtained display parameters, at least one preset grayscale value mapping model is established.

[0119] In this embodiment of the application, based on multiple obtained display parameters, at least one preset grayscale value mapping model can be established using brightness as a bridge.

[0120] Finally, select the target grayscale value mapping model from at least one preset grayscale value mapping model.

[0121] In practice, if only one preset grayscale value mapping model is established, it can be directly used as the target grayscale value mapping model; if multiple preset grayscale value mapping models are established, one of them needs to be selected as the target grayscale value mapping model.

[0122] See Figure 7 and Figure 8 As shown, in practice, the global view can be divided into multiple preset view ranges, and a preset grayscale value mapping model can be established for each view range. In this way, more refined partition grayscale compensation is achieved.

[0123] Correspondingly, the establishment of at least one preset grayscale value mapping model can be achieved in, but is not limited to, the following ways: For any preset viewing angle range among multiple preset viewing angle ranges, a preset grayscale value mapping model corresponding to the preset viewing angle range is established based on the display parameters corresponding to the second calibration image of the preset viewing angle range among multiple display parameters.

[0124] In this embodiment of the application, the global view is divided into multiple preset view ranges, and a second calibration image is obtained for each preset view range. Based on the display parameters of these calibration images, a preset grayscale value mapping model for the corresponding preset view range is established.

[0125] Correspondingly, the above-mentioned selection of the target grayscale value mapping model can be achieved through, but is not limited to, the following methods: From multiple preset viewing angle ranges, obtain the preset viewing angle range that matches the viewing angle of the pixel group, and select the preset grayscale value mapping model corresponding to the preset viewing angle range that matches the viewing angle as the target grayscale value mapping model.

[0126] Based on the above Figure 7 For example, the global viewpoint is divided into five preset viewpoint ranges: top left (A1), top right (A2), center (A3), bottom left (A4), and bottom right (A5). Based on the display parameters of the second calibration image for each preset viewpoint range, a preset grayscale value mapping model is established for that preset viewpoint range. The viewpoint containing the pixel group to be compensated within each of the five preset viewpoint ranges is determined, and its corresponding preset grayscale value mapping model is used as the target grayscale value mapping model.

[0127] Based on the above Figure 7For example, the global viewpoint is divided into three preset viewpoint ranges: top (B1), middle (B2), and bottom (B3). Based on the display parameters of the second calibration image for each preset viewpoint range, a preset grayscale value mapping model is established for that preset viewpoint range. The viewpoint containing the pixel group to be compensated within each of the three preset viewpoint ranges is determined, and its corresponding preset grayscale value mapping model is used as the target grayscale value mapping model.

[0128] In some embodiments, the above-mentioned establishment of the preset grayscale value mapping model corresponding to the preset viewing angle range can be achieved in, but is not limited to, the following ways: First, for any pixel group type, based on the display parameters corresponding to that pixel group type in the second calibration image of the preset viewing angle range among multiple display parameters, a corresponding first brightness model is established; wherein, the first brightness model is used to describe the correlation between the actual brightness value and the actual grayscale value in the pixel group type for the preset viewing angle range.

[0129] In practice, for each preset viewing angle range, a first brightness model is established based on the display parameters of the same pixel group type to represent the correlation between the actual brightness value and the actual grayscale value in that pixel group type.

[0130] The embodiments of this application do not specifically limit the implementation of the first brightness model. The selected model type can be determined according to the change of the actual brightness value with the actual grayscale value.

[0131] Taking the first brightness model as an example, which is a cubic polynomial: The first luminance model of a 1-pixel group (Block 1) is represented as Y1=a1x 3 +b1x 2 +c1x+d1; The first luminance model for a 2-pixel group (Block 2) is represented as Y2=a2x 3 +b2x 2 +c2x+d2; The first luminance model for a 3-pixel group (Block 3) is represented as Y3=a3x 3 +b3x 2 +c3x+d3; Each first brightness model has four coefficients to be solved. Therefore, only four corresponding display parameters are needed to obtain the corresponding first brightness model.

[0132] Then, based on the established first brightness models within the preset viewing angle range, the actual grayscale value of the reference pixel group type is used as the initial grayscale value, and the actual grayscale value of at least one non-reference pixel group type under the same brightness is used as the corresponding target grayscale value, thus establishing a preset grayscale value mapping model corresponding to the preset viewing angle range.

[0133] Taking the first brightness model as an example of the above cubic polynomial, the brightness difference between pixel group 1 and pixel group 2 is expressed as: Y 1-2 =(a1-a2)x 3 +(b1-b2)x 2 +(c1-c2)x+(d1-d2).

[0134] As can be seen, different grayscale values ​​are required for different pixel group types to achieve the same brightness.

[0135] Based on this, in this embodiment of the application, a pixel group type is selected as the reference pixel group type, and the other pixel group types are non-reference pixel group types. The actual grayscale value of the reference pixel group type is used as the initial grayscale value, and the actual grayscale value of at least one non-reference pixel group type under the same brightness is used as the corresponding target grayscale value. In this way, a preset grayscale value mapping model corresponding to the preset viewing angle range is established.

[0136] Specifically, the preset grayscale value mapping model can include a mapping relationship table as shown in Table 1: Table 1

[0137] The above example uses a preset grayscale value mapping model containing a mapping relationship table. In practice, a mapping relationship function can also be established.

[0138] See Figure 9 As shown, in practice, the global perspective can be divided without dividing it, and a preset grayscale value mapping model corresponding to the global perspective C can be directly established.

[0139] Correspondingly, the establishment of at least one preset grayscale value mapping model can be achieved in, but is not limited to, the following ways: Based on multiple display parameters, establish a preset grayscale value mapping model; Correspondingly, the above-mentioned selection of the target grayscale value mapping model can be achieved through, but is not limited to, the following methods: Use this preset grayscale mapping model as the target grayscale value mapping model.

[0140] In some embodiments, the above-mentioned establishment of a preset grayscale value mapping model can be achieved in, but is not limited to, the following ways: For any pixel group type, a corresponding second brightness model is established based on the display parameters corresponding to that pixel group type among multiple display parameters; wherein, the second brightness model is used to describe the relationship between the actual brightness value and the actual grayscale value in that pixel group type from a global perspective. Based on the established multiple second brightness models, a preset grayscale value mapping model is established, using the actual grayscale value of the reference pixel group type as the initial grayscale value and the actual grayscale value of at least one non-reference pixel group type under the same brightness as the corresponding target grayscale value.

[0141] The specific implementation method of establishing a preset grayscale value mapping model in this embodiment is similar to the specific implementation method of establishing a preset grayscale value mapping model corresponding to a certain preset viewing angle range, the only difference being the display parameters selected when establishing the brightness model (the former does not distinguish between viewing angles, while the latter does). Therefore, it will not be described in detail here.

[0142] In some embodiments, prior to step S402 described above, the following steps are also performed: The pixel group type of the above pixel group is determined to be any non-reference pixel group type.

[0143] In practice, since the grayscale value of the reference pixel group type is itself the target grayscale value used as the reference standard, the reference pixel group type of pixel group does not need to be compensated for grayscale.

[0144] Therefore, only when a pixel group is determined to be a non-reference pixel group type is subsequent compensation required.

[0145] In practice, based on any embodiment, after displaying an image on the display screen based on the target image data, the uniformity of the displayed image can be further evaluated to determine whether the target grayscale value mapping model needs to be adjusted to further improve the display quality of the display screen.

[0146] like Figure 10 As shown, this disclosure provides a first image data processing apparatus 1000, which includes: The initial grayscale acquisition module 1001 is used to obtain the initial grayscale value corresponding to the pixel region where any pixel group is located in the target image data. The target grayscale acquisition module 1002 is used to determine the target grayscale value corresponding to the pixel group based on the target grayscale value mapping model, combined with the pixel group type and the initial grayscale value of the pixel group; wherein, the target grayscale value mapping model is used to describe the correlation between the initial grayscale value and the target grayscale value of at least one pixel group type; The grayscale compensation module 1003 is used to replace the initial grayscale value of the pixel region where the pixel group is located in the target image data with the target grayscale value, and display the image on the display screen based on the target image data.

[0147] In some embodiments, a pixel group identification module 1004 is further included, which is used to obtain the pixel region where each of the multiple pixel groups in the target image data is located by means of: Pixel group identification is performed on a first calibration image containing at least one bright spot to obtain the initial region corresponding to each of the multiple pixel groups in the first calibration image; wherein, the first calibration image is obtained by capturing the display screen from a target viewing angle when multiple bright spots arranged in a preset manner on the display screen are lit. Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, pixel group analysis is performed on the target image data to obtain the pixel regions where each of the multiple pixel groups in the target image data is located.

[0148] In some embodiments, the pixel group recognition module 1004 is specifically used for: The first calibration image is input into the segmentation model. The segmentation model, combined with at least one bright spot, performs grid segmentation on the first calibration image to obtain the grid segmentation result corresponding to the first calibration image. Based on the grid segmentation results, the first calibration image is divided into initial regions corresponding to multiple pixel groups.

[0149] In some embodiments, the pixel group recognition module 1004 is specifically used for: Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, the pixel regions where the pixel groups corresponding to the target viewpoint are located in the target image data are obtained. Based on the pixel region where the pixel group corresponding to the target viewpoint is located, and combined with the preset pixel group distribution method, the pixel region where the pixel group corresponding to other viewpoints in the target image data is located is obtained.

[0150] In some embodiments, the segmentation model is obtained in the following manner: The segmentation model is trained multiple times using the sample set to adjust the first parameter corresponding to the bright spot detection in the segmentation model. The segmentation model is trained multiple times using the sample set to adjust the target parameters in the segmentation model. The target parameters include the second parameter and the first parameter corresponding to the grid segmentation in the segmentation model.

[0151] In some embodiments, the sample set includes multiple samples, each sample including a sample calibration image; wherein the calibration image of a target sample among the multiple samples contains at least one bright spot, and the number of bright spots contained in the calibration images of at least some of the target samples is different.

[0152] In some embodiments, each first iteration of training includes: Select a first sample from the sample set and input the first sample labeling image of the first sample into the segmentation model. Perform bright spot detection on the first sample labeling image through the segmentation model to obtain the corresponding first predicted bright spot detection result. A first loss is obtained to represent the difference between the first predicted bright spot detection result and the actual bright spot detection result in the first sample, and the first parameter is adjusted based on the first loss.

[0153] In some embodiments, each second iteration of training includes: A second sample is selected from the sample set, and the labeled image of the second sample is input into the segmentation model. The segmentation model performs bright spot detection on the labeled image of the second sample to obtain the corresponding second predicted bright spot detection result; and... The second sample calibration image is segmented into grids using a segmentation model to obtain the corresponding predicted grid distribution. A second loss is obtained, representing the difference between the second predicted bright spot detection result and the actual bright spot detection result in the second sample, and a third loss is obtained, representing the difference between the predicted grid distribution and the actual grid distribution in the second sample; The target parameters are adjusted by combining the second and third losses.

[0154] In some embodiments, a mapping establishment module 1005 is further included, for obtaining a target grayscale value mapping model in the following manner: When multiple bright spots arranged in a preset pattern on the display screen are lit, multiple images are captured on the display screen to obtain multiple second calibration images; For any given second calibration image, obtain the display parameters corresponding to each of the multiple pixel groups in the second calibration image; wherein, each display parameter includes the actual brightness value and the actual grayscale value of the corresponding pixel group; Based on the obtained multiple display parameters, establish at least one preset grayscale value mapping model; Select the target grayscale value mapping model from at least one preset grayscale value mapping model.

[0155] In some embodiments, the mapping establishment module 1005 is specifically used for: For any preset viewing angle range among multiple preset viewing angle ranges, based on the display parameters corresponding to the second calibration image of the preset viewing angle range among multiple display parameters, a preset grayscale value mapping model corresponding to the preset viewing angle range is established. From multiple preset viewing angle ranges, obtain the preset viewing angle range that matches the viewing angle of the pixel group, and select the preset grayscale value mapping model corresponding to the preset viewing angle range that matches the viewing angle as the target grayscale value mapping model.

[0156] In some embodiments, the mapping establishment module 1005 is specifically used for: Based on multiple display parameters, establish a preset grayscale value mapping model; Use a pre-defined grayscale mapping model as the target grayscale value mapping model.

[0157] In some embodiments, the mapping establishment module 1005 is specifically used for: For any pixel group type, based on the display parameters corresponding to that pixel group type in the second calibration image of the preset viewing angle range among multiple display parameters, a corresponding first brightness model is established; wherein, the first brightness model is used to describe the correlation between the actual brightness value and the actual grayscale value in the pixel group type for the preset viewing angle range. Based on the established first brightness models within the preset viewing angle range, the actual grayscale value of the reference pixel group type is used as the initial grayscale value, and the actual grayscale value of at least one non-reference pixel group type under the same brightness is used as the corresponding target grayscale value, thereby establishing a preset grayscale value mapping model corresponding to the preset viewing angle range.

[0158] In some embodiments, the mapping establishment module 1005 is specifically used for: For any pixel group type, a corresponding second brightness model is established based on the display parameters corresponding to that pixel group type among multiple display parameters; wherein, the second brightness model is used to describe the relationship between the actual brightness value and the actual grayscale value in that pixel group type from a global perspective. Based on the established multiple second brightness models, a preset grayscale value mapping model is established, using the actual grayscale value of the reference pixel group type as the initial grayscale value and the actual grayscale value of at least one non-reference pixel group type under the same brightness as the corresponding target grayscale value.

[0159] In some embodiments, before determining the target grayscale value corresponding to the pixel group, the target grayscale acquisition module 1002 is further configured to: Determine the pixel group type of this pixel group as any non-reference pixel group type.

[0160] For a description of the features in the embodiment corresponding to the image data processing apparatus, please refer to the relevant description in the embodiment corresponding to the image data processing method, which will not be repeated here.

[0161] Some embodiments of this disclosure provide an electronic device, such as Figure 11 As shown, it includes at least one processor 1101 and a memory 1102 connected to at least one processor. In this embodiment, the specific connection medium between the processor 1101 and the memory 1102 is not limited. Figure 11Taking the connection between processor 1101 and memory 1102 via bus 1103 as an example. Buses can be categorized into path buses, data buses, control buses, etc. For ease of illustration, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0162] The processor 1101 is the control center of the electronic device, capable of connecting various parts of the device via various interfaces and lines. It performs data processing by running or executing instructions stored in the memory 1102 and accessing data stored in the memory 1102. Optionally, the processor 1101 may include one or more processing units. The processor 1101 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles issuing instructions. It is understood that the modem processor may not be integrated into the processor 1101. In some embodiments, the processor 1101 and the memory 1102 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0163] The processor 1101 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the image data processing method can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0164] Memory 1102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1102 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 1102 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Memory 1102 in this embodiment may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0165] In this embodiment of the disclosure, the memory 1102 stores a computer program (Agent) that, when executed by the processor 1101, causes the processor 1101 to implement the image data processing method as described in any of the embodiments above.

[0166] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer (e.g., an electronic device), cause the computer to perform an image data processing method as described in any of the above embodiments.

[0167] Exemplary examples of computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., CDs (Compact Disks), DVDs (Digital Versatile Disks), etc.), smart cards, and flash memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memory), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0168] Some embodiments of this disclosure also provide a computer program product, for example, stored on a non-transitory computer-readable storage medium. The computer program product includes computer program instructions that, when executed on a computer (e.g., an electronic device), cause the computer to perform the image data processing method as described in the above embodiments.

[0169] Some embodiments of this disclosure also provide a computer program. When executed on a computer (e.g., an electronic device), the computer program causes the computer to perform the image data processing method as described in the above embodiments.

[0170] The beneficial effects of the aforementioned computer-readable storage medium, computer program product, and computer program are the same as the beneficial effects of the image data processing methods described in some of the above embodiments, and will not be repeated here.

[0171] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this disclosure, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0172] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image data processing method, characterized in that, include: For any group of pixels in the target image data, obtain the initial grayscale value corresponding to the pixel region where the group of pixels is located; Based on the target grayscale value mapping model, combined with the pixel group type and the initial grayscale value, the target grayscale value corresponding to the pixel group is determined; wherein, the target grayscale value mapping model is used to describe the correlation between the initial grayscale value and the target grayscale value of at least one pixel group type. The initial grayscale value of the pixel region where the pixel group is located in the target image data is replaced with the target grayscale value, and the image is displayed on the display screen based on the target image data.

2. The method as described in claim 1, characterized in that, The pixel regions of each of the multiple pixel groups in the target image data are obtained in the following way: Pixel group identification is performed on a first calibration image containing at least one bright spot to obtain the initial region corresponding to each of the multiple pixel groups in the first calibration image; wherein, the first calibration image is obtained by capturing the display screen from a target viewing angle when multiple bright spots arranged in a preset manner on the display screen are lit. Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, pixel group analysis is performed on the target image data to obtain the pixel regions where each of the multiple pixel groups in the target image data is located.

3. The method as described in claim 2, characterized in that, Pixel group identification is performed on a first calibration image containing at least one bright spot to obtain initial regions corresponding to each of multiple pixel groups in the first calibration image, including: The first calibration image is input into the segmentation model. The first calibration image is then segmented into grids using the segmentation model, in conjunction with the at least one bright spot, to obtain the grid segmentation result corresponding to the first calibration image. Based on the grid segmentation results, the first calibration image is divided into initial regions corresponding to each of the plurality of pixel groups.

4. The method as described in claim 2, characterized in that, Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, pixel group analysis is performed on the target image data to obtain the pixel regions where each of the multiple pixel groups in the target image data is located, including: Based on the initial regions corresponding to each of the multiple pixel groups in the first calibration image, the pixel regions where the pixel groups corresponding to the target viewpoint are located in the target image data are obtained. Based on the pixel region where the pixel group corresponding to the target viewpoint is located, and combined with the preset pixel group distribution method, the pixel region where the pixel group corresponding to other viewpoints in the target image data is located is obtained.

5. The method as described in claim 3, characterized in that, The segmentation model is obtained through the following method: The segmentation model is trained multiple times using the sample set to adjust the first parameter corresponding to the bright spot detection in the segmentation model. The segmentation model is trained multiple times using the sample set to adjust the target parameters in the segmentation model; wherein the target parameters include the second parameter and the first parameter corresponding to the grid segmentation in the segmentation model.

6. The method as described in claim 5, characterized in that, The sample set contains multiple samples, each containing a sample calibration image; wherein, the calibration image of a target sample among the multiple samples contains at least one bright spot, and the number of bright spots contained in the calibration images of at least some target samples is different.

7. The method as described in claim 5 or 6, characterized in that, Each first iteration of training includes: A first sample is selected from the sample set, and the first sample labeling image in the first sample is input into the segmentation model. The segmentation model is used to perform bright spot detection on the first sample labeling image to obtain the corresponding first predicted bright spot detection result. A first loss is obtained to represent the difference between the first predicted bright spot detection result and the actual bright spot detection result in the first sample, and the first parameter is adjusted based on the first loss.

8. The method as described in claim 5 or 6, characterized in that, Each second iteration of training includes: A second sample is selected from the sample set, and the second sample's labeled image is input into the segmentation model. The segmentation model then performs bright spot detection on the second sample's labeled image to obtain the corresponding second predicted bright spot detection result; and... The second sample calibration image is segmented into grids using the segmentation model to obtain the corresponding predicted grid distribution. A second loss is obtained to represent the difference between the second predicted bright spot detection result and the actual bright spot detection result in the second sample, and a third loss is obtained to represent the difference between the predicted grid distribution and the actual grid distribution in the second sample; The target parameter is adjusted by combining the second loss and the third loss.

9. The method according to any one of claims 1 to 6, characterized in that, The target grayscale value mapping model is obtained through the following method: When multiple bright spots arranged in a preset pattern on the display screen are lit, multiple images are captured on the display screen to obtain multiple second calibration images; For any second calibration image, obtain the display parameters corresponding to each of the multiple pixel groups in the second calibration image; wherein, each display parameter includes the actual brightness value and the actual grayscale value of the corresponding pixel group; Based on the obtained multiple display parameters, establish at least one preset grayscale value mapping model; Select the target grayscale value mapping model from at least one preset grayscale value mapping model.

10. The method as described in claim 9, characterized in that, Based on the obtained multiple display parameters, establish at least one preset grayscale value mapping model, including: For any preset viewing angle range among multiple preset viewing angle ranges, a preset grayscale value mapping model corresponding to the preset viewing angle range is established based on the display parameters corresponding to the second calibration image of the preset viewing angle range among the multiple display parameters. Selecting the target grayscale value mapping model from at least one preset grayscale value mapping model includes: From multiple preset viewing angle ranges, a preset viewing angle range that matches the viewing angle of the pixel group is obtained, and a preset grayscale value mapping model corresponding to the preset viewing angle range that matches the viewing angle is selected as the target grayscale value mapping model.

11. The method as described in claim 9, characterized in that, Based on the obtained multiple display parameters, establish at least one preset grayscale value mapping model, including: Based on the multiple display parameters, a preset grayscale value mapping model is established; Selecting the target grayscale value mapping model from at least one preset grayscale value mapping model includes: The preset grayscale mapping model is used as the target grayscale value mapping model.

12. The method as described in claim 9, characterized in that, Based on the display parameters corresponding to the second calibration image within the preset viewing angle range from the plurality of display parameters, a preset grayscale value mapping model corresponding to the preset viewing angle range is established, including: For any pixel group type, a corresponding first brightness model is established based on the display parameters corresponding to the pixel group type in the second calibration image of the preset viewing angle range among the plurality of display parameters; wherein, the first brightness model is used to describe the correlation between the actual brightness value and the actual grayscale value in the pixel group type for the preset viewing angle range. Based on the established first brightness models of the preset viewing angle range, the actual grayscale value of the reference pixel group type is used as the initial grayscale value, and the actual grayscale value of at least one non-reference pixel group type under the same brightness is used as the corresponding target grayscale value to establish a preset grayscale value mapping model corresponding to the preset viewing angle range.

13. The method as described in claim 10, characterized in that, Based on the multiple display parameters, a preset grayscale value mapping model is established, including: For any pixel group type, a corresponding second brightness model is established based on the display parameters corresponding to the pixel group type among the plurality of display parameters; wherein, the second brightness model is used to describe the correlation between the actual brightness value and the actual grayscale value in the pixel group type from a global perspective; Based on the established multiple second brightness models, the actual grayscale value of the reference pixel group type is used as the initial grayscale value, and the actual grayscale value of at least one non-reference pixel group type under the same brightness is used as the corresponding target grayscale value to establish the preset grayscale value mapping model.

14. The method as described in claim 12 or 13, characterized in that, Before determining the target grayscale value corresponding to the pixel group, the method further includes: The pixel group type of the pixel group is determined to be any non-reference pixel group type.

15. An image data processing apparatus, characterized in that, The device includes: The initial grayscale acquisition module is used to obtain the initial grayscale value corresponding to the pixel region where any pixel group is located in the target image data. The target grayscale acquisition module is used to determine the target grayscale value corresponding to the pixel group based on the target grayscale value mapping model, combined with the pixel group type of the pixel group and the initial grayscale value; wherein, the target grayscale value mapping model is used to describe the correlation between the initial grayscale value and the target grayscale value of at least one pixel group type; The grayscale compensation module is used to replace the initial grayscale value of the pixel region where the pixel group is located in the target image data with the target grayscale value, and to display the image on the display screen based on the target image data.

16. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 14.

Images