Large-size Demura sub-pixel positioning method and system based on multiple cameras

Abstract

The invention provides a large-size Demura sub-pixel positioning method, system and device based on multiple cameras, and relates to the technical field of screen calibration and optimizing.The method comprises the steps that firstly, an overlapping sub-region positioning map is designed according to screen parameters; then, images of all the sub-regions are collected; extracting features through an image processing technology, and establishing mapping between a screen and a camera; correcting features by using mapping and a two-dimensional Gaussian function, and establishing high-precision mapping; calculating coordinates of R and B channel cameras, and performing polynomial fitting and secondary correction on curved surface pixel points; and finally, combining sub-image coordinates, evaluating the imaging quality, and selecting an optimal coordinate as a final result to ensure high precision and stability. According to the method, the positioning precision and the processing efficiency are improved, the stability and the anti-interference capability of the mapping relation are enhanced, and powerful support is provided for the Demura process of the vehicle-mounted display screen.

Description

Technical Field

[0001] The present invention relates to the technical field of screen calibration and optimization, and in particular to a large-size Demura sub-pixel positioning method and system based on multiple cameras. Background Art

[0002] With the rapid development of display technology, display screens have been widely used in various electronic devices, including but not limited to mobile phone screens, TV screens, notebook screens, smart watches, and virtual reality (VR) devices. These screens play a vital role in improving user experience. However, different types of screens face different challenges in display quality adjustment and optimization due to differences in their physical properties (such as pixel density PPI, shape, cost, etc.).

[0003] Among the existing display screen calibration and optimization technologies, Demura (brightness uniformity correction) technology is an important means. It collects brightness data from various areas of the screen and adjusts the brightness of each pixel on the screen accordingly to achieve uniform distribution of screen brightness, thereby improving display quality. For small-sized and relatively regular-shaped screens such as mobile phone screens and notebook screens, the current mainstream Demura technology mostly uses a single high-resolution camera to complete tasks such as data collection. This method is widely used due to its high efficiency and economy.

[0004] However, when it comes to in-vehicle display screens, the situation becomes much more complicated. In-vehicle display screens often have a large aspect ratio, complex shapes (such as S-shape, V-shape, etc.) and large curvatures. These characteristics make traditional single-camera Demura technology face many challenges. Specifically, the working field of a single camera may not cover the entire screen, resulting in incomplete data collection; at the same time, due to the influence of the screen curvature, the depth of field of the camera may not be sufficient to accurately capture the brightness information of each area of ​​the screen, which in turn affects the Demura correction effect.

[0005] In summary, although the existing demura technology has achieved remarkable results in the field of small-sized and regular screens, no mature technical solutions have been proposed for the specific field of vehicle-mounted display screens. This is mainly due to the particularity of vehicle-mounted display screens, which makes it difficult to directly apply traditional methods. Therefore, a new positioning solution is urgently needed to solve the problems of insufficient field of view and insufficient depth of field faced by vehicle-mounted display screens during the demura process. Summary of the invention

[0006] To this end, an embodiment of the present invention provides a large-size Demura sub-pixel positioning method and system based on multiple cameras, which are used to solve the problems of insufficient field of view and insufficient depth of field faced by vehicle-mounted display screens during the Demura process in the prior art.

[0007] In order to solve the above problem, an embodiment of the present invention provides a large-size Demura sub-pixel positioning method based on multiple cameras, the method comprising:

[0008] According to the detailed parameters of the vehicle-mounted heterogeneous screen, a positioning map containing multiple overlapping sub-areas is designed;

[0009] Collecting a positioning map of each designed sub-area;

[0010] By using image processing technology, the first feature is calculated, the second feature is calculated and a screen-to-camera mapping function is established;

[0011] The third feature is modified by using a mapping function and a two-dimensional Gaussian function to establish a high-precision mapping relationship;

[0012] Calculate the camera coordinates of R and B channels according to the screen design drawings and high-precision mapping relationships;

[0013] Extract the coordinates of M pixels on the non-curved surface for polynomial fitting, calculate the corrected coordinates of the new pixels on the curved surface, and perform secondary correction until the coordinates of all pixels are corrected;

[0014] Get the pixel coordinates of each sub-image, merge all pixel coordinate points, evaluate the imaging quality of the overlapping parts, and select the coordinates with high spot concentration and small variance in the x and y directions as the final coordinates.

[0015] Preferably, the first feature is an easily identifiable circumscribed rectangular frame used for image segmentation and positioning; the second feature is an array-distributed geometric image used to calculate the spatial transformation relationship between the camera and the screen; the third feature is a G single pixel array used for sub-pixel positioning.

[0016] Preferably, the method for calculating the first feature is:

[0017] Use Canny edge detection and Hough line detection to complete rectangular border detection and generate a valid area Mask.

[0018] Preferably, the second feature and the method of establishing the screen-to-camera mapping function are:

[0019] Binarize the image;

[0020] Perform feature statistics of center of gravity, area, perimeter, and deflection angle for each connected domain in the binary graph;

[0021] Use the classifier to process all connected domains and obtain the coordinates of the second feature in the camera coordinate system, which is recorded as Camera_elliptical_1;

[0022] Perform Canny edge processing on the second feature sub-image, use the least squares method to fit the ellipse, and obtain the center coordinates of the ellipse, which is recorded as Camera_elliptical;

[0023] According to screen_elliptical in the screen coordinate system and Camera_elliptical in the camera coordinate system, polynomial fitting is used to obtain the screen-to-camera mapping function: Camera_elliptical = Fun(screen_elliptical).

[0024] Preferably, the method of using a mapping function and a two-dimensional Gaussian function to correct the third feature and establish a high-precision mapping relationship is:

[0025] According to the screen to camera mapping function, the screen_pixel_G coordinates are converted to the coordinates camera_pixel_G_1 in the camera coordinate system;

[0026] Use a two-dimensional Gaussian function to approximate the point spread function;

[0027] Take the logarithm of the two-dimensional Gaussian function, sort it out, and use the least squares method to solve the parameters to obtain the center coordinates of the light spot (x0, y0);

[0028] A BP neural network is used to establish a high-precision mapping relationship from screen_pixel_G to camera_pixel_G: camera_pixel_G=Fun2(screen_pixel_G).

[0029] Preferably, the method for calculating the camera coordinates of the R and B channels according to the screen design drawing and the high-precision mapping relationship is:

[0030] Calculate the screen coordinates of the R and B channels according to the screen design drawing: screen_all_pixel_R, screen_all_pixel_B;

[0031] Get the camera coordinates under the R and B channels through high-precision mapping relationships: Camera_all_pixel_R, Camera_all_pixel_B.

[0032] Preferably, the method of extracting the coordinates of M pixel points on the non-curved surface for polynomial fitting, calculating the corrected coordinates of new pixel points on the curved surface, and performing secondary correction until the coordinate correction of all pixel points is completed is:

[0033] Extract the coordinates of M pixel points on the non-curved surface as the basis for correcting the curved surface;

[0034] Perform N-order polynomial fitting on the coordinates of M pixel points, and use the least square method to solve the coefficients of the fitting polynomial to obtain the fitting polynomial;

[0035] The corrected coordinates of the next pixel are calculated using the fitted polynomial to obtain new corrected coordinates;

[0036] Perform secondary correction on the new corrected coordinates;

[0037] Add the new corrected coordinates to the pixel set, and delete the first pixel coordinate in the pixel set, keeping the set size M;

[0038] Repeat the above steps to continue correcting subsequent pixels one by one until all pixels are corrected.

[0039] Preferably, the method of evaluating the imaging quality of the overlapped portion and selecting coordinates with high spot concentration and small variance in the x and y directions as the final coordinates is:

[0040] The brightness of the sub-images in the original images of adjacent cameras is normalized respectively, and the two-dimensional Gaussian model is calculated. The corresponding coordinates with high spot concentration, small variance in the x and y directions and high values ​​of repeated areas are selected as the final coordinates.

[0041] The embodiment of the present invention further provides a large-size Demura sub-pixel positioning system based on multiple cameras, which is used to implement the large-size Demura sub-pixel positioning method based on multiple cameras described above, specifically comprising:

[0042] Multi-camera positioning map design module, used to design a positioning map containing multiple overlapping sub-areas according to the detailed parameters of the vehicle-mounted heterogeneous screen;

[0043] A positioning map acquisition module is used to acquire the positioning map of each designed sub-area;

[0044] A mapping function establishment module, used to calculate the first feature, calculate the second feature and establish a screen-to-camera mapping function through image processing technology;

[0045] A high-precision mapping relationship establishment module, used to modify the third feature using a mapping function and a two-dimensional Gaussian function to establish a high-precision mapping relationship;

[0046] The R and B channel camera coordinate calculation module is used to calculate the camera coordinates of the R and B channels according to the screen design drawings and high-precision mapping relationships;

[0047] The pixel point coordinate correction module is used to extract the coordinates of M pixel points on the non-curved surface for polynomial fitting, calculate the corrected coordinates of new pixel points on the curved surface, and perform secondary correction until the coordinate correction of all pixel points is completed;

[0048] The sub-positioning map coordinate merging module is used to obtain the pixel coordinates of each sub-image, merge all pixel coordinate points, evaluate the imaging quality of the overlapping parts, and select the coordinates with high spot concentration and small variance in the x and y directions as the final coordinates.

[0049] An embodiment of the present invention also provides an electronic device, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned large-size Demura sub-pixel positioning method based on multiple cameras.

[0050] It can be seen from the above technical solutions that the present invention has the following beneficial effects:

[0051] (1) By combining the G single pixel array, the least squares method to fit the ellipse equation and the precise mapping of the BP neural network, the present invention can significantly improve the accuracy of sub-pixel positioning and effectively reduce the positioning error. At the same time, the coordinates of the curved surface of the screen are accurately corrected to solve the problem of coordinate deviation caused by the curvature of the screen, thereby improving the accuracy of the overall image.

[0052] (2) The present invention designs multiple sub-region positioning maps including overlapping parts based on the shape, resolution and PPI of the vehicle-mounted heterogeneous screen, effectively handling the special viewing angle and shape of the vehicle-mounted screen. In addition, the BP neural network is used to optimize the mapping relationship, which improves the anti-interference ability of screen pixel defects and other interference factors, ensuring that a stable mapping effect can be provided under different environments and conditions.

[0053] (3) The present invention combines image processing algorithms and machine learning techniques to automate the entire process from positioning map design to final pixel coordinate merging, significantly reducing manual intervention and improving processing efficiency. At the same time, by normalizing the brightness of the original images of adjacent cameras and evaluating the two-dimensional Gaussian model, the selection criteria for coordinate merging are optimized, ensuring that the selected pixel coordinates have high imaging quality, thereby improving the accuracy and reliability of the final positioning map. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0055] Figure 1It is a flow chart of a large-size Demura sub-pixel positioning method based on multiple cameras provided in an embodiment;

[0056] Figure 2 It is the vehicle-mounted multi-camera positioning diagram designed in the embodiment;

[0057] Figure 3 This is a schematic cross-sectional view of the heterogeneous vehicle-mounted screen in the embodiment;

[0058] Figure 4 It is a positioning map taken by the dual cameras in the embodiment;

[0059] Figure 5 It is a partial schematic diagram of the curved surface to be corrected in the embodiment;

[0060] Figure 6 A block diagram of a large-size Demura sub-pixel positioning system based on multiple cameras provided in an embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Embodiment 1

[0063] In order to solve the problems of insufficient field of view and insufficient depth of field faced by the on-board display screen in the demura process in the prior art. Figure 1 As shown, an embodiment of the present invention proposes a large-size Demura sub-pixel positioning method based on multiple cameras, the method comprising:

[0064] S1: Design a positioning map containing multiple overlapping sub-areas based on the detailed parameters of the vehicle-mounted heterogeneous screen;

[0065] S2: Collect the localization map of each designed sub-region;

[0066] S3: Calculate the first feature, calculate the second feature and establish a screen-to-camera mapping function through image processing technology;

[0067] S4: Use the mapping function and the two-dimensional Gaussian function to correct the third feature and establish a high-precision mapping relationship;

[0068] S5: Calculate the camera coordinates of the R and B channels according to the screen design drawings and high-precision mapping relationship;

[0069] S6: extracting the coordinates of M pixel points on the non-curved surface for polynomial fitting, calculating the corrected coordinates of the new pixel points on the curved surface, and performing secondary correction until the coordinate correction of all pixel points is completed;

[0070] S7: Obtain the pixel coordinates of each sub-image, merge all pixel coordinate points, evaluate the imaging quality of the overlapping parts, and select the coordinates with high spot concentration and small variance in the x and y directions as the final coordinates.

[0071] It can be seen from the above technical solution that the present invention provides a large-size Demura sub-pixel positioning method based on multiple cameras. By designing a positioning map containing multiple overlapping sub-areas and using multiple cameras to collect data in different areas, full coverage of the vehicle screen is achieved. By gradually extracting the first, second, and third feature information, combined with two-dimensional Gaussian function fitting and BP neural network precise mapping, the present invention significantly improves the accuracy of sub-pixel positioning. At the same time, for the coordinate correction of the curved surface part of the screen, the present invention effectively handles the coordinate deviation caused by the curvature of the screen through polynomial fitting and quadratic correction. Finally, by evaluating the imaging quality of the original images of adjacent cameras and selecting the optimal coordinates for merging, the accuracy and reliability of the final positioning map are ensured. This method not only improves the positioning accuracy and processing efficiency, but also enhances the stability and anti-interference ability of the mapping relationship, and provides strong support for the Demura process of the vehicle display screen.

[0072] Combine the following Figures 2 to 5 , describe the technical solution of the present invention in detail.

[0073] In step S1, a positioning map including a plurality of overlapping sub-areas is designed according to detailed parameters of the vehicle-mounted heterogeneous screen.

[0074] Specifically, a multi-camera positioning map is designed based on the shape, resolution, and PPI of the vehicle-mounted heterogeneous screen. The positioning map is designed into multiple sub-areas according to the working field of view of each camera. Adjacent sub-positioning maps contain overlapping parts, so that the coordinates in the sub-positioning maps of different camera fields of view can be merged later. The features in the sub-positioning map include:

[0075] The first feature: Design an easily recognizable external rectangular frame for image segmentation and positioning.

[0076] Second feature: geometric images of array distribution (such as ellipse, rectangle, circle, for ease of description, the present invention will use ellipse as a description in the following), which is used to calculate the spatial transformation relationship between the camera and the screen. These ellipses are evenly distributed on the screen and are clearly visible from different camera viewing angles, denoted as screen_elliptical.

[0077] The third feature: G single pixel array, used for sub-pixel positioning, these points are sparsely distributed to avoid interference. The coordinates of the RGB single pixel points distributed in the array are recorded as screen_pixel_R, screen_pixel_G, screen_pixel_B.

[0078] At the same time, a unified screen coordinate system and camera coordinate system are defined to facilitate subsequent data processing and mapping. The screen RGB pixel coordinates are recorded as screen_all_pixel_R, screen_all_pixel_G, and screen_all_pixel_B.

[0079] In step S2, an industrial camera is used to collect a positioning map of each designed sub-area.

[0080] In step S3, the first feature is calculated, the second feature is calculated and a screen-to-camera mapping function is established by using image processing technology.

[0081] Specifically, the method for calculating the first feature is:

[0082] Use Canny edge detection and Hough line detection to complete rectangular border detection and generate a valid area Mask.

[0083] The method to calculate the second feature and establish the screen to camera mapping function is:

[0084] Binarize the image; perform feature statistics such as center of gravity, area, perimeter, and deflection angle on each connected domain in the binary image; use a classifier to process all connected domains and obtain the coordinates of the second feature in the camera coordinate system, recorded as Camera_elliptical_1.

[0085] Since there is a large error in the coordinates at this time, Canny edge processing is performed on the second feature subimage, and the least squares method is used to fit the ellipse to obtain the center coordinates of the ellipse, which is recorded as Camera_elliptical.

[0086] Specifically, the ellipse equation is defined as follows:

[0087] Ax 2 +Bxy+Cy 2 +Dx+Ey+1=0;

[0088]

[0089] The least squares method is used to fit the above formula to obtain parameters such as A, B, C, D, and E. After conversion, the center coordinate (x0, y0) can be obtained, which is recorded as the second characteristic coordinate Camera_elliptical.

[0090] According to screen_elliptical in the screen coordinate system and Camera_elliptical in the camera coordinate system, polynomial fitting is used to obtain the screen-to-camera mapping function: Camera_elliptical = Fun(screen_elliptical).

[0091] In step S4, the third feature is corrected using a mapping function and a two-dimensional Gaussian function to establish a high-precision mapping relationship, specifically including:

[0092] S41: According to the screen-to-camera mapping function (Fun function), the screen_pixel_G coordinates are converted to the coordinates camera_pixel_G_1 in the camera coordinate system:

[0093] camera_pixel_G_1=Fun(screen_pixel_G).

[0094] S42: Since camera_pixel_G is calculated by the Fun function, there are some errors. Therefore, the sub-image corresponding to each feature point in camera_pixel_G is re-positioned to achieve sub-pixel positioning. The present invention uses a two-dimensional Gaussian function to approximate the point spread function, and its formula is:

[0095]

[0096] Where A represents the peak brightness or amplitude of the light spot; σx and σy represent the standard deviation of the light spot in the x and y directions respectively; exp represents the natural exponential function; (x, y) represents the sub-image coordinates corresponding to each feature point in camera_pixel_G; (x0, y0) represents the center coordinates of the light spot.

[0097] S43: Take the logarithm of the two-dimensional Gaussian function, sort it out, and use the least square method to solve the parameters to obtain the center coordinates of the light spot (x0, y0).

[0098] Specifically, taking the logarithm of the above two-dimensional Gaussian function, we get: ln f = a*x 2 +b*y 2 +c*x+dy+e;

[0099] in

[0100] In the formula, a, b, c, d, and e all represent coefficients;

[0101] Using the extracted light spot sub-image data f(x, y) and the corresponding coordinates (x, y), the polynomial ln f = a*x is solved by the least squares method. 2+b*y 2 +c*x+dy+e coefficients a, b, c, d, e;

[0102] According to the coefficients a, b, c, d, and e obtained by solving, the original relationship is brought back to solve the amplitude A of the Gaussian function. The amplitude A is the brightness data of the current sub-pixel coordinate P (x0, y0).

[0103] S44: After performing the above coordinate secondary correction on all third features to improve their accuracy, the third feature that completes sub-pixel positioning is recorded as camera_pixel_G in the camera coordinate system. The present invention uses BP neural network to establish a high-precision mapping relationship from screen_pixel_G to camera_pixel_G: camera_pixel_G=Fun2(screen_pixel_G).

[0104] Specifically, for the point-to-point mapping relationship, traditional methods such as perspective transformation and polynomial fitting are often insufficient in anti-interference ability and accuracy when processing screens with problems such as bad pixels (i.e., feature points may be missing or redundant in the screen_pixel_G set). To this end, the present invention proposes to use a BP (Back Propagation) neural network to achieve this mapping relationship. The specific scheme is as follows:

[0105] a. Data preparation stage:

[0106] Collect a sufficient number of screen_pixel_G and its corresponding camera_pixel_G as training data sets. Preprocess the collected data, including noise removal and normalization, to improve training efficiency and model performance.

[0107] b. Network design phase:

[0108] Design a reasonable BP neural network architecture, which should include an input layer, one or more hidden layers, and an output layer. Select appropriate activation functions (such as Sigmoid, ReLU, etc.) and loss functions (such as mean square error MSE) to optimize the network's learning process and output accuracy.

[0109] c. Network training phase:

[0110] The BP neural network is trained using the preprocessed training data set. The network parameters are continuously adjusted through the back propagation algorithm to achieve a better mapping relationship. During the training process, methods such as cross-validation can be used to evaluate the generalization ability of the model, and the network structure and parameters can be adjusted accordingly.

[0111] d. Model evaluation and optimization stage:

[0112] Use an independent test data set to evaluate the trained model to verify its mapping accuracy and anti-interference ability. According to the evaluation results, make necessary optimizations to the model, such as adjusting the network structure and increasing training data, to improve model performance.

[0113] e. Application and deployment phase:

[0114] Deploy the trained and optimized BP neural network model to actual application scenarios.

[0115] Realize the accurate mapping of screen_pixel_G to camera_pixel_G, that is, Camera_all_pixel_G=Fun2(screen_all_pixel_G).

[0116] Compared with the Fun function, the Fun2 function can establish a more accurate mapping relationship because it introduces more diverse data sources and higher precision requirements. By inputting all the data of the screen coordinate system under the G channel, Fun2 can accurately realize the positioning of the screen coordinate system under the G channel. The positioning of the screen coordinate system under the G channel is as follows:

[0117] Camera_all_pixel_G=Fun2(screen_all_pixel_G).

[0118] In step S5, the camera coordinates of the R and B channels are calculated according to the screen design drawings and the high-precision mapping relationship, specifically including:

[0119] Calculate the screen coordinates of the R and B channels according to the screen design drawing: screen_all_pixel_R, screen_all_pixel_B;

[0120] Get the camera coordinates under the R and B channels through high-precision mapping relationships: Camera_all_pixel_R, Camera_all_pixel_B.

[0121] Specifically, in the screen coordinate system, since the relative relationship of RGB sub-pixels has been determined in the screen design, the relative relationship can be obtained from the screen design drawings and other materials, and remember how the relative relationship is:

[0122] screen_all_pixel_R=Fun3(screen_all_pixel_G);

[0123] screen_all_pixel_B=Fun4(screen_all_pixel_G);

[0124] Substitute it into the fun2 function and we get:

[0125] Camera_all_pixel_R=Fun2(fun3(screen_all_pixel_G));

[0126] Camera_all_pixel_B=Fun2(fun4(screen_all_pixel_G)).

[0127] In step S6, the coordinates of M pixels on the non-curved surface are extracted for polynomial fitting, the corrected coordinates of new pixels on the curved surface are calculated, and secondary correction is performed until the coordinate correction of all pixels is completed.

[0128] S61: Extract the coordinates of M pixels on the non-curved surface These coordinates are used for polynomial fitting as a basis for the correction of the surface part.

[0129] S62: Perform N-order polynomial fitting on the coordinates of the M pixel points, and use the least square method to solve the coefficients of the fitting polynomial to obtain the fitting polynomial.

[0130] Specifically, an N-order polynomial fitting is performed on the coordinates of the M pixel points, and the fitting curve equation is:

[0131]

[0132] The coefficients of the fitting polynomial {a0, a1, a2, ..., an} are solved by the least squares method. The calculation formula of the fitting polynomial is as follows:

[0133]

[0134] By solving this optimization problem, the optimal polynomial coefficients are obtained.

[0135] S63: Calculate the corrected coordinates (x) of the next pixel using the fitted polynomial. M+1 ,y M+1 ), and obtain the new corrected coordinates, where the correction formula is:

[0136]

[0137] S64: Perform secondary correction on the new corrected coordinates.

[0138] S65: Add the new corrected coordinates to the pixel point set, and delete the first pixel point coordinates in the pixel point set, keeping the set size M. The updated pixel point coordinate set is:

[0139] S66: Repeat the above steps to continue correcting subsequent pixels one by one until all pixels are corrected.

[0140] In step S7, the pixel coordinates of each sub-image are obtained, all pixel coordinate points are merged, the imaging quality of the overlapping parts is evaluated, and the coordinates with high spot concentration and small variance in the x and y directions are selected as the final coordinates. The present invention normalizes the brightness of the sub-images in the original images of adjacent cameras, calculates the two-dimensional Gaussian model, and selects the corresponding coordinates with high spot concentration, small variance in the x and y directions and high repeated area values ​​as the final coordinates.

[0141] Specifically, for the overlapping part, the image quality of the current position in the original image of the adjacent camera is evaluated for selection, including:

[0142] Normalize the brightness of the sub-images in the original images of adjacent cameras respectively (divided by exposure and gain);

[0143] Calculate the two-dimensional Gaussian model in the above model respectively, and calculate A, σx, and σy respectively;

[0144] Calculate the evaluation value evaluationVal = A((σx*σy) respectively, remember to optimize the selection of the spot with high concentration and small variance in the x and y directions;

[0145] The corresponding coordinates of the repeated area with higher evaluationVal values ​​are the coordinates of the current pixel.

[0146] Embodiment 2

[0147] like Figure 6 As shown, the present invention provides a large-size Demura sub-pixel positioning system based on multiple cameras, which is used to implement the large-size Demura sub-pixel positioning method based on multiple cameras in the above-mentioned embodiment 1, specifically comprising:

[0148] Multi-camera positioning map design module, used to design a positioning map containing multiple overlapping sub-areas according to the detailed parameters of the vehicle-mounted heterogeneous screen;

[0149] A positioning map acquisition module is used to acquire the positioning map of each designed sub-area;

[0150] A mapping function establishment module, used to calculate the first feature, calculate the second feature and establish a screen-to-camera mapping function through image processing technology;

[0151] A high-precision mapping relationship establishment module, used to modify the third feature using a mapping function and a two-dimensional Gaussian function to establish a high-precision mapping relationship;

[0152] The R and B channel camera coordinate calculation module is used to calculate the camera coordinates of the R and B channels according to the screen design drawings and high-precision mapping relationships;

[0153] The pixel point coordinate correction module is used to extract the coordinates of M pixel points on the non-curved surface for polynomial fitting, calculate the corrected coordinates of new pixel points on the curved surface, and perform secondary correction until the coordinate correction of all pixel points is completed;

[0154] The sub-positioning map coordinate merging module is used to obtain the pixel coordinates of each sub-image, merge all pixel coordinate points, evaluate the imaging quality of the overlapping parts, and select the coordinates with high spot concentration and small variance in the x and y directions as the final coordinates.

[0155] A large-size Demura sub-pixel positioning system based on multiple cameras in this embodiment is used to implement the aforementioned large-size Demura sub-pixel positioning method based on multiple cameras. Therefore, the specific implementation method of the large-size Demura sub-pixel positioning system based on multiple cameras can be seen in the embodiment part of the large-size Demura sub-pixel positioning method based on multiple cameras in the previous text. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part. In order to avoid redundancy, it will not be repeated here.

[0156] Embodiment 3

[0157] An embodiment of the present invention provides an electronic device, which includes a processor, a memory and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned large-size Demura sub-pixel positioning method based on multiple cameras.

[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0159] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0161] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A large-size Demura sub-pixel positioning method based on multiple cameras, characterized in that: include: According to the detailed parameters of the vehicle-mounted heterogeneous screen, a positioning map containing multiple overlapping sub-areas is designed; Collecting a positioning map of each designed sub-area; By using image processing technology, the first feature is calculated, the second feature is calculated and a screen-to-camera mapping function is established; The third feature is modified by using a mapping function and a two-dimensional Gaussian function to establish a high-precision mapping relationship; Calculate the camera coordinates of R and B channels according to the screen design drawings and high-precision mapping relationships; Extract the coordinates of M pixels on the non-curved surface for polynomial fitting, calculate the corrected coordinates of the new pixels on the curved surface, and perform secondary correction until the coordinates of all pixels are corrected; Get the pixel coordinates of each sub-image, merge all pixel coordinate points, evaluate the imaging quality of the overlapping parts, and select the coordinates with high spot concentration and small variance in the x and y directions as the final coordinates.

2. The large-size Demura sub-pixel positioning method based on multiple cameras according to claim 1 is characterized in that: The first feature is an easily recognizable external rectangular frame, which is used for image segmentation and positioning; the second feature is an array-distributed geometric image, which is used to calculate the spatial transformation relationship between the camera and the screen; the third feature is the G single pixel array, which is used for sub-pixel positioning.

3. The large-size Demura sub-pixel positioning method based on multiple cameras according to claim 1 is characterized in that: The method for calculating the first feature is: Use Canny edge detection and Hough line detection to complete rectangular border detection and generate a valid area Mask.

4. The large-size Demura sub-pixel positioning method based on multiple cameras according to claim 1 is characterized in that: The second feature and the method of establishing the screen to camera mapping function are: Binarize the image; Perform feature statistics of center of gravity, area, perimeter, and deflection angle for each connected domain in the binary graph; Use the classifier to process all connected domains and obtain the coordinates of the second feature in the camera coordinate system, which is recorded as Camera_elliptical_1; Perform Canny edge processing on the second feature sub-image, use the least squares method to fit the ellipse, and obtain the center coordinates of the ellipse, which is recorded as Camera_elliptical; According to screen_elliptical in the screen coordinate system and Camera_elliptical in the camera coordinate system, polynomial fitting is used to obtain the screen-to-camera mapping function: Camera_elliptical = Fun(screen_elliptical).

5. The large-size Demura sub-pixel positioning method based on multiple cameras according to claim 1 is characterized in that: The method of using the mapping function and the two-dimensional Gaussian function to correct the third feature and establish a high-precision mapping relationship is: According to the screen to camera mapping function, the screen_pixel_G coordinates are converted to the coordinates camera_pixel_G_1 in the camera coordinate system; Use a two-dimensional Gaussian function to approximate the point spread function; Take the logarithm of the two-dimensional Gaussian function, sort it out, and use the least squares method to solve the parameters to obtain the center coordinates of the light spot (x0, y0); A BP neural network is used to establish a high-precision mapping relationship from screen_pixel_G to camera_pixel_G: camera_pixel_G=Fun2(screen_pixel_G).

6. The large-size Demura sub-pixel positioning method based on multiple cameras according to claim 1 is characterized in that: The method for calculating the camera coordinates of the R and B channels according to the screen design drawings and the high-precision mapping relationship is: Calculate the screen coordinates of the R and B channels according to the screen design drawing: screen_all_pixel_R, screen_all_pixel_B; Get the camera coordinates under the R and B channels through high-precision mapping relationships: Camera_all_pixel_R, Camera_all_pixel_B.

7. The large-size Demura sub-pixel positioning method based on multiple cameras according to claim 1 is characterized in that: The method of extracting the coordinates of M pixels on the non-curved surface for polynomial fitting, calculating the corrected coordinates of new pixels on the curved surface, and performing secondary correction until the coordinate correction of all pixels is completed is: Extract the coordinates of M pixels on the non-curved surface as the basis for correcting the curved surface; Perform N-order polynomial fitting on the coordinates of M pixel points, and use the least square method to solve the coefficients of the fitting polynomial to obtain the fitting polynomial; The corrected coordinates of the next pixel are calculated using the fitted polynomial to obtain new corrected coordinates; Perform secondary correction on the new corrected coordinates; Add the new corrected coordinates to the pixel set, and delete the first pixel coordinate in the pixel set, keeping the set size M; Repeat the above steps to continue correcting subsequent pixels one by one until all pixels are corrected.

8. The large-size Demura sub-pixel positioning method based on multiple cameras according to claim 1 is characterized in that: The method for evaluating the imaging quality of the overlapped portion and selecting coordinates with high spot concentration and small variance in the x and y directions as the final coordinates is: The brightness of the sub-images in the original images of adjacent cameras is normalized respectively, and the two-dimensional Gaussian model is calculated. The corresponding coordinates with high spot concentration, small variance in the x and y directions and high values ​​of repeated areas are selected as the final coordinates.

9. A large-scale Demura sub-pixel positioning system based on multiple cameras, characterized in that: The system is used to implement the large-size Demura sub-pixel positioning method based on multiple cameras according to any one of claims 1 to 8, specifically comprising: Multi-camera positioning map design module, used to design a positioning map containing multiple overlapping sub-areas according to the detailed parameters of the vehicle-mounted heterogeneous screen; A positioning map acquisition module is used to acquire the positioning map of each designed sub-area; A mapping function establishment module, used to calculate the first feature, calculate the second feature and establish a screen-to-camera mapping function through image processing technology; A high-precision mapping relationship establishment module, used to modify the third feature using a mapping function and a two-dimensional Gaussian function to establish a high-precision mapping relationship; The R and B channel camera coordinate calculation module is used to calculate the camera coordinates of the R and B channels according to the screen design drawings and high-precision mapping relationships; The pixel point coordinate correction module is used to extract the coordinates of M pixel points on the non-curved surface for polynomial fitting, calculate the corrected coordinates of new pixel points on the curved surface, and perform secondary correction until the coordinate correction of all pixel points is completed; The sub-positioning map coordinate merging module is used to obtain the pixel coordinates of each sub-image, merge all pixel coordinate points, evaluate the imaging quality of the overlapping parts, and select the coordinates with high spot concentration and small variance in the x and y directions as the final coordinates.

10. An electronic device, characterized in that: The electronic device includes a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the large-size Demura sub-pixel positioning method based on multiple cameras as described in any one of claims 1 to 8.

Images