An image calibration method, device, equipment and storage medium

Abstract

The application discloses an image calibration method, device and equipment and a storage medium. The method comprises the following steps: acquiring a boundary fitting straight line of each image region to be processed; determining a stripe angle based on the slope and length of each boundary fitting straight line; and determining a stripe direction based on the change direction of the grayscale value on both sides of each boundary fitting straight line. The boundary fitting straight line of each image region to be processed is acquired, so that the image noise and discrete points can be eliminated to disturb the boundary, and a regular and accurate boundary reference is obtained. The stripe angle is determined based on the slope and length of each boundary fitting straight line, so that the stripe feature can be accurately calculated based on the direction and effectiveness of the straight line, and the reliability of the angle calculation is ensured. The stripe direction is determined based on the change direction of the grayscale value on both sides of each boundary fitting straight line, so that the target direction can be accurately locked based on the grayscale change rule, accurate direction parameters are provided for image calibration, and the image calibration accuracy and reliability are improved.

Description

Technical Field

[0001] This invention relates to the field of image calibration, and more particularly to an image calibration method, apparatus, device, and storage medium. Background Technology

[0002] Head-up display (HUD) technology uses an optomechanical system to emit display light, which is then projected onto the vehicle's windshield through optical lenses to form a virtual image, enhancing the display of the real scene and the virtual image. However, surface errors and assembly deviations in the optical lenses and windshield can easily lead to distortion of the virtual image projected onto the windshield; simultaneously, dynamic changes in the user's eye position require the display method to have real-time adjustment capabilities. Therefore, accurate and efficient image calibration is crucial for the application of 3D HUD technology.

[0003] Currently, HUD image calibration mostly adopts the control point matching method, which involves setting control points with different display forms in a standard dot matrix diagram, projecting them onto the windshield to obtain the projected display image, and then determining the distortion rate of the optical system by identifying the position displacement of the control points in the projected image to complete the calibration.

[0004] Traditional calibration methods struggle to quickly and accurately establish the correlation between corresponding control points before and after projection during control point matching, resulting in low calibration efficiency and failing to meet the real-time calibration requirements of 3D HUDs. Furthermore, these traditional calibration methods have limited effectiveness in correcting optical system deviations, making it difficult to accurately reproduce the original image features and failing to adequately adapt to the dynamic changes in the user's eye position. Summary of the Invention

[0005] This invention provides an image calibration method, apparatus, device, and storage medium, which solves the technical problems that existing calibration methods cannot meet the rapid calibration requirements of 3D HUDs and are difficult to correct image distortion caused by optical system deviations and adapt to real-time changes in the user's eye position.

[0006] According to one aspect of the present invention, an image calibration method is provided, the method comprising: Obtain the boundary fitting straight lines for each region of the image to be processed; The stripe angle is determined based on the slope and length of the fitted straight line at each boundary. The direction of the stripes is determined based on the direction of change in gray values ​​on both sides of the fitted straight line at each boundary.

[0007] Optionally, obtaining each image region to be processed includes: determining the region of interest based on the display area corresponding to the preset display content in each acquired original image; and for each original image, determining the image region to be processed from the original image based on the region of interest.

[0008] The advantage of this setting is that by locking the region of interest corresponding to the preset display content from the original image, irrelevant background interference can be eliminated, reducing the amount of computation and improving computational efficiency.

[0009] Optionally, obtaining the boundary fitting line for each image region to be processed includes: extracting the boundary of each image region to be processed based on the gray value of each pixel in each image region to obtain the coordinates of each boundary pixel; and performing line fitting based on the coordinates of each boundary pixel to determine the boundary fitting line.

[0010] The advantage of this setup is that by extracting boundary pixels from grayscale values ​​and performing line fitting, the boundary pixel coordinates of each image region to be processed can be determined quickly and accurately. The subsequent boundary fitting lines determined based on the boundary pixel coordinates are more conducive to improving calibration accuracy and processing efficiency.

[0011] Optionally, based on the grayscale values ​​of each pixel in each image region to be processed, boundary extraction is performed on each image region to be processed to obtain the coordinates of each boundary pixel, including: for each image region to be processed, determining the gradient of the grayscale values ​​of each pixel in the image region to be processed; based on each gradient, boundary extraction is performed on the image region to be processed to obtain the coordinates of each boundary pixel.

[0012] The advantage of this setup is that it uses grayscale gradient analysis to accurately identify boundaries by utilizing the abrupt changes in pixel grayscale, resulting in strong anti-interference capabilities and high-precision extraction of boundary positions.

[0013] Optionally, based on each gradient, boundary extraction is performed on the image region to be processed to obtain the coordinates of each boundary pixel, including: determining a gradient threshold; obtaining each first pixel in the image region to be processed whose gradient is greater than or equal to the gradient threshold, and each second pixel in the image region to be processed whose gradient is less than the gradient threshold; and based on each first pixel and each second pixel, boundary extraction is performed on the image region to be processed to obtain the coordinates of each boundary pixel.

[0014] The advantage of this setting is that by setting a gradient threshold and distinguishing between the first pixel and the second pixel based on the relationship between the pixel gradient and the gradient threshold, the coordinates of each boundary pixel can be quickly filtered out based on the first and second pixel, which significantly reduces the interference of image noise, non-edge textures and uneven lighting on boundary extraction and improves the accuracy and robustness of boundary pixel coordinates.

[0015] Optionally, based on the grayscale values ​​of each pixel in each image region to be processed, boundary extraction is performed on each image region to be processed to obtain the coordinates of each boundary pixel. This includes: sorting the grayscale values ​​of the same pixel in each image region to be processed; determining the index value of the pixel in each index distribution map based on the sorting result; and performing boundary extraction on each index distribution map based on the index values ​​of each pixel in the index distribution map to obtain the coordinates of each boundary pixel.

[0016] The advantages of this setup are as follows: by sorting the grayscale values ​​of the same pixel in multiple image regions to be processed, and generating index values ​​based on the sorting results, the interference of uneven illumination, grayscale drift, and optical noise on boundary extraction can be effectively eliminated, significantly improving the stability and consistency of boundary pixel coordinates. At the same time, converting grayscale differences into index value differences can effectively weaken the boundary recognition error caused by grayscale fluctuations in an image, making the region boundaries clearer and more continuous, and improving the robustness and accuracy of boundary extraction.

[0017] Optionally, the same pixel in each image region to be processed is determined by either of the following methods: determining pixels with the same pixel coordinates in each image region to be processed as the same pixel; or, aligning the pixel coordinates of the pixels in each image region to be processed, and determining pixels with the same pixel coordinates in each image region to be processed as the same pixel.

[0018] The advantage of this setting is that by identifying pixels with the same coordinates in each image region to be processed, or pixels with the same coordinates after coordinate alignment, as the same pixel, a unified and corresponding pixel benchmark can be established among multiple image regions to be processed. This ensures the consistency and accuracy of subsequent grayscale sorting and index value calculation, improves the robustness and universality of pixel matching among multiple image regions to be processed, avoids grayscale sorting errors caused by coordinate misalignment, and further improves the stability and accuracy of boundary extraction.

[0019] Optionally, for each index distribution map, based on the index value of each pixel in the index distribution map, boundary extraction is performed to obtain the coordinates of each boundary pixel, including: obtaining an index threshold for each index distribution map; determining each third pixel in the index distribution map whose index value is greater than or equal to the index threshold, and each fourth pixel in the index distribution map whose index value is less than the index threshold; and generating the coordinates of each boundary pixel in the index distribution map based on each third pixel and each fourth pixel in the index distribution map.

[0020] The advantage of this setup is that by setting index thresholds for each index distribution map and dividing pixels into third and fourth pixels based on the relationship between their index values ​​and the index thresholds, the coordinates of each boundary pixel can be quickly filtered out based on these third and fourth pixels. This significantly reduces the interference of image noise, non-edge textures, and uneven lighting on boundary extraction, thus improving the accuracy and robustness of boundary pixel coordinates.

[0021] Optionally, obtaining the index threshold includes: determining the index threshold based on each index value contained in the index distribution map.

[0022] The advantage of this setting is that by determining the index threshold based on the index value distribution of the index distribution map itself, the index threshold can be matched with the actual index distribution of the image region to be processed. It can automatically adapt to the differences in index distribution under different scenes, different degrees of distortion, and different lighting conditions, avoid the problems of poor universality and weak adaptability of fixed thresholds, improve the accuracy, robustness and adaptability of boundary extraction, and ensure that the boundary pixel coordinates are stable and reliable.

[0023] Optionally, based on the grayscale values ​​of each pixel in each image region to be processed, the boundary of each image region to be processed is extracted, and after obtaining the coordinates of each boundary pixel, a straight line is fitted based on the coordinates of each boundary pixel. Before determining the fitted straight line of the boundary, the method further includes: filtering the coordinates of each boundary pixel based on preset feature parameters; wherein, the preset feature parameters include, but are not limited to, one or more of the following: position features, orientation features, and shape features.

[0024] The advantage of this setting is that it filters the obtained boundary pixel coordinates based on preset feature parameters to filter out abnormal boundary points, eliminate isolated noise points, ensure that the pixel set of the input line fitting is a valid boundary, and prevent abnormal points from causing the fitted line to deviate.

[0025] Optionally, determining the fringe angle based on the slope and length of each boundary fitted line includes: filtering each boundary fitted line based on the slope and length of the boundary fitted line; and determining the fringe angle based on the slope of each filtered boundary fitted line.

[0026] The advantage of this setting is that it filters the fitted straight lines based on slope and length, eliminates short / deviation lines affected by noise, retains valid lines with high confidence, and ensures the accuracy and reliability of the stripe angle calculation results.

[0027] According to another aspect of the present invention, an image calibration apparatus is provided, the apparatus comprising: The boundary fitting line generation module is used to obtain the boundary fitting lines for each image region to be processed. The stripe angle determination module is used to determine the stripe angle based on the slope and length of the fitted straight line at each boundary. The stripe direction determination module is used to determine the stripe direction based on the direction of gray value change on both sides of the fitted straight line at each boundary.

[0028] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform an image calibration method according to any embodiment of the present invention.

[0029] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement an image calibration method according to any embodiment of the present invention.

[0030] The technical solution of this invention, by obtaining the boundary fitting lines of each image region to be processed, can eliminate the interference of image noise and discrete points on the boundaries, and obtain a regular and accurate boundary reference. By determining the stripe angle based on the slope and length of each boundary fitting line, the overall direction and size characteristics of the stripes can be accurately characterized in a quantitative form, ensuring the reliability and accuracy of the angle calculation. By determining the stripe direction based on the direction of gray value change on both sides of each boundary fitting line, the stripe direction can be accurately locked through the gray value change law, avoiding the influence of image distortion, uneven illumination, or local texture interference, providing accurate directional parameters for image calibration, and improving the overall accuracy and reliability of image calibration.

[0031] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart of an image calibration method provided according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of another image calibration method provided according to Embodiment 2 of the present invention; Figure 3 This is a flowchart of another image calibration method provided according to Embodiment 2 of the present invention; Figure 4 This is a flowchart of another image calibration method provided according to Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of an index distribution map sorting process provided according to Embodiment 2 of the present invention; Figure 6 This is a schematic diagram of the structure of an image calibration device according to Embodiment 3 of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device that implements an image calibration method according to an embodiment of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1 Figure 1 This is a flowchart illustrating an image calibration method provided in Embodiment 1 of the present invention. This embodiment is applicable to image calibration scenarios. The method can be executed by an image calibration device, which can be implemented in hardware and / or software. The image calibration device can be configured in a smart device, such as a vehicle controller, a personal computer, or a cloud server, such as a business server or application server. Figure 1 As shown, the method includes: S110. Obtain the boundary fitting straight line for each image region to be processed.

[0037] 3D image calibration requires determining the pose, angle, and positional relationship of the projected virtual image in space. However, during the calibration process, interference from elements such as windshield black edges, interior trim, and stray light, as well as installation deviations between the camera and the HUD, makes accurate calibration difficult to achieve directly based on the acquired images. Therefore, in this application, before 3D image calibration, each image region to be processed can be acquired. Subsequently, boundary extraction and line fitting are performed on each image region to obtain a boundary fitting line. The image region to be processed can be an image containing the projected virtual image (denoted as the original image). For example, the projected virtual image is a virtual image projected onto the windshield from a Head-Up Display (HUD) device. Alternatively, it can be the image region selected from the original image after removing invalid areas such as windshield black edges, interior trim, and environmental interference, which is then used for image feature extraction and calibration analysis. This reduces interference and computational load. The boundary fitting line refers to the straight line obtained by the line fitting algorithm for the set of edge points of the feature structure within the image region to be processed, which best fits the boundary of the feature structure. By fitting a straight line to this boundary, the overall orientation of the image region to be processed can be characterized, providing a stable directional reference for subsequent features such as stripes, corners, and distortions, and avoiding confusion in the calibration benchmark due to image rotation or shooting tilt.

[0038] Optionally, obtaining each image region to be processed includes: determining the region of interest based on the display area corresponding to the preset display content in each acquired original image; and for each original image, determining the image region to be processed from the original image based on the region of interest.

[0039] The preset display content can be specific display content that is pre-set and output in the 3D HUD device for analyzing feature structures.

[0040] Because the camera's shooting range and the actual display range of the projected virtual image do not perfectly match, and the original image inevitably contains a large number of irrelevant interference areas (such as black borders on the windshield, vehicle interior, ambient light, background objects, etc.); at the same time, due to factors such as the precision of the camera's optical lenses, windshield surface deviation, and HUD and camera assembly deviation, the original image will also have a certain degree of distortion. If the entire original image is processed directly, on the one hand, interference from irrelevant background areas will lead to misjudgments in subsequent feature extraction, reducing calibration accuracy; on the other hand, it will increase the amount of data processing, prolong calibration time, and affect calibration efficiency. Therefore, it is necessary to locate and determine the display area corresponding to the preset display content in the original image, and based on this display area, determine the region of interest. By determining the region of interest, the required effective image range can be delineated from a large area of ​​the original image, avoiding interference from irrelevant areas in subsequent operations. For example, if the acquired original image is a single image, the display area corresponding to the preset display content is delineated from the original image; this area is the region of interest for that single original image. Another example is that if multiple original images are acquired, the display areas corresponding to the preset display content in each original image are obtained. From each display area, the region of interest is determined. For example, the largest display area is determined as the region of interest, or the display area with the clearest display effect is determined as the region of interest, etc.

[0041] After determining the region of interest (ROI) based on the above embodiments, each original image can be cropped using the ROI as a reference to extract the region to be processed. For example, the region to be processed can be determined by cropping based on the pixel coordinates of the ROI. This method allows subsequent operations to focus on the effective image range, reducing data processing volume and improving the efficiency and accuracy of image calibration. It also ensures that the region to be processed in each original image corresponds to a consistent preset display content range, providing a unified image basis for subsequent feature analysis and data comparison of multiple images.

[0042] S120. Determine the stripe angle based on the slope and length of the fitted straight line at each boundary.

[0043] Stripes refer to linear image structures in a HUD projection image that have specific grayscale variations and clear boundary features. Their angles directly reflect the optical characteristics of the projected image and are applied in the image fusion stage of subsequent 3D display processes. This provides a precise angular reference for image fusion from different viewpoints and projection positions, solving the geometric matching problem of 3D HUD image fusion. Therefore, in this application, after obtaining the fitted lines of each boundary based on the above embodiments, the stripe angle can be roughly determined based on the slope of each fitted line. Here, the slope refers to the inclination rate of the fitted line, representing the direction of the fitted line. For example, determining the stripe angle based on the slope can be divided into two cases: a single image to be processed and multiple images to be processed. When processing a single image to be processed, the slopes of all fitted lines of the boundary in the image can be directly extracted. Based on these slopes, a representative slope can be determined. For example, the slope with the highest frequency can be used as the representative slope, or the average value of these slopes can be used as the representative slope, and then the stripe angle can be determined from the representative slope. When there are multiple images of the region to be processed, the slope of the boundary fitted line in each image can be extracted separately. Based on these slopes, the overall representative slope can be determined. For example, statistical clustering can be performed on all slopes, and the slope corresponding to the cluster center can be taken as the overall representative slope. Alternatively, the average value of all slopes can be taken as the overall representative slope, and then the stripe angle can be determined from the overall representative slope.

[0044] By determining the fringe angle, calibration accuracy can be significantly improved, errors caused by distortion and interference data can be reduced, and accurate parameters can be provided for subsequent image distortion correction and virtual image position calibration. At the same time, the fringe angle can be made to better match the real optical characteristics, ensuring accurate image fusion and alignment, stable human eye tracking, and improving the display effect and viewing experience of 3D HUD.

[0045] In one possible implementation, different boundary fitting lines have different lengths. The length of the boundary fitting line can characterize the integrity and detection confidence of the corresponding stripe boundary. The longer the length, the more significant the continuous features of the corresponding stripe boundary and the more reliable the detection result. Conversely, it indicates that the boundary may have local breaks, noise interference, or false boundary features of non-target stripes. In the process of determining the stripe angle based on the slope, a weighting coefficient can be set in conjunction with the length of the boundary fitting line. Longer boundary fitting lines are given higher weights, while shorter boundary fitting lines are given lower weights or directly eliminated to avoid short boundary false features interfering with the calculation accuracy of the stripe angle. For example, weights can be assigned according to the length of the boundary fitting lines. The slope of each line is multiplied by its weight, the sum is taken, and the average is obtained to obtain the weighted average slope. The final stripe angle is calculated using this weighted average slope, improving the reliability of the angle.

[0046] Optionally, determining the fringe angle based on the slope and length of each boundary fitted line includes: filtering each boundary fitted line based on the slope and length of the boundary fitted line; and determining the fringe angle based on the slope of each filtered boundary fitted line.

[0047] It is known that although the image region to be processed can fit the region boundary features, due to factors such as distortion of the original image, pixel interference during image preprocessing, and minor errors in the fitting algorithm, not all the fitted lines are valid lines. Some lines may have slopes that deviate from the reasonable range or be too short. These invalid lines cannot accurately reflect the optical characteristics of the original lines. If they are directly used for subsequent calculation of the fringe angle, the accuracy of the fringe angle determination will decrease, thus affecting the image fusion effect in the subsequent head-up display. Therefore, it is necessary to first filter and select all boundary fitted lines to identify valid boundary fitted lines, and then determine the fringe angle based on the valid boundary fitted lines to reduce the impact of errors on the fringe angle.

[0048] For example, this embodiment also provides three methods for filtering all boundary fitted lines, namely: Method 1: Filtering based on length.

[0049] In this embodiment, by discarding incorrectly fitted lines that are too short, and retaining boundary fitted lines of effective length for subsequent fringe angle calculations, the accuracy of the determined fringe angle can be improved, thereby increasing the accuracy of subsequent calibration based on that fringe angle. For example, the setting of the preset minimum length threshold needs to balance preserving feature details with eliminating interfering data. When the calibration requirement leans towards preserving as much detail as possible of the fringe features, the minimum length threshold can be set smaller, thus retaining more short boundary fitted lines. When the calibration requirement focuses more on avoiding interference from broken lines, noise, and other data, the minimum length threshold can be set larger, retaining only boundary fitted lines of sufficient length and higher reliability. Boundary fitted lines with a length greater than or equal to the minimum length threshold can be determined as valid lines, while boundary fitted lines with a length less than the minimum length threshold can be directly discarded.

[0050] Another example is to sort multiple boundary fitting lines by length and select a preset number of boundary fitting lines with the highest length as valid lines. The setting of the preset number also follows the principle of balance. If more feature details need to be retained, the preset number can be set to be larger to include more boundary fitting lines with the highest length. If it is necessary to reduce interference, simplify calculations, and improve the stability of fringe angles, the preset number can be set to be smaller, selecting only a small number of boundary fitting lines with the longest length and the strongest representativeness.

[0051] Another example is the adaptive filtering achieved through cluster analysis of the lengths of the fitted boundary lines. This automatically distinguishes between valid lines and short, noisy lines without requiring manual preset of a fixed length threshold or specification of the selection quantity. Specifically, the lengths of all fitted boundary lines are first extracted. Then, clustering algorithms such as K-means are used to divide all fitted boundary lines into several data clusters based on their length values. Clusters with concentrated and generally large length values ​​typically correspond to valid fitted lines generated from real fringe boundaries, while clusters with discrete, small, and randomly distributed lengths are often invalid short lines formed by image noise, false edges, or interference points. After clustering, fitted boundary lines belonging to short-length noise clusters are directly discarded, retaining only lines belonging to long-length valid clusters for subsequent fringe angle calculations. For example, adaptive discrimination conditions can be set based on the cluster center length or the average length within a cluster, classifying clusters with cluster center lengths greater than the overall average length of all fitted boundary lines as valid clusters, and discarding the rest as noise clusters. This clustering method can automatically adjust the filtering rules according to the actual length distribution of stripes in the current calibration image. This avoids the problem of poor adaptability of fixed thresholds under different calibration conditions and reduces the subjective error caused by manually setting parameters. While retaining effective stripe features, it can more stably filter out random noise and interference short lines, thereby further improving the reliability of stripe angle solution and overall calibration accuracy.

[0052] Method 2: Filtering based on slope.

[0053] For example, valid lines can be obtained by filtering boundary-fitted lines based on their slopes. A reasonable slope range can be preset as a filtering threshold. Boundary-fitted lines with slopes within this reasonable range are judged as valid lines, while those with slopes exceeding the reasonable range are judged as abnormal lines and removed. The setting of the reasonable slope range also needs to balance the preservation of feature details and the elimination of interfering data. When it is desired to retain more feature details with similar slopes, the range can be appropriately widened. To avoid abnormal data with excessive slope deviations affecting the calibration results, the range can be appropriately narrowed.

[0054] Another example is that multiple boundary fitting lines can be sorted according to the degree of their slopes being close to the target slope, and a preset number of boundary fitting lines with the highest degree of closeness can be selected as valid lines. The preset number can be flexibly adjusted according to calibration requirements. When it is important to retain details, the number can be increased appropriately, and when it is important to ensure the robustness of the results, the number can be reduced appropriately. This ensures that the slopes of the boundary fitting lines used to calculate the fringe angles are true and reliable, and improves calibration accuracy.

[0055] Another example is the adaptive selection of boundary fitted lines, which can be achieved through slope clustering without manually presetting a fixed slope range or specifying the number of selections. Specifically, the slopes corresponding to each boundary fitted line can be used as clustering features. The clustering algorithm automatically divides all boundary fitted lines into several data clusters according to their slope distribution. Since the boundaries of real stripes in the calibration scene usually have highly consistent slope characteristics, they will naturally form a main cluster with concentrated slopes and a compact distribution. Lines fitted by noise, false edges, or interference points will form small clusters with discrete slopes and a messy distribution. After clustering, the boundary fitted lines belonging to these discrete and messy clusters are removed, and only the lines within the main cluster with concentrated slopes are retained as valid lines to participate in subsequent stripe slope calculations. For example, adaptive discrimination conditions can be set based on the cluster center slope or the mean slope within the cluster results. Clusters with a cluster center slope greater than the overall slope of all boundary fitted lines are judged as valid clusters, while the rest are judged as noise clusters and removed. This clustering method can automatically adjust the filtering rules according to the actual slope distribution of the stripes in the current calibration image. This avoids the problem of poor adaptability of fixed thresholds under different calibration conditions and reduces the subjective error caused by manually setting parameters. While retaining effective stripe features, it can more stably filter out random noise and interference boundaries, thereby further improving the reliability of stripe angle solution and overall calibration accuracy.

[0056] Method 3: Filter based on length and slope.

[0057] By combining dual screening, we can eliminate excessively short lines by length and abnormal lines by slope, retaining valid lines that meet both length and slope requirements, and finally determining the stripe angle based on the slope of the valid lines.

[0058] It should be noted that the method of selecting boundary fitting lines based on both length and slope is similar to the above embodiments, and will not be described in detail here.

[0059] S130. Determine the stripe direction based on the direction of gray value change on both sides of the fitted straight line at each boundary.

[0060] Grayscale value refers to the brightness value of a pixel in an image, and it is a fundamental quantitative indicator for image feature analysis. The differences in grayscale values ​​between different pixels constitute the brightness and boundary features of an image. The direction of grayscale value change refers to the trend of increasing or decreasing grayscale values ​​of pixels on both sides of the boundary fitting line. For example, the grayscale value on the left side of the line gradually increases from left to right, and the grayscale value on the right side gradually decreases from left to right. Stripe direction is a core target grayscale feature parameter for 3D HUD image calibration. It is determined by the direction of grayscale value change on both sides of the boundary fitting line, reflecting the grayscale distribution pattern within the image area to be processed. Stripe direction can also be applied to the eye-tracking stage in the subsequent 3D display process, providing accurate grayscale feature basis for real-time capture of changes in user eye position and dynamic adjustment of the 3D HUD image display mode, achieving real-time and accurate eye tracking.

[0061] Specifically, for each boundary fitted line, the trend of grayscale value changes of pixels on both sides can be detected. For example, by detecting the difference between the grayscale values ​​of pixels on the left and right sides of the boundary fitted line, it can be determined whether the grayscale value increases or decreases from left to right, thus determining the direction of grayscale value change. By statistically analyzing the grayscale value change directions on both sides of all boundary fitted lines, the most prevalent and representative change directions are selected and identified as the stripe directions, providing a directional basis for subsequent image calibration.

[0062] The technical solution of this invention, by acquiring the boundary fitting lines of each image region to be processed, can eliminate the interference of image noise and discrete points on the boundaries, obtaining a regular and accurate boundary reference. By determining the stripe angle based on the slope and length of each boundary fitting line, stripe features can be accurately calculated based on the direction and effectiveness of the lines, ensuring the reliability of the angle calculation. By determining the stripe direction based on the direction of grayscale value changes on both sides of each boundary fitting line, the target orientation can be accurately locked through the grayscale change pattern, providing accurate directional parameters for image calibration, thus improving the overall accuracy and reliability of image calibration.

[0063] Example 2 Figure 2 This is a flowchart of an image calibration method provided in Embodiment 2 of the present invention. This embodiment adds a specific process for obtaining the boundary fitting lines of each image region to be processed, based on Embodiment 1. The specific content of steps S230-S240 is largely the same as steps S120-S130 in Embodiment 1, and therefore will not be repeated in this embodiment. Figure 2 As shown, the method includes: S210. Based on the grayscale values ​​of each pixel in each image region to be processed, perform boundary extraction on each image region to be processed and obtain the coordinates of each boundary pixel.

[0064] Specifically, the grayscale values ​​of pixels within the image region to be processed vary. Boundary pixels, which are the dividing points between the image region and the background region, exhibit significant abrupt changes in grayscale value. Therefore, boundary pixels can be used as the basis for boundary extraction. In practical applications, boundaries can be extracted based on grayscale values ​​in various ways, such as gradient analysis and index distribution analysis. Using these methods, pixels with abrupt changes in grayscale value within the image region to be processed—i.e., boundary pixels—can be accurately identified. Then, the two-dimensional coordinates of each boundary pixel are collected and recorded to obtain the set of boundary pixel coordinates corresponding to each image region to be processed.

[0065] For example, based on the grayscale values ​​of each pixel in each image region to be processed, boundary extraction is performed on each image region to be processed, and the coordinates of each boundary pixel can be determined in the following two ways: Method 1: First, calculate the grayscale gradient of each pixel in the image region to be processed. Then, based on the preset gradient threshold, select the first pixel with a gradient greater than or equal to the threshold as the boundary candidate pixel, and the second pixel with a gradient less than the threshold as the non-boundary pixel. Finally, determine the intersection based on the distribution of the two types of pixels, and collect the pixel coordinates at the intersection as the boundary pixel coordinates.

[0066] Figure 3 This is a flowchart of an image calibration method provided in Embodiment 1 of the present invention. Step S210 mainly includes the following steps S2101 to S2102: S2101. For each image region to be processed, determine the gradient of the grayscale value of each pixel in that image region.

[0067] Specifically, the image calibration device calculates the grayscale gradient for each pixel within each region of the image to be processed. The grayscale gradient refers to the rate of change of grayscale value between a pixel and its neighboring pixels; its magnitude directly reflects the degree of abrupt change in the grayscale value of that pixel. Boundary pixels, located at region boundaries, have significantly different grayscale values ​​from their neighboring pixels, thus their grayscale gradient values ​​are usually much larger than those of non-boundary pixels. Non-boundary pixels, situated within the region, exhibit gentler grayscale changes, resulting in smaller grayscale gradient values. In practical calculations, mature gradient calculation algorithms such as the Sobel algorithm, Prewitt algorithm, and Canny algorithm can be used. By calculating the grayscale changes of a pixel in the horizontal, vertical, or diagonal directions, the gradient value of that pixel is accurately obtained, providing a basis for subsequent boundary selection.

[0068] S2102. Based on each gradient, perform boundary extraction on the image region to be processed and obtain the coordinates of each boundary pixel.

[0069] Specifically, after obtaining the gradient values ​​of all pixels within the image region to be processed, boundary pixels are selected from all pixels using preset filtering rules. These filtering rules can be set according to the actual image features and calibration accuracy requirements. Pixels whose grayscale gradient values ​​meet preset conditions are selected; for example, pixels with gradient values ​​greater than a preset threshold are selected. These pixels are considered boundary pixels with significant grayscale value abrupt changes. Finally, the coordinates of the selected boundary pixels are collected and organized to obtain the boundary pixel coordinates of the image region to be processed. For example, preliminary edge filtering is performed using a preset gradient magnitude threshold, removing background and smooth region pixels with gradient magnitudes below the threshold, and retaining candidate edge pixels with high gradient magnitudes. Then, non-maximum suppression is performed along the gradient direction of each pixel, retaining only pixels with the largest local magnitude in the gradient direction and removing redundant pseudo-edges. Finally, through dual-threshold detection and edge connectivity tracking, pixels that meet the strong edge threshold are used as reference edge points, and pixels within the weak edge threshold and connected to the reference edge are added to form a continuous and complete boundary. The horizontal and vertical coordinates of all pixels on each boundary are recorded to obtain the set of boundary pixel coordinates.

[0070] Optionally, based on each gradient, boundary extraction is performed on the image region to be processed to obtain the coordinates of each boundary pixel, including: determining a gradient threshold; obtaining each first pixel in the image region to be processed whose gradient is greater than or equal to the gradient threshold, and each second pixel in the image region to be processed whose gradient is less than the gradient threshold; and based on each first pixel and each second pixel, boundary extraction is performed on the image region to be processed to obtain the coordinates of each boundary pixel.

[0071] Specifically, the gradient threshold is used to distinguish between boundary pixels and non-boundary pixels. In practical applications, the gradient threshold can be flexibly selected according to the actual application scenario. It supports manually fixed threshold settings, such as directly taking fixed parameters like 50 or 100; it can also adopt adaptive threshold calculation methods, such as automatically deriving the optimal threshold based on the grayscale distribution characteristics of the image region; and it can also adapt to the differentiated setting requirements of global threshold and local threshold. When presetting the gradient threshold, if it is necessary to highlight image edges and retain more fine boundary features, the gradient threshold can be set smaller, while if it is necessary to reduce noise interference and eliminate weak false boundaries, the gradient threshold can be set larger. After obtaining the gradient values ​​of all pixels in the image region to be processed, all pixels are divided into two categories according to the set gradient threshold. One category is pixels with gradient values ​​greater than or equal to the gradient threshold, defined as the first pixel. The grayscale value of the first pixel changes significantly, which meets the characteristics of a boundary pixel and is likely to be a boundary pixel. The other category is pixels with gradient values ​​less than the gradient threshold, defined as the second pixel. The grayscale value of the second pixel changes gradually and belongs to non-boundary pixels within the image region to be processed.

[0072] Furthermore, the image calibration device analyzes the distribution of the first and second pixels to determine the boundary between the two types of pixels. The pixels at the boundary are the actual boundary pixels of the image region to be processed. By collecting and organizing the coordinates of the pixels at the boundary, the set of boundary pixel coordinates of the image region to be processed can be obtained, thus completing the boundary extraction process.

[0073] Method 2: First, sort the gray values ​​of the same pixel in each image region to be processed. Then, determine the index value of the pixel in each index distribution map based on the sorting result. Finally, distinguish boundary and non-boundary pixels in each index distribution map according to the index threshold, and extract the coordinates of the boundary pixel at the intersection of the two types of pixels.

[0074] Figure 4 This is a flowchart of an image calibration method provided in Embodiment 1 of the present invention. Step S210 mainly includes the following steps S2101 to S2103: S2101. For the same pixel in each image region to be processed, sort the gray values ​​of the pixel in each image region to be processed; based on the sorting results, determine the index value corresponding to the pixel in each index distribution map.

[0075] Specifically, for the same pixel in all image regions to be processed, firstly, the grayscale value of that pixel in each image region to be processed is obtained, i.e., the grayscale data of that pixel in different images to be processed. Then, the grayscale values ​​are sorted according to a preset order, such as from smallest to largest or from largest to smallest. Then, based on that pixel, for each image region to be processed, the index value of the pixel at the same coordinate position in the index distribution map corresponding to that image region is determined according to the position of the pixel's grayscale value in the sorting result. This process is repeated for all pixels in each image region to be processed, finally generating an index distribution map that corresponds one-to-one with each image region to be processed. For example, if there are 3 images to be processed, and the gray values ​​of a certain pixel in these 3 images are 120, 150, and 180 respectively, and after sorting the gray values ​​from smallest to largest, they are 120 (sort 1), 150 (sort 2), and 180 (sort 3), then the index value of this pixel in the index distribution map corresponding to the first image to be processed is 1, the index value in the index distribution map corresponding to the second image is 2, and the index value in the index distribution map corresponding to the third image is 3.

[0076] Figure 5 This is a schematic diagram of an index distribution map sorting process provided in Embodiment 2 of the present invention. Figure 5The upper half of the diagram displays the grayscale values ​​of two pixel coordinates from eight images. The horizontal columns 1-8 represent the eight original images, with each column corresponding to one image. The two values ​​within each column represent the grayscale values ​​of two different pixel coordinates within the same image. The lower half displays eight sorted index distribution maps. Based on the grayscale values ​​from the upper half, eight index distribution maps were generated, each corresponding to one of the original images. The horizontal columns 1-8 represent these eight index distribution maps, with each column representing one index map. The two values ​​within each column represent the index values ​​of the two pixel coordinates from the upper half within the corresponding index distribution map.

[0077] In one possible implementation, if the same pixel has the same grayscale value in different image regions to be processed, then the neighboring pixels of that pixel are used as a reference to extract the already determined index value distribution features within the neighborhood. Based on these index value distribution features, the ranking of the same pixel with the same grayscale value in different image regions to be processed is determined. For example, the adjacent pixels within the 4-neighborhood or 8-neighborhood of that pixel are obtained, and the distribution trend of the already determined index values ​​of these neighboring pixels in the corresponding index distribution map is analyzed. If the index value within the neighborhood of that pixel in one image region shows an increasing trend, while it shows a decreasing trend in another image region, then the pixels with the same grayscale value are secondary-ranked according to a preset priority rule or combined with the average intensity difference of the neighborhood index values, ensuring the uniqueness and continuity of index allocation. By introducing neighborhood feature analysis, the ranking ambiguity caused by the same grayscale value is effectively solved, and the accuracy and stability of the index distribution map are improved.

[0078] Optionally, the same pixel in each image region to be processed is determined by either of the following methods: determining pixels with the same pixel coordinates in each image region to be processed as the same pixel; or, aligning the pixel coordinates of the pixels in each image region to be processed, and determining pixels with the same pixel coordinates in each image region to be processed as the same pixel.

[0079] Specifically, the same pixel in each region of the image to be processed is determined by any of the following methods: The first method involves identifying pixels with the same coordinates in different image regions as the same pixel. This method is suitable for scenarios where pixel coordinate alignment has already been achieved in the image regions, meaning that the acquisition angles and positions of multiple images are identical, and their pixel coordinate systems are the same. For example, multiple images obtained by acquiring the same target from the same location using the same acquisition device and parameters. In this case, pixels with the same coordinates in each image correspond to pixels at the same physical location, and therefore can be directly identified as the same pixel without additional coordinate alignment processing. This method is simple, efficient, and saves processing time. The second method involves aligning the pixel coordinates of pixels in different image regions, identifying pixels with the same coordinates after alignment as the same pixel.

[0080] The second method is suitable for scenarios where the coordinates of the image regions to be processed are not aligned. For example, slight positional shifts, angular deviations, or minor shaking of the acquisition device may occur during the acquisition of multiple images, resulting in inconsistent pixel coordinate systems among the images. In this case, pixel coordinate alignment of all image regions to be processed needs to be performed first. Mature coordinate alignment algorithms currently available can be used, such as feature point matching alignment algorithms and perspective transformation alignment algorithms. Through coordinate alignment algorithms, the pixel coordinates of all image regions to be processed are unified into the same coordinate system.

[0081] S2102. For each index distribution map, based on the index value of each pixel in the index distribution map, perform boundary extraction to obtain the coordinates of each boundary pixel.

[0082] Specifically, for each index distribution map, the image calibration device analyzes the distribution of index values ​​for each pixel in the image. Since the grayscale variation patterns of the boundary regions in multiple images differ significantly from those within the regions, their corresponding index values ​​also exhibit abrupt changes. For example, the index values ​​of pixels in boundary regions typically show significant jumps, while those in non-boundary regions are relatively flat. Therefore, by extracting the boundaries of the index distribution map, boundary pixels can be accurately identified, and their coordinates obtained. For instance, by using a gradient thresholding method or edge detection operator to traverse and calculate the index distribution map, when the difference between the index value of a pixel and its neighboring pixels exceeds a preset difference threshold, the pixel is determined to be a boundary pixel, and its coordinates are recorded. After traversing all pixels within the index distribution map, all boundary pixel coordinates are summarized to form the set of boundary coordinates corresponding to the index distribution map. The preset difference threshold can be set manually or through adaptive calculation. Manual setting allows you to directly set a specific value based on calibration requirements, while adaptive calculation automatically determines the optimal difference threshold based on the index value distribution characteristics of the index distribution map. When manually setting the preset difference threshold, if you need to fully preserve the fine boundary features of the image and capture as many feature details as possible, you can set the difference threshold to a smaller value. Conversely, if you need to reduce noise and interference data and avoid weak edges and false boundaries affecting the boundary extraction effect, you can set the difference threshold to a larger value.

[0083] Optionally, for each index distribution map, based on the index value of each pixel in the index distribution map, boundary extraction is performed to obtain the coordinates of each boundary pixel, including: obtaining an index threshold for each index distribution map; determining each third pixel in the index distribution map whose index value is greater than or equal to the index threshold, and each fourth pixel in the index distribution map whose index value is less than the index threshold; and generating the coordinates of each boundary pixel in the index distribution map based on each third pixel and each fourth pixel in the index distribution map.

[0084] Specifically, the index threshold is a parameter that distinguishes boundary pixels from non-boundary pixels in the index distribution map. Similar to the gradient threshold, the index threshold can be determined using a fixed threshold, a custom threshold, or a global / regional local threshold. When setting the index threshold, if more subtle boundary features need to be preserved and image details fully captured, the index threshold can be set smaller; conversely, if noise interference needs to be reduced and weak or false boundaries eliminated, the index threshold can be set larger. After determining the index threshold, the image calibration device divides all pixels in the index distribution map into two categories. One category consists of pixels with an index value greater than or equal to the index threshold, defined as the third pixel. The third pixel has a higher index value and corresponds to pixels with relatively high grayscale values ​​in multiple images to be processed; its distribution boundary usually coincides with the actual boundary of the image region to be processed. The other category consists of pixels with an index value less than the index threshold, defined as the fourth pixel. The fourth pixel has a lower index value and corresponds to pixels with relatively low grayscale values ​​in multiple images to be processed; these are non-boundary pixels within the image region to be processed. Finally, the image calibration device analyzes the distribution boundaries of the third and fourth pixels in the index distribution map. The intersection of the two types of pixels is the boundary of the index distribution map. By collecting the coordinates of the pixels at the intersection, a set of boundary pixel coordinates of the image region to be processed corresponding to the index distribution map can be generated.

[0085] In one possible implementation, all index values ​​except the largest index value can be determined as index thresholds. Based on each determined index threshold, the boundary pixel coordinates of each index distribution map are extracted to ensure comprehensive capture of various boundary features and improve the completeness and accuracy of boundary extraction.

[0086] In another possible implementation, obtaining the index threshold includes: determining the index threshold based on each index value contained in the index distribution map.

[0087] To better extract boundary pixel coordinates, this application extracts boundary pixel coordinates for each index distribution map based on the index values ​​contained in the current index distribution map and a determined index threshold. This ensures comprehensive capture of various boundary features of the current index distribution map, improving the completeness and accuracy of boundary extraction. Furthermore, extracting boundaries based on the actual index values ​​contained in the current index distribution map reduces computational load. For example, all index values ​​except the maximum value in the current index distribution map are determined as index thresholds. For instance, if the current index distribution map A contains index values ​​from 1 to 8, then values ​​from 1 to 8 are all determined as index thresholds, and boundary pixel coordinates are extracted for the current index distribution map A based on values ​​from 1 to 8. Similarly, if the current index distribution map B contains index values ​​from 1 to 6, then values ​​from 1 to 6 are all determined as index thresholds, and boundary pixel coordinates are extracted for the current index distribution map B based on values ​​from 1 to 6.

[0088] S220. Perform line fitting based on the coordinates of each boundary pixel to determine the boundary fitting line.

[0089] After obtaining the coordinates of each boundary pixel based on the above embodiments, a straight line fitting algorithm can be used to fit the collected boundary pixel coordinates. The straight line fitting algorithm can be the least squares method, Hough transform, etc. The boundary fitting line obtained by the straight line fitting algorithm can fit all boundary pixel coordinates to the greatest extent.

[0090] Optionally, based on the grayscale values ​​of each pixel in each image region to be processed, the boundary of each image region to be processed is extracted, and after obtaining the coordinates of each boundary pixel, a straight line is fitted based on the coordinates of each boundary pixel. Before determining the fitted straight line of the boundary, the method further includes: filtering the coordinates of each boundary pixel based on preset feature parameters; wherein, the preset feature parameters include, but are not limited to, one or more of the following: position features, orientation features, and shape features.

[0091] The preset feature parameters include, but are not limited to, one or more of positional features, orientation features, and shape features. In practical applications, any one of these feature parameters can be selected individually for discrimination, two feature parameters can be combined, or all three feature parameters can be combined for analysis to adapt to different recognition scenarios and calibration requirements. Positional features are used to filter boundary pixels whose positions meet preset requirements; for example, pixels that exceed a reasonable range of the image area to be processed or deviate from the approximate position of the preset boundary are excluded. Orientation features are used to filter pixels whose orientation is consistent with the preset boundary orientation; for example, if the preset boundary is approximately horizontal, boundary pixels with a nearly horizontal orientation are filtered out, excluding outliers with excessively large orientation deviations. Shape features are used to filter pixels that conform to the preset boundary shape characteristics; for example, if the preset boundary is a straight line, continuous pixels that can form a straight line are filtered out, excluding discrete, isolated pixels that cannot form a straight line.

[0092] Specifically, filtering can be performed by setting thresholds or conditions for each feature parameter. For example, thresholds for positional deviation, orientation deviation, and shape similarity can be set, and then the feature parameters corresponding to each boundary pixel coordinate can be checked one by one to determine whether they meet the preset filtering conditions. Alternatively, cluster analysis can be introduced for filtering, relying on the distribution patterns of feature parameters to perform clustering and remove abnormal data that deviates from the overall feature distribution. The image calibration device can then remove boundary pixel coordinates that do not meet the filtering conditions from the boundary pixel coordinate set.

[0093] It should be noted that the selection of preset feature parameters and the setting of filtering conditions can be flexibly adjusted according to the actual features of the image to be processed and the calibration accuracy requirements. For example, in noisy scenes, the filtering weights of shape and orientation features can be increased to further filter noisy pixels; in scenes with relatively fixed boundary positions, the filtering weights of position features can be increased to ensure the accuracy of boundary pixel coordinates. Through filtering, the purity of the boundary pixel coordinate set can be effectively improved, providing a more reliable input for the subsequent line fitting step, thereby improving the accuracy of the boundary fitted line.

[0094] S230. Determine the stripe angle based on the slope and length of the fitted straight line at each boundary.

[0095] Optionally, determining the fringe angle based on the slope and length of each boundary fitted line includes: filtering each boundary fitted line based on the slope and length of the boundary fitted line; and determining the fringe angle based on the slope of each filtered boundary fitted line.

[0096] S240. Determine the stripe direction based on the direction of gray value change on both sides of the fitted straight line at each boundary.

[0097] The technical solution of this invention, by acquiring the boundary fitting lines of each image region to be processed, can eliminate the interference of image noise and discrete points on the boundaries, obtaining a regular and accurate boundary reference. By determining the stripe angle based on the slope and length of each boundary fitting line, stripe features can be accurately calculated based on the direction and effectiveness of the lines, ensuring the reliability of the angle calculation. By determining the stripe direction based on the direction of grayscale value changes on both sides of each boundary fitting line, the target orientation can be accurately locked through the grayscale change pattern, providing accurate directional parameters for image calibration, thus improving the overall accuracy and reliability of image calibration.

[0098] Example 3 Figure 6 This is a schematic diagram of the structure of an image calibration device provided in Embodiment 3 of the present invention. Figure 6 As shown, the device includes: a boundary fitting line generation module 310, used to obtain boundary fitting lines for each image region to be processed; The stripe angle determination module 320 is used to determine the stripe angle based on the slope and length of the fitted straight line of each boundary; The stripe direction determination module 330 is used to determine the stripe direction based on the direction of change in gray values ​​on both sides of the fitted straight line of each boundary.

[0099] Optionally, the boundary fitting line generation module 310 specifically includes: a region acquisition unit for the image to be processed, used to: determine the region of interest based on the display area corresponding to the preset display content in each acquired original image; and for each original image, determine the region of the image to be processed from the original image based on the region of interest.

[0100] Optionally, the boundary fitting line generation module 310 specifically includes: a boundary pixel coordinate extraction unit, used to extract the boundaries of each image region to be processed based on the gray values ​​of each pixel in each image region to be processed, and obtain the boundary pixel coordinates; and a boundary fitting line determination unit, used to perform line fitting based on the boundary pixel coordinates and determine the boundary fitting line.

[0101] Optionally, the boundary pixel coordinate extraction unit specifically includes: a gradient determination subunit, used to: determine the gradient of the grayscale value of each pixel in each image region to be processed; and a boundary extraction subunit, used to: extract the boundary of the image region to be processed based on each gradient and obtain the coordinates of each boundary pixel.

[0102] Optionally, the boundary extraction subunit is specifically used for: determining a gradient threshold; obtaining each first pixel in the image region to be processed whose gradient is greater than or equal to the gradient threshold, and each second pixel in the image region to be processed whose gradient is less than the gradient threshold; and performing boundary extraction on the image region to be processed based on each first pixel and each second pixel to obtain the coordinates of each boundary pixel.

[0103] Optionally, the boundary pixel coordinate extraction unit specifically includes: a grayscale value sorting subunit, used to: sort the grayscale values ​​of the same pixel in each image region to be processed; an index value determination subunit, used to: determine the index value corresponding to the pixel in each index distribution map based on the sorting result; and a boundary extraction subunit, used to: extract the boundary for each index distribution map based on the index value of each pixel in the index distribution map, and obtain the coordinates of each boundary pixel.

[0104] Optionally, the boundary pixel coordinate extraction unit further includes: a same pixel point determination subunit, used to: determine pixels with the same pixel coordinates in each image region to be processed as the same pixel point; or, perform pixel coordinate alignment on pixels in each image region to be processed, and determine pixels with the same pixel coordinates in each image region to be processed as the same pixel point after alignment.

[0105] Optionally, the boundary extraction subunit is used to: obtain an index threshold for each index distribution map; determine each third pixel point in the index distribution map whose index value is greater than or equal to the index threshold, and each fourth pixel point in the index distribution map whose index value is less than the index threshold; and generate the coordinates of each boundary pixel in the index distribution map based on each third pixel point and each fourth pixel point in the index distribution map.

[0106] Optionally, a boundary extraction subunit is used to determine the index threshold based on the index values ​​contained in the index distribution map.

[0107] Optionally, the device further includes: a pixel coordinate filtering module, used to extract the boundaries of each image region to be processed based on the gray values ​​of each pixel in each image region to be processed, obtain the boundary pixel coordinates, perform line fitting based on the boundary pixel coordinates, and filter the boundary pixel coordinates based on preset feature parameters before determining the boundary fitted line; wherein, the preset feature parameters include, but are not limited to, one or more of the following: position features, orientation features, and shape features.

[0108] Optionally, the stripe angle determination module 320 is specifically used to: determine the stripe angle based on the slope and length of each boundary fitted line, including: filtering each boundary fitted line based on the slope and length of each boundary fitted line; and determining the stripe angle based on the slope of each filtered boundary fitted line.

[0109] The technical solution of this invention, by acquiring the boundary fitting lines of each image region to be processed, can eliminate the interference of image noise and discrete points on the boundaries, obtaining a regular and accurate boundary reference. By determining the stripe angle based on the slope and length of each boundary fitting line, stripe features can be accurately calculated based on the direction and effectiveness of the lines, ensuring the reliability of the angle calculation. By determining the stripe direction based on the direction of grayscale value changes on both sides of each boundary fitting line, the target orientation can be accurately locked through the grayscale change pattern, providing accurate directional parameters for image calibration, thus improving the overall accuracy and reliability of image calibration.

[0110] The image calibration device provided in this embodiment of the invention can execute an image calibration method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0111] Example 4 Figure 7 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0112] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0113] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0114] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as an image calibration method.

[0115] In some embodiments, an image calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image calibration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform an image calibration method by any other suitable means (e.g., by means of firmware).

[0116] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0118] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0120] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0121] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0122] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0123] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image calibration method, characterized in that, include: Obtain the boundary fitting straight lines for each region of the image to be processed; The stripe angle is determined based on the slope and length of the fitted straight line of each boundary; The direction of the stripes is determined based on the direction of change in gray values ​​on both sides of the fitted straight line of each boundary.

2. The method according to claim 1, characterized in that, Obtaining each image region to be processed includes: Based on the display area corresponding to the preset display content in each of the acquired original images, the region of interest is determined; For each original image, based on the region of interest, the image region to be processed is determined from the original image.

3. The method according to claim 1, characterized in that, The step of obtaining the boundary fitting lines for each image region to be processed includes: Based on the grayscale values ​​of each pixel in each image region to be processed, the boundaries of each image region to be processed are extracted to obtain the coordinates of each boundary pixel. Line fitting is performed based on the coordinates of each boundary pixel to determine the boundary fitted line.

4. The method according to claim 3, characterized in that, The step of extracting the boundaries of each image region based on the grayscale values ​​of each pixel in each image region to be processed, and obtaining the coordinates of each boundary pixel, includes: For each image region to be processed, determine the gradient of the grayscale value of each pixel in that image region. Based on the gradients, the boundaries of the image region to be processed are extracted to obtain the coordinates of each boundary pixel.

5. The method according to claim 4, characterized in that, The step of extracting the boundaries of the image region to be processed based on each gradient and obtaining the coordinates of each boundary pixel includes: Determine the gradient threshold; Obtain each first pixel in the image region to be processed whose gradient is greater than or equal to the gradient threshold, and each second pixel in the image region to be processed whose gradient is less than the gradient threshold; Based on the first pixel and the second pixel, the boundary of the image region to be processed is extracted to obtain the coordinates of each boundary pixel.

6. The method according to claim 3, characterized in that, The step of extracting the boundaries of each image region based on the grayscale values ​​of each pixel in each image region to be processed, and obtaining the coordinates of each boundary pixel, includes: For the same pixel in each of the image regions to be processed, sort the grayscale values ​​of that pixel in each of the image regions to be processed; Based on the sorting results, determine the index value of the pixel in each index distribution map; For each index distribution map, based on the index value of each pixel in the index distribution map, boundary extraction is performed to obtain the coordinates of each boundary pixel.

7. The method according to claim 6, characterized in that, The same pixel in each image region to be processed is determined by any of the following methods: Pixels with the same pixel coordinates in each of the image regions to be processed are identified as the same pixel; or, The pixel coordinates of the pixels in each of the image regions to be processed are aligned, and the pixels with the same pixel coordinates after alignment in each of the image regions to be processed are determined to be the same pixel.

8. The method according to claim 6, characterized in that, For each index distribution map, based on the index value of each pixel in the index distribution map, boundary extraction is performed to obtain the coordinates of each boundary pixel, including: For each index distribution map, obtain the index threshold; Identify the third pixels in the index distribution map whose index value is greater than or equal to the index threshold, and the fourth pixels in the index distribution map whose index value is less than the index threshold; Based on the third and fourth pixels in the index distribution map, the coordinates of each boundary pixel in the index distribution map are generated.

9. The method according to claim 8, characterized in that, The process of obtaining the index threshold includes: The index threshold is determined based on the index values ​​contained in the index distribution map.

10. The method according to claim 3, characterized in that, After extracting the boundaries of each image region based on the grayscale values ​​of each pixel in each image region to be processed and obtaining the coordinates of each boundary pixel, before performing line fitting based on the coordinates of each boundary pixel and determining the boundary fitting line, the method further includes: Based on preset feature parameters, the coordinates of each boundary pixel are filtered; The preset feature parameters include, but are not limited to, one or more of the following: position features, orientation features, and shape features.

11. The method according to claim 1, characterized in that, Determining the stripe angle based on the slope and length of the fitted straight lines at each of the boundaries includes: Based on the slope and length of the boundary fitted lines, each boundary fitted line is filtered. The stripe angle is determined based on the slope of the fitted lines of each of the filtered boundaries.

12. An image calibration device, characterized in that, include: The boundary fitting line generation module is used to obtain the boundary fitting lines for each image region to be processed. The stripe angle determination module is used to determine the stripe angle based on the slope and length of the fitted straight line of each boundary; The stripe direction determination module is used to determine the stripe direction based on the direction of change in gray values ​​on both sides of the fitted straight line of each boundary.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-11.

14. A computer storage medium, characterized in that, The computer storage medium stores computer instructions that are used to cause a processor to execute the method of any one of claims 1-11.

Images