Image processing method and device, vehicle and storage medium

Abstract

The application provides an image processing method and device, a vehicle and a storage medium. When performing binarization processing on an image containing a checkerboard, the electronic device can use different algorithms to process different regions of the image in combination with the characteristics of different regions of the image. In this way, the electronic device can more accurately detect the specific position of the checkerboard in the image, and then accurately calibrate the camera.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and in particular to an image processing method and device, a vehicle and a storage medium. BACKGROUND

[0002] Camera calibration is one of the most basic and important techniques in machine vision, and a checkerboard is often used as a calibration object for camera calibration. The detection accuracy of the checkerboard depends on the effect of binary morphological processing of the image. However, in actual detection scenarios, there can be various interferences, for example, when the ground paint is not matte, the black checkerboard can have a highlight area; or when the wall surface is connected with the black checkerboard, the checkerboard and the wall shadow can also be connected together.

[0003] The traditional binary morphological processing method is difficult to identify and suppress the highlight area when performing binary processing on an image containing a checkerboard, and cannot separate the wall shadow from the checkerboard area, so it can lead to inaccurate detection results of the checkerboard. Therefore, it is necessary to study a more accurate image processing method. SUMMARY

[0004] The present application aims to provide an image processing method and device, a vehicle and a storage medium. By implementing the method, when performing binary processing on an image containing a checkerboard, an electronic device can use different algorithms to process different regions of the image in combination with the characteristics of different regions of the image. In this way, the electronic device can more accurately detect the specific position of the checkerboard in the image, and further more accurately calibrate the camera.

[0005] The above and other objects will be achieved by the features of the independent claims. Further implementation forms will become clear from the dependent claims, the description and the drawings.

[0006] In a first aspect, the present application provides an image processing method, which includes: splitting an initial image to obtain an upper region image and a lower region image, the upper region image including an upper boundary of a checkerboard, and the lower region image including a lower boundary of the checkerboard; processing the upper region image using Otsu method to obtain a first image; processing the lower region image using an adaptive local mean threshold method to obtain a second image; and fusing the first image and the second image to obtain a target image.

[0007] It can be understood that when calibrating the camera, the objects in the viewfinder area of the camera can be very complex, and the chessboard is often used as a calibration object for camera calibration. Therefore, it is usually more difficult to accurately detect the chessboard in the image. For example, when the ground is not matte, the black chessboard will have a highlight area; or when the wall is connected with the black chessboard, the chessboard and the wall shadow will also be connected together. At present, the traditional binary morphological processing method is difficult to identify and suppress the highlight area when performing binary processing on the image containing the chessboard, and it is also difficult to separate the wall shadow from the chessboard area. Therefore, it is likely to cause inaccurate detection results of the chessboard.

[0008] It can be understood that due to the relative position relationship between the wall and the ground, when calibrating the camera, the wall and the wall shadow will generally be displayed in the upper half of the image, and the ground will generally be displayed in the lower half of the image. Therefore, in the present method, the electronic device can divide the initial image into two parts according to the specific position of the chessboard in the initial image, wherein the upper part is the upper area image, and the lower part is the lower area image. It should be understood that the "upper" and "lower" mentioned here are based on the premise that the objects in the image are displayed upright.

[0009] As for the characteristics of the two binary algorithms, namely the adaptive local mean algorithm and the OTSU algorithm, the adaptive local mean algorithm can effectively suppress the highlight area in the image, but cannot separate the black chessboard in the upper half and the wall shadow. Therefore, its processing effect on the lower half chessboard is better, and the OTSU algorithm can effectively separate the object shadow and the region of interest, but its processing effect on the highlight area is poor (there is a lot of white pollution), so its processing effect on the upper chessboard is better.

[0010] Therefore, in the present method, the electronic device can use the OTSU algorithm to perform binary processing on the upper area image to obtain the first image, and use the adaptive local mean threshold method to perform binary processing on the lower area image to obtain the second image. In this way, the boundaries of the chessboard and each black and white square in the obtained first image and second image can be more complete and clear, so as to more accurately detect the specific position of the chessboard in the image, and further better calibrate the camera.

[0011] In a possible implementation of the first aspect, the target image is obtained based on the first image and the second image, including: fusing the first image and the second image to obtain a third image; when the chessboard in the third image is occluded, determining a slope of a boundary of the chessboard in the occluded part of the third image based on a first inner corner point and a second inner corner point of the chessboard in the third image; and determining a first straight line and a second straight line according to the slope of the boundary of the chessboard in the occluded part of the third image, the first straight line and the second straight line being used to divide the third image into the target image and a fourth image, the fourth image containing a region in which the chessboard in the third image is occluded.

[0012] It should be understood that in an actual camera calibration scenario, the object can be very complex, which means that the chessboard in the first image and the second image can be occluded by other objects (for example, when calibrating a camera of a vehicle-mounted device, the chessboard in the captured image is likely to be occluded by the tire of the vehicle). Such occlusion undoubtedly reduces the detection accuracy of the chessboard.

[0013] Therefore, in the present embodiment, when the chessboard in the third image is occluded, the slope of the boundary of the occluded part of the chessboard can be determined based on two inner corner points, i.e., the first inner corner point and the second inner corner point, selected from the part of the chessboard that is not occluded, and two straight lines can be determined based on the slope to divide the third image into two images, i.e., the target image and the fourth image.

[0014] Specifically, whether the chessboard in the third image is occluded can be determined by a detection personnel and the determination result can be input into an electronic device responsible for executing the method, or the electronic device can detect whether the chessboard is occluded based on a corresponding visual algorithm. In addition, the two inner corner points, i.e., the first inner corner point and the second inner corner point, can be the two inner corner points closest to the occluded part and located in the part of the chessboard that is not occluded. The fourth image contains the part of the chessboard in the third image that is originally occluded, and the chessboard in the target image is a clear boundary chessboard. In this way, the detection accuracy of the chessboard can be further improved, and in the subsequent calibration process, the fourth image will actually be discarded, and only the target image is used for calibration.

[0015] In a possible implementation of the first aspect, in the method, the first region of the initial image is processed by using the Otsu method to obtain a first image, including: traversing a first interval with a first step length to obtain a suboptimal threshold value, the first interval being [min, max], the min being greater than 0, and the max being less than 255; determining a second interval based on the suboptimal threshold value, the second interval containing the suboptimal threshold value and being contained in the first interval; traversing the second interval with a second step length to obtain an optimal threshold value, the second step length being smaller than the first step length; and performing binaryzation processing on the image in the first region based on the optimal threshold value to obtain the first image.

[0016] Specifically, the first interval can be [0, 255], the second interval can be a subinterval of the first interval, and the second interval contains the suboptimal threshold value.

[0017] In the traditional Otsu method algorithm, the entire gray interval [0, 255] is usually traversed with a step length of 1 to accurately find the optimal threshold value, that is, 256 times of system operation process are required to obtain the optimal threshold value, and therefore a large amount of computing power and time is consumed. This undoubtedly increases the power consumption of the device and the efficiency of the algorithm is also low.

[0018] In the embodiment, by optimizing the step length to the first step length, where d is an odd number, the suboptimal threshold value temp is first found by traversing with the step length d, and then the optimal threshold value is found in the field centered on the suboptimal threshold value with the second step length, which can greatly reduce the number of calculations of the device, reduce the power consumption of the device, and improve the efficiency of the algorithm.

[0019] In a possible implementation of the first aspect, the first interval is [min, max], the min is greater than 0, the max is less than 255, the value of the first step length is the nearest odd integer to the arithmetic square root of the distance (max-min), and the value of the second step length is 1.

[0020] It should be understood that, in general, the optimal threshold value generally does not fall in such relatively extreme intervals as [0, 20] or [230, 255], and therefore it is not necessary to calculate the gray values in these intervals when traversing. Therefore, in the embodiment, by reducing the traversal interval to the first interval [min, max], the number of calculations of the device can be further reduced, the power consumption of the device can be reduced, and the efficiency of the algorithm can be improved. In addition, in the embodiment, the value of the first step length is set to the nearest odd integer to the arithmetic square root of the distance (max-min), and the value of the second step length is set to 1, which can prevent the suboptimal threshold value from not being referential due to the step length being too long, and also ensure that the optimal threshold value can be accurately determined in the field of the suboptimal threshold value.

[0021] In a possible implementation of the first aspect, the suboptimal threshold is temp, and the value of the first step length is d, and the second interval is [temp-(d-1) / 2, temp+(d-1) / 2].

[0022] The optimal threshold generally exists in a neighborhood centered at the suboptimal threshold and with a radius of 1 / 2 of the first step length. If the optimal threshold does not exist in the neighborhood, the suboptimal threshold generally will not be temp. For example, if the suboptimal threshold obtained in the first iteration is temp, but the actual optimal threshold does not exist in the interval [temp-(d-1) / 2, temp+(d-1) / 2], but is temp+(d-1) / 2+1, then the suboptimal threshold obtained in the first iteration should not be temp, but should be (temp+d). Therefore, in this implementation, the second interval is set to [temp-(d-1) / 2, temp+(d-1) / 2] in the second iteration, so that the accurate optimal threshold can be obtained based on the suboptimal threshold.

[0023] In a possible implementation of the first aspect, the first interval is [min, max], and obtaining the suboptimal threshold based on the first step length and the first interval comprises: in a case where the value of the first step length cannot divide (max-min) evenly, adjusting the upper limit of the first interval to obtain a third interval [min, max+m], where m is less than the value of the first step length, and (max+m-min) is an integer multiple of the value of the first step length; and traversing the third interval using the first step length to obtain the suboptimal threshold.

[0024] In this implementation, the upper limit of the first interval is optimized to max+m, to prevent too many thresholds at the tail of the threshold interval from being crossed by the step length, where m (m<=255-max, and m

[0025] In a possible implementation of the first aspect, the pixels in the second region include a first set of pixels and a second set of pixels, any pixel in the first set of pixels has a gray value greater than or equal to a first threshold, and any pixel in the second set of pixels has a gray value less than the first threshold. Before the second region of the initial image is processed using the adaptive local mean thresholding method to obtain a second image, the method further includes: reducing the gray value of all pixels in the first set of pixels by the first threshold, and reducing the gray value of all pixels in the second set of pixels to 0.

[0026] It should be understood that although the adaptive local mean is better for processing the lower half of the checkerboard, there is still a small amount of white noise in the processed image. In order to avoid the influence of the white noise on the accuracy of the detection result of the checkerboard, in the embodiment, before the adaptive local mean threshold method is used to process the second region of the initial image, the overall gray value of the image corresponding to the second region can be reduced as a whole, that is, the pixel value in the corresponding region of the gray image is uniformly reduced by δ, where δ is an integer in [0, 255], thereby suppressing the highlight region to prevent a large amount of white noise points from being generated in the image of the highlight region after the binarization processing.

[0027] It can be understood that the greater the δ, the deeper the suppression. It can be understood that if the current pixel value of the image is less than δ, the pixel value at this position is set to 0. Optionally, when the gray value of the image is suppressed (i.e., the process of uniformly reducing the gray value of the image), the strength of the highlight can be determined first; when the highlight is strong and weak, the local pixel mean value is used to determine the strength of the highlight before processing, and only the strong highlight (i.e., the highlight degree is greater than a certain threshold, which can be set according to the specific calibration scene) is processed. Finally, the adaptive local mean threshold method is used for image binarization to obtain an ideal effect.

[0028] In a second aspect, the present application provides an image processing device, the device comprising: a splitting unit configured to split an initial image to obtain an upper region image and a lower region image, the upper region image comprising an upper boundary of a checkerboard, and the lower region image comprising a lower boundary of the checkerboard; a first processing unit configured to process the upper region image using the Otsu method to obtain a first image; a second processing unit configured to process the lower region image using an adaptive local mean threshold method to obtain a second image; and a fusion unit configured to fuse the first image and the second image to obtain a target image.

[0029] In combination with the second aspect, in a possible implementation, the device further comprises a segmentation unit, and the fusion unit is specifically configured to: fuse the first image and the second image to obtain a third image; and when the checkerboard is occluded in the third image, determine a slope of a boundary of the checkerboard of the occluded part in the third image based on a first internal corner point and a second internal corner point of the checkerboard in the third image; and determine a first straight line and a second straight line according to the slope of the boundary of the checkerboard of the occluded part in the third image, the first straight line and the second straight line being used to divide the third image into the target image and a fourth image, the fourth image comprising a region in which the checkerboard is occluded in the third image.

[0030] In a possible implementation of the second aspect, the first processing unit is specifically configured to: traverse the first interval using a first step length to obtain a sub-optimal threshold, the first interval being [min, max], the min being greater than 0, and the max being less than 255; determine a second interval based on the sub-optimal threshold, the second interval containing the sub-optimal threshold, and the second interval being contained in the first interval; traverse the second interval using a second step length to obtain an optimal threshold, the second step length being smaller than the first step length; and perform binarization processing on the image in the first region based on the optimal threshold to obtain the first image.

[0031] In a possible implementation of the second aspect, the first interval is [min, max], the min is greater than 0, the max is less than 255, the value of the first step length is the nearest odd integer to the arithmetic square root of (max-min), and the value of the second step length is 1.

[0032] In a possible implementation of the second aspect, the sub-optimal threshold is temp, the value of the first step length is d, and the second interval is [temp-(d-1) / 2, temp+(d-1) / 2].

[0033] In a possible implementation of the second aspect, the first interval is [min, max], and the first processing unit is specifically configured to: in a case where the value of the first step length cannot divide (max-min) evenly, adjust the upper limit of the first interval to obtain a third interval [min, max+m], the m being smaller than the value of the first step length, and the (max+m-min) being an integer multiple of the value of the first step length; and traverse the third interval using the first step length to obtain the sub-optimal threshold.

[0034] In a possible implementation of the second aspect, the pixels in the second region include a first set of pixels and a second set of pixels, any pixel in the first set of pixels having a gray value greater than or equal to a first threshold, and any pixel in the second set of pixels having a gray value smaller than the first threshold; and the second processing unit is further configured to: reduce the gray value of all pixels in the first set of pixels by the first threshold, and reduce the gray value of all pixels in the second set of pixels to 0.

[0035] In a third aspect, the present application provides a vehicle, including at least one processor and a memory, the memory being coupled to the processor, the memory being configured to store computer program code, the computer program code including computer instructions, and the processor being configured to invoke the computer instructions to enable the electronic device to perform the method in the first aspect or any possible implementation manner of the first aspect.

[0036] In a fourth aspect, the present application provides a chip system applied to an electronic device, the chip system comprising one or more processors configured to invoke computer instructions to cause the electronic device to perform the method according to the first aspect or any possible implementation of the first aspect.

[0037] In a fifth aspect, the present application provides a computer-readable storage medium comprising instructions which, when executed on an electronic device, cause the electronic device to perform the method according to the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A schematic diagram of the effect of binarization processing of an image using different algorithms provided by an embodiment of the present application;

[0039] Figure 2 A flowchart of an image processing method provided by an embodiment of the present application;

[0040] Figure 3 A process schematic diagram of binarization processing of an image provided by an embodiment of the present application;

[0041] Figure 4 A flowchart of an algorithm for optimizing Otsu method provided by an embodiment of the present application;

[0042] Figure 5 A flowchart of a method for dividing a chessboard provided by an embodiment of the present application;

[0043] Figure 6 A process schematic diagram of dividing an image provided by an embodiment of the present application;

[0044] Figure 7 A structural schematic diagram of an image processing apparatus provided by an embodiment of the present application;

[0045] Figure 8 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] The terms used in the following embodiments of the present application are only for the purpose of describing the particular embodiments and are not intended to be limiting of the present application. As used in the specification and the appended claims of the present application, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used in the specification and in the claims, are used to mean any one of the items in the list or all of the items in the list.

[0047] Since the embodiments of the present application relate to image processing method and chessboard calibration principle, in order to facilitate understanding, the related terms involved in the embodiments of the present application are introduced as follows.

[0048] (1) Camera calibration and chessboard

[0049] Since the distortion degree of each lens in the production and assembly process is different, the camera calibration can correct the lens distortion, and generate the corrected image-the corrected lens distortion; in addition, the camera calibration can also be used to establish the camera imaging geometric model, and reconstruct the three-dimensional scene from the photographed image based on the geometric model. Specifically: when we take pictures with a camera, we get some spatial information (such as distance, size, etc.) from the photo, which is to get three-dimensional information from two-dimensional image. When we take pictures, the spatial object information is changed into two-dimensional image through the camera, which is originally irreversible. But if we can find a mathematical model of the camera, we can: from two-dimensional image + model to get the original three-dimensional information. Calibration is to find this model.

[0050] The objects photographed by the camera are in the three-dimensional world coordinate system, and the three-dimensional camera coordinate system is converted to the two-dimensional image coordinate system when the camera is imaged. The conversion matrix of different lenses when imaging is different, which may introduce distortion, and the role of calibration is to approximately estimate the conversion matrix and distortion coefficient. In order to estimate, the coordinates of a number of points in the three-dimensional world coordinate system and the coordinates in the two-dimensional image coordinate system are needed. The traditional camera calibration method is to calculate the 11 unknown parameters in the camera projection matrix M through the world coordinate set (Xi, Yi, Zi) and their projection coordinates set (ui, vi) on the image plane, which needs to give three mutually perpendicular planes to calibrate (the condition is relatively strict, which is difficult to realize in general case). The chessboard calibration only needs two planes, and only the corner points of the black and white grid are needed to mark, which is much easier than the original calibration. The significance of chessboard calibration is to overcome the defect of high-precision calibration object required by traditional calibration method, and only a printed chessboard is needed.

[0051] (2) Otsu method

[0052] OTSU algorithm (also known as the Otsu method) is a high efficiency algorithm for image binarization proposed by Japanese scholar OTSU in 1979. The original image is divided into foreground and background images by using threshold. According to the principle of Otsu method, the method is also called maximum inter-class variance method, because the inter-class variance of foreground and background images is maximum after image binarization and segmentation by the threshold obtained by Otsu method. It is considered as the best algorithm for threshold selection in image segmentation, and it is simple to calculate and is not affected by image brightness and contrast, so it has been widely used in digital image processing. It is to divide the image into background and foreground parts according to the gray level characteristics of the image. Because variance is a measure of uniformity of gray level distribution, the greater the inter-class variance between background and foreground, the greater the difference between the two parts of the image. When part of the foreground is misclassified as background or part of the background is misclassified as foreground, the difference between the two parts will become smaller. Therefore, the segmentation that maximizes the inter-class variance means the minimum misclassification probability.

[0053] The assumption of OTSU algorithm is that there is a threshold TH to divide all pixels of the image into two classes C1 (less than TH) and C2 (greater than TH), and the mean values of the two classes of pixels are m1 and m2 respectively, and the global mean value of the image is mG. At the same time, the probabilities of pixels being divided into C1 and C2 classes are p1 and p2 respectively. Therefore, we have:

[0054] p1m1+p2m2=mG;(1)

[0055] p1+p2=1;(2)

[0056] According to the concept of variance, the inter-class variance expression is:

[0057] σ 2 =(m1-mG) 2 +2(m2-mG) 2 ;(3)

[0058] Substitute equation (1) into equation (3) to obtain:

[0059] σ 2 =p1p2(1-m2) 2 ;(4)

[0060] The gray level k that can maximize the above equation is the optimal threshold of OTSU.

[0061] Wherein:

[0062]

[0063]

[0064]

[0065] In order to find the gray level k that maximizes equation (4), the traditional Otsu method will iterate through 0 to 255 gray levels in sequence to find the k that maximizes equation (4).

[0066] (3) Adaptive Local Mean Thresholding Method

[0067] For images with balanced colors, a single threshold can be used to perform thresholding. However, sometimes the colors in an image are uneven. In such cases, a single threshold will not yield a clear and effective image segmentation. Therefore, an adaptive local mean thresholding method is needed in these situations.

[0068] Simply put, the adaptive local mean thresholding method obtains a threshold by calculating the weighted average of the neighboring regions along the axis of each pixel, and then uses this threshold to process the current pixel. Its advantage is that it can better handle images with large differences in brightness.

[0069] Camera calibration is one of the most fundamental and important technologies in machine vision, and checkerboard patterns are frequently used as calibration objects. The accuracy of checkerboard detection depends on the effectiveness of binary morphological processing of the image. Currently, most traditional binary morphological algorithms are based on detection in ideal environments and cannot handle the various interferences that occur in real-world situations. Please refer to [link / reference needed] for details. Figure 1 .

[0070] Figure 1 This is a schematic diagram illustrating the image binarization processing effects of different algorithms provided in the embodiments of this application.

[0071] Figure 1 In this context, (A) is the initial image captured by the camera of the camera-calibrated scene with a checkerboard pattern. For example... Figure 1 As shown in (A), in the initial image, the image area containing the checkerboard 11 is connected to the image area containing the wall 12. Since the inherent color of an object changes not only due to the influence of the light source color but also due to the surrounding color environment, in the initial image, the area where the wall 12 is located and connected to the black squares of the checkerboard will be slightly darker than other areas of the wall (i.e., the grayscale value of the pixels in that area will be smaller). In simpler terms, the wall will be affected by the black squares of the checkerboard 11, resulting in a shadow. Figure 1 The shadow 121 shown in (A) is shown in the image. Furthermore, if the paint used during the production of the checkerboard 11 is not matte, when photographing the checkerboard 11, due to the reflection of light by the checkerboard, the imaged area of ​​the checkerboard in the initial image may contain partially reflective areas, for example... Figure 1The reflection region 111 shown in (A) in FIG. 11 is originally supposed to be black, but appears gray or even white due to the reflection effect of the checkerboard when the image is taken.

[0072] Figure 1 (B) in FIG. 11 is an image obtained by performing binaryzation processing on the initial image by using an adaptive local mean threshold method (hereinafter referred to as image P1). As can be known from the foregoing description, the adaptive local mean threshold method can better process an image with large light-dark differences, and thus is suitable for an image with a reflection region. For example, Figure 1 As shown in (B) in FIG. 11, when the initial image is binaryzated by using the adaptive local mean threshold method, the lower half of the checkerboard 11 is processed well, the boundary of the checkerboard is clear, and the positions of the corner points are clear. However, because of the shadow 121 on the wall 12 in the initial image, the upper boundary (black grid part) of the checkerboard 11 is connected with the black part 122 formed by the shadow of the wall. Therefore, in the image P1 obtained based on the initial image, the upper boundary of the checkerboard 11 is not clear enough.

[0073] Figure 1 (C) in FIG. 11 is an image obtained by performing binaryzation processing on the initial image by using the OTSU method (hereinafter referred to as image P2). As can be known from the foregoing description, the OTSU algorithm is prone to erroneous binaryzation segmentation for an image with uneven illumination, and thus the processing effect of the OTSU algorithm on a reflection region in an image is not ideal, but the processing effect of the OTSU algorithm on a shadow region affected by the environment color in an image is good. Therefore, as shown in (C) in FIG. 11, when the initial image is binaryzated by using the OTSU algorithm, the upper half of the checkerboard 11 is processed well, the boundary of the checkerboard is clear, and the positions of the corner points are clear. However, because of the reflection region in the lower half of the initial image, in the image P1 obtained based on the initial image, there are a large number of white contaminations in the lower half of the checkerboard 11, that is, Figure 1 As shown in (C) in FIG. 11, when the initial image is binaryzated by using the OTSU algorithm, the upper half of the checkerboard 11 is processed well, the boundary of the checkerboard is clear, and the positions of the corner points are clear. However, because of the reflection region in the lower half of the initial image, in the image P1 obtained based on the initial image, there are a large number of white contaminations in the lower half of the checkerboard 11, that is, Figure 1 The white noise point group 112 shown in (C) in FIG. 11.

[0074] As can be known from the foregoing description, when the original conventional binaryzation algorithm is used to process the above problems, it can be difficult to identify and suppress the highlight region, or it can be unable to separate the wall shadow from the checkerboard region and miss the checkerboard information, and thus the detection result of the checkerboard can be inaccurate.

[0075] In view of the defects of the original conventional binaryzation algorithm, an image processing method is provided in the embodiments of the present application. The method uses different algorithms to process different regions of an image according to the characteristics of the image, so that the electronic device can more accurately detect the specific position of the checkerboard in the image, and better calibrate the camera. For details, please refer to Figure 2 .

[0076] Figure 2 A flow chart of an image processing method provided by an embodiment of the present application. As shown in the figure, the method includes but is not limited to the following steps: Figure 2

[0077] 201. The electronic device splits the initial image to obtain an upper region image and a lower region image.

[0078] Specifically, the electronic device can be a mobile phone, a pad, a computer (such as a notebook computer, a palm computer, etc.) with data transceiving function, a mobile internet device (MID), a terminal in industrial control, a terminal device in 5G network, or a terminal device in future evolved public land mobile network (PLMN), etc. Optionally, when the electronic device is a vehicle-mounted device in a vehicle, the vehicle can be a general vehicle, or a special vehicle (including but not limited to a police car, a tractor, etc.), or a rescue vehicle (including but not limited to an ambulance, a fire engine, a rescue vehicle, etc.). It can be understood that the specific form of the terminal device is not limited in the present application. Any device that can communicate with a roadside device, or a vehicle, or a vehicle management platform, etc. falls within the protection scope of the terminal device. It can be understood that the specific form of the electronic device is not limited in the present application.

[0079] The initial image is an image captured by the electronic device, and the initial image contains a checkerboard. Specifically, the initial image can be an image containing a checkerboard captured by the electronic device for calibrating the camera of the electronic device. The format of the initial image can be any one of JPEG, TIFF, RAW, BMP, GIF, PNG, and can also be other formats, which are not limited in the present application.

[0080] The electronic device divides the initial image into two images according to the specific display position of the checkerboard in the initial image. The upper region image corresponds to the image in the upper half region of the initial image, and the lower region image corresponds to the image in the lower half region of the initial image. For details, please refer to Figure 3 In Figure 3 , the image 301 can be the initial image, the image 302 can be the upper region image, and the image 303 can be the lower region image.

[0081] ​It should be noted that the initial image, the upper region image and the lower region image are not binary images, and the partial checkerboard images included in the upper region image and the lower region image can be spliced and fused into a complete checkerboard image in the initial image.

[0082] 202. The electronic device processes the upper region image by using the Otsu method to obtain a first image.

[0083] In combination with the foregoing related description of Figure 1 and the image 301 in Figure 3 , it can be known that, due to the relative position relationship between the wall surface and the ground, when the image containing the checkerboard is used for camera calibration of the electronic device, the wall surface and the wall shadow are generally displayed in the upper half of the initial image, and the ground is generally displayed in the lower half of the initial image. Therefore, the upper half of the initial image, i.e., the upper region image, is likely to have a partial shadow area affected by the environment color of the black squares of the checkerboard, and the Otsu method has good processing effect on the shadow area affected by the environment color in the image, which can effectively separate the object shadow and the region of interest, but has poor processing effect on the reflective area (with a large amount of white pollution), so the binaryzation effect of the Otsu method on the upper region of the initial image, i.e., the upper region image, is good.

[0084] Therefore, in the method, the electronic device can perform binaryzation processing on the upper region image by using the Otsu method to obtain the first image, which is a binary image; correspondingly, the first image contains the binary image of part of the checkerboard in the initial image. For details, refer to the image 304 and the image 302 in Figure 3 . It can be seen from the comparison between the image 302 and the image 304 that after the image in the first region (i.e., the image 302) is processed by using the Otsu method for binaryzation, the binary image (i.e., the image 304) obtained has clear boundaries of the checkerboard and clear positions of the corners, and has good detection effect on the checkerboard.

[0085] 203. The electronic device processes the lower region image by using the adaptive local mean method to obtain a second image.

[0086] In combination with the foregoing related description, it can be known that the ground is generally displayed in the lower half of the initial image. Therefore, the lower half of the initial image and the lower region image are likely to have a partial reflective area due to the non-matt paint of the ground. The adaptive local mean threshold method can better process images with large light-dark differences, and therefore is suitable for images with reflective areas, so the binaryzation effect of the adaptive local mean threshold method on the lower region of the initial image, i.e., the image in the lower region image, is good.

[0087] Therefore, in the method, the electronic device can perform binarization processing on the lower region image by using the adaptive local mean threshold method to obtain the second image; the second image is a binarization image; correspondingly, the second image contains a binarization image of part of the checkerboards in the initial image; in addition, the binarization image of the checkerboards in the first image plus the binarization image of the checkerboards in the second image can correspond to a binarization image of complete checkerboards in the initial image. For details, refer to images 303 and 305 in Figure 3 As can be seen from the comparison between images 303 and 305, after the adaptive local mean threshold method is used to perform binarization processing on the lower region image (i.e., image 302), the binarization image (i.e., image 304) obtained has clear boundaries of the checkerboards and clear positions of the corners, and the detection effect on the checkerboards is good.

[0088] In addition, as can be seen from image 305, although the adaptive local mean threshold method has a good processing effect on the initial image, a small amount of white noise points still exist in part of the black squares in the checkerboards in the second image obtained by the processing.

[0089] To avoid the generation of the white noise points, in an optional embodiment, before the adaptive local mean method is used to detect the second region of the initial image, the electronic device can reduce the gray values of all the pixel points in a first set of pixel points in the image corresponding to the second region by the first threshold value, and reduce the gray values of all the pixel points in a second set of pixel points to 0; the first set of pixel points is the pixel points in the second region whose gray values are greater than or equal to the first threshold value, and the second set of pixel points is the pixel points in the second region whose gray values are less than the first threshold value. That is, the overall gray values of the image corresponding to the second region are reduced by a certain value (the certain value is the first threshold value, and the certain value is an integer in [0, 255]), so as to suppress the highlight region to prevent a large amount of white noise points from being generated in the image of the highlight region after the binarization processing. It can be understood that the greater the certain value, the deeper the suppression. It can be understood that if the current pixel value of the image is less than the certain value, the pixel value at this position is set to 0. Optionally, when the gray values of the image are suppressed (i.e., the process of uniformly reducing the gray values of the image), the strength of the highlight can be determined first; when the highlight is strong and weak, the local pixel mean value is used to determine the strength of the highlight before the processing, and only the strong highlight (i.e., the highlight is greater than a certain threshold value, and the threshold value can be set according to the specific calibration scene) is processed. Finally, the adaptive local mean threshold method is used to perform image binarization to obtain an ideal effect.

[0090] 204、The electronic device fuses the first image and the second image to obtain a target image.

[0091] After obtaining the first image and the second image, the electronic device can fuse the first image and the second image, that is, combine and splice, to obtain the target image. For details, refer to the image 306 in Figure 3 From the image 306, it can be seen that the upper and lower boundaries of the checkerboard in the image 306 are clear, and the positions of the respective corner points are clear, and there is no large-area white noise. That is, the method can identify and suppress the highlight region, and separate the wall shadow and the checkerboard region, which can improve the accuracy of the detection result of the checkerboard.

[0092] It should be understood that in the traditional Otsu algorithm, the entire gray interval [0, 255] is usually traversed with a step size of 1 to accurately find the optimal threshold, that is, 256 system operation processes are required to obtain the optimal threshold, and therefore a large amount of computing power and time is required. This undoubtedly increases the power consumption of the device, and the efficiency of the algorithm is also low. Therefore, based on the image processing method provided in the present application and the traditional Otsu algorithm, the present application further provides an optimized Otsu method, which can more quickly determine the optimal threshold by improving the traversal step size and traversal interval of the traditional Otsu method, thereby reducing the power consumption of the electronic device. Specifically, the optimized Otsu method provided in the present application can be used to implement step 202 in the foregoing image processing method.

[0093] As shown in Figure 4 The optimized Otsu method provided in the present application can specifically include but is not limited to the following steps:

[0094] 401、The electronic device traverses the first interval using a first step size to obtain a suboptimal threshold.

[0095] Specifically, the electronic device can be a mobile phone, a tablet computer, a computer (such as a notebook computer, a palm computer, etc.) with data transceiving function, a mobile internet device (MID), a terminal in industrial control, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc. Specifically, the electronic device can be the electronic device in Figure 2 It should be understood that the specific form of the electronic device is not limited in the present application.

[0096] In the method, the electronic device can traverse the first interval according to the first step length (the value of the first step length is greater than 1) to determine the suboptimal threshold temp, so as to find the optimal threshold in the field centered on the suboptimal threshold in the subsequent process, thereby reducing the number of calculations of the device, reducing the power consumption of the device, and improving the efficiency of the algorithm

[0097] Here, it is assumed that the first interval is [min, max], and it can be understood that, generally, the optimal threshold generally does not fall in a relatively extreme interval such as [0, 20] or [230, 255], and therefore it is not necessary to calculate the gray value in these intervals during traversal. Therefore, optionally, the min can be greater than 0, and the max can be less than 255; by reducing the traversal interval to the first interval [min, max], the number of calculations of the device can be further reduced, the power consumption of the device can be reduced, and the efficiency of the algorithm can be improved. Specifically, the specific values of the min and the max can be determined according to experience or can be determined according to the brightness of the specific scene of the camera calibration, and the present application does not limit this.

[0098] The value of the first step length is an odd number greater than 1. Specifically, the value of the first step length is the nearest odd integer to the arithmetic square root of (max-min); for example, it is assumed that the values of the max and the min are 225 and 25 respectively, that is, the value of (max-min) is 200, and the arithmetic square root of 200 is about 14.14, and the value of the first step length can be 15. By setting the value of the first step length to the nearest odd integer to the arithmetic square root of (max-min), it can be prevented that the step length is too long to cause the suboptimal threshold to be not referential.

[0099] 402. The electronic device determines a second interval based on the suboptimal threshold.

[0100] Here, it is assumed that the suboptimal threshold is temp and the value of the first step length is d, and the electronic device can set the second interval as [temp-(d-1) / 2, temp+(d-1) / 2] based on the suboptimal threshold temp.

[0101] It should be understood that the optimal threshold value is generally in a neighborhood centered at the suboptimal threshold value with a radius of 1 / 2 of the first step length. If the optimal threshold value is not in this neighborhood, then the suboptimal threshold value is generally not temp. For example, if the suboptimal threshold value obtained in the first traversal is temp, but the actual optimal threshold value is not in the interval [temp-(d-1) / 2, temp+(d-1) / 2], but is temp+(d-1) / 2+1, then the suboptimal threshold value determined in the first traversal should not be temp, but should be (temp+d). Therefore, in the present embodiment, when the second traversal is performed, the second interval is set to [temp-(d-1) / 2, temp+(d-1) / 2], which ensures that the optimal threshold value is accurately obtained based on the suboptimal threshold value.

[0102] 403. The electronic device traverses the second interval using a second step length to obtain an optimal threshold value.

[0103] The value of the second step length is less than the value of the first step length. Specifically, the value of the second step length can be 1, which prevents the optimal threshold value from being crossed due to a too large step length, and ensures that the optimal threshold value can be accurately determined in the neighborhood of the suboptimal threshold value.

[0104] 404. The electronic device processes the to-be-processed image based on the optimal threshold value to obtain a binary image.

[0105] Specifically, the to-be-processed image can be the image corresponding to the first region described above.

[0106] The electronic device can adjust the gray value of each pixel in the to-be-processed image to 0 or 255 based on the optimal threshold value. Specifically, the gray value of a pixel with a gray value greater than the optimal threshold value is adjusted to 255, and the gray value of a pixel with a gray value less than or equal to the optimal threshold value is adjusted to 0.

[0107] It should be understood that in an actual camera calibration scenario, the object can be very complex, which means that the checkerboard in the first image, the second image, and the target image can be blocked by other objects (for example, when calibrating the camera of a vehicle, the checkerboard in the captured image is likely to be blocked by the tire of the vehicle). Such blocking undoubtedly reduces the detection accuracy of the checkerboard.

[0108] Therefore, based on the above defects, the present application also provides a method for dividing a checkerboard. The method determines two straight lines with the same slope as the slope of the boundary of the blocked part of the checkerboard by using the inner corner points of the checkerboard and the slope of the boundary of the blocked part of the checkerboard, cuts off the blocked part of the checkerboard, and obtains a checkerboard with a clear boundary. For example, Figure 5As shown, the method may include, but is not limited to, the following steps:

[0109] 501. When a checkerboard pattern in a binarized image is occluded, the electronic device determines the slope of the boundary of the occluded portion of the checkerboard pattern in the binarized image based on the first and second interior corner points of the checkerboard pattern in the binarized image.

[0110] Specifically, the aforementioned electronic device can be a mobile phone, tablet computer, computer with data transceiver capabilities (such as a laptop or PDA), mobile internet device (MID), terminal in industrial control, terminal device in a 5G network, or terminal device in a future public land mobile network (PLMN), etc. Figure 2 or Figure 4 The electronic devices mentioned above. It is understood that this application does not limit the specific form of the aforementioned electronic devices.

[0111] The aforementioned binarized image can be the aforementioned pair Figure 2 The first or second image mentioned in the related description can also be a third image obtained by combining and stitching the first and second images to obtain the target image. In this method, the electronic device can receive user instructions to indicate whether the checkerboard pattern in the binarized image is occluded. Specifically, whether the checkerboard pattern in the third image is occluded can be determined by a detection personnel, and the determination result can be input into the electronic device responsible for executing this method; or, the electronic device can detect whether the checkerboard pattern is occluded based on a corresponding visual algorithm.

[0112] Furthermore, the lines containing the first and second interior corner points are parallel to the lines in the binarized image (i.e., the slope of the lines containing the first and second interior corner points is the same as the slope of the boundary of the occluded checkerboard grid). Specifically, the two interior corner points (i.e., the first and second interior corner points) can be the two interior corner points closest to the occluded portion and located within the unoccluded portion. It should be noted that when the occluded portion of the checkerboard grid involves multiple boundaries of the checkerboard grid, the number of the first and second interior corner points can also be multiple.

[0113] 502. The electronic device determines the first straight line and the second straight line based on the slope of the boundary of the checkerboard grid of the occluded part in the above-mentioned binarized image.

[0114] The first straight line and the second straight line are used to divide the binary image into an image to be used and an interference image, the interference image contains the area of the checkerboard that is blocked in the binary image, which can be discarded in the subsequent camera calibration process, and the image to be used will continue to be used in the subsequent camera calibration process.

[0115] Figure 6 The method shown is used to divide the third image. Figure 5 The specific process of dividing the third image by the method shown is shown.

[0116] As shown in Figure 6 , in Figure 6 , the image 601 is the third image obtained by splicing the first image and the second image in the foregoing Figure 2 , which also corresponds to the binary image in the foregoing Figure 5 . As can be seen from the image 601, the lower boundary of the checkerboard in the image 601 (i.e. the position of the straight line L1) is connected with the interference area 6011 due to the blocking of the car tire, so that the lower boundary of the checkerboard is not clear.

[0117] Therefore, the electronic device can determine two inner corner points 6012 and 6013 (the inner corner points 6012 and 6013 are the first inner corner point and the second inner corner point in the foregoing description), and determine a straight line L2 based on the two inner corner points (the straight line L2 is parallel to the straight line L1, that is, the slope of the straight line L2 is the same as the slope of the straight line L1, and two straight lines parallel to the straight line L2 (i.e. the slope is the same as the slope of the straight line L2 and L1) are obtained through the straight line L2, that is Figure 6 the straight line L3 and the straight line L4 in the foregoing description).

[0118] Here, it is assumed that the coordinates of the inner corner point 6012 and the inner corner point 6013 in the image are (a, b) and (c, d) respectively, and specifically, the electronic device can first calculate the slope k of the straight line L2 as (b-d) / (a-c), which can be understood that the slope k is also the slope of the straight line L1. Then, the electronic device can select a point near the midpoint of the original checkerboard, for example Figure 6 the inner corner point 6014 in the foregoing, whose coordinates are (e, f), and solve the analytical expressions of the two parallel straight lines y=k(x-e)+f, y=k(x-e)+f+g based on the slope of the straight line L2, where y=k(x-e)+f is the analytical expression of the straight line L3, y=k(x-e)+f+g is the analytical expression of the straight line L4, and |g| is the line width used in the division. Finally, the electronic device can set the pixel values in the area surrounded by the two parallel straight lines L3 and L4 to 0. From Figure 6As can be seen, the white area formed by straight lines L3 and L4 divides image 601 into image 602 and image 603, where image 602 is the target image and image 603 is the interference image (in some embodiments of this application, it may be referred to as the fourth image).

[0119] It should be understood that, according to the principle of checkerboard corner detection, the accuracy of the detection result depends only on the detection precision of the inner corner points. Understandably, in the image 602 within the dashed box above line L3, the checkerboard boundary is complete and clear, and the position of the inner corner points is also clear enough, so it can be used in the subsequent camera calibration process. However, the image 603 within the dashed box below line L4 should contain the interference region 6011 in the original image (i.e., image 601) and be discarded, and will no longer be used in the subsequent camera calibration process.

[0120] The following is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Please refer to [link / reference]. Figure 7 .like Figure 7 As shown, Figure 7 The storage device in it can execute Figure 2 The process of image processing methods in China Figure 4 The process of optimizing Otsu's algorithm and Figure 5 The process of the image segmentation method, the apparatus includes:

[0121] The splitting unit 701 is used to split the initial image to obtain an upper region image and a lower region image, wherein the upper region image includes the upper boundary of the checkerboard grid and the lower region image includes the lower boundary of the checkerboard grid; the first processing unit 702 is used to process the upper region image using the Otsu method to obtain a first image; the second processing unit 703 is used to process the lower region image using an adaptive local mean thresholding method to obtain a second image; and the fusion unit 704 is used to fuse the first image and the second image to obtain a target image.

[0122] In one possible implementation, the apparatus further includes a segmentation unit 705, wherein the fusion unit 704 is specifically configured to: fuse the first image and the second image to obtain a third image; the segmentation unit 705 is configured to, when the checkerboard pattern in the third image is occluded, determine the slope of the boundary of the occluded portion of the checkerboard pattern in the third image based on the first and second interior corner points of the checkerboard pattern in the third image; determine a first straight line and a second straight line according to the slope of the boundary of the occluded portion of the checkerboard pattern in the third image, wherein the first straight line and the second straight line are used to divide the third image into the target image and a fourth image, wherein the fourth image includes the area of ​​the checkerboard pattern occluded in the third image.

[0123] In a possible implementation, the first processing unit 702 is specifically configured to: traverse a first interval using a first step length to obtain a suboptimal threshold, the first interval being [min, max], the min being greater than 0, and the max being less than 255; determine a second interval based on the suboptimal threshold, the second interval containing the suboptimal threshold, and the second interval containing the first interval; traverse the second interval using a second step length to obtain an optimal threshold, the second step length being smaller than the first step length; and perform binarization processing on the image in the first region based on the optimal threshold to obtain the first image.

[0124] In a possible implementation, the first interval is [min, max], the min is greater than 0, the max is less than 255, the value of the first step length is the nearest odd integer to the arithmetic square root of (max-min), and the value of the second step length is 1.

[0125] In a possible implementation, the suboptimal threshold is temp, the value of the first step length is d, and the second interval is [temp-(d-1) / 2, temp+(d-1) / 2].

[0126] In a possible implementation, the first interval is [min, max], and the first processing unit 701 is specifically configured to: in a case where the value of the first step length cannot divide (max-min) evenly, adjust the upper limit of the first interval to obtain a third interval [min, max+m], the m being smaller than the value of the first step length, and (max+m-min) being an integer multiple of the value of the first step length; and traverse the third interval using the first step length to obtain the suboptimal threshold.

[0127] In a possible implementation, the pixels in the second region include a first set of pixels and a second set of pixels, a gray value of any pixel in the first set of pixels being greater than or equal to a first threshold, and a gray value of any pixel in the second set of pixels being less than the first threshold; and the second processing unit 703 is further configured to: reduce the gray value of each pixel in the first set of pixels by the first threshold, and reduce the gray value of each pixel in the second set of pixels to 0.

[0128] It should be understood that the division of the above image processing apparatus into various units is merely a logical functional division, and in actual implementation, all or part of the units can be integrated into one physical entity, or can be physically separated. For example, the above various units can be separate processing elements, or can be integrated into the same chip, and in addition, the functions of the above various units can also be stored in the storage unit of the controller in the form of program code, and called and executed by a processing element of the processor. In addition, the various units can be integrated together or implemented independently. The processing element here can be an integrated circuit chip with signal processing capability. In the implementation process, the steps of the method or the above various units can be completed by the integrated logic circuit of the hardware in the processor element or the instructions in the form of software. The processing element can be a general-purpose processor, such as a CPU, and can also be one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), etc.

[0129] Figure 8 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. As shown in the figure, the electronic device 80 includes a processor 801, a memory 802, and a communication interface 803; the above-mentioned processor 801, memory 802, and communication interface 803 are connected to each other through a bus 804. Specifically, the electronic device 80 can be the electronic device in the foregoing description. Figure 8

[0130] The memory 802 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a compact disc read-only memory (CD ROM), and the above-mentioned memory 802 is used for related instructions and data. The communication interface 803 is used for receiving and sending data. Specifically, the communication interface 803 can be used to receive instructions input by a user, such as instructions indicating whether there is an occlusion in the checkerboard in the image.

[0131] ​The processor 801 can be one or more central processing units (CPUs), in the case of the processor 801 being a CPU, the CPU can be a single core processor or a multiple core processor. Specifically, the processor 801 can implement the functions of the splitting unit 701, the first processing unit 702, the second processing unit 703, the fusing unit 704, and the dividing unit 705 in the above embodiment. Figure 7

[0132] In an embodiment of the present application, another computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the following steps: splitting an initial image to obtain an upper region image and a lower region image, the upper region image including an upper boundary of a checkerboard, and the lower region image including a lower boundary of the checkerboard; processing a first region of the initial upper region image using the Otsu method to obtain a first image, the first region including the upper boundary of the checkerboard; processing a second region of the initial lower region image using an adaptive local mean threshold method to obtain a second image, the second region including the lower boundary of the checkerboard; fusing the first image and the second image to obtain a target image, the target image including a binary image of the checkerboard

[0133] In the above embodiments, according to the context, the term “when” can be interpreted to mean “if” or “after” or “in response to determining” or “in response to detecting”. Similarly, according to the context, the phrase “on determining” or “if detecting (a stated condition or event)” can be interpreted to mean “if determining” or “in response to determining” or “on detecting (a stated condition or event)” or “in response to detecting (a stated condition or event)”.

[0134] ​In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk) and the like.

[0135] Those of ordinary skill in the art understand that all or part of the processes in the above embodiments can be implemented by a computer program to instruct the relevant hardware, which can be stored in a computer readable storage medium. The program can include the processes of the above method embodiments when executed. The aforementioned storage medium includes ROM or random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

Claims

1. An image processing method, characterized by, The method comprises the following steps: splitting an initial image to obtain an upper region image and a lower region image, the upper region image comprising an upper boundary of a checkerboard, and the lower region image comprising a lower boundary of the checkerboard; processing the upper region image by using the Otsu method to obtain a first image; reducing the gray value of all pixel points in a first pixel point set by a first threshold value, and reducing the gray value of all pixel points in a second pixel point set to 0, the pixel points in the lower region comprising the first pixel point set and the second pixel point set, the gray value of any pixel point in the first pixel point set being greater than or equal to the first threshold value, and the gray value of any pixel point in the second pixel point set being less than the first threshold value; processing the lower region image by using an adaptive local mean threshold value method to obtain a second image; fusing the first image and the second image to obtain a third image; when the checkerboard in the third image is blocked, determining the slope of the boundary of the checkerboard of the blocked part in the third image based on a first internal corner point and a second internal corner point of the checkerboard in the third image; determining a first straight line and a second straight line according to the slope of the boundary of the checkerboard of the blocked part in the third image, the first straight line and the second straight line being used to divide the third image into a target image and a fourth image, and the fourth image comprising the region in which the checkerboard is blocked in the third image.

2. The method of claim 1, wherein, The processing of the upper region image by using the Otsu method to obtain a first image comprises the following steps: traversing a first interval by using a first step length to obtain a suboptimal threshold value, the first interval being [min, max], the min being greater than 0, and the max being less than 255; determining a second interval based on the suboptimal threshold value, the second interval comprising the suboptimal threshold value, and the second interval being included in the first interval; traversing the second interval by using a second step length to obtain an optimal threshold value, the second step length being smaller than the first step length; performing binarization processing on the upper region image based on the optimal threshold value to obtain the first image.

3. The method of claim 2, wherein, The first interval is [min, max], the min is greater than 0, the max is less than 255, the value of the first step length is the nearest odd integer to the arithmetic square root of (max-min), and the value of the second step length is 1.

4. The method of claim 2, wherein, The suboptimal threshold value is temp, the value of the first step length is d, and the second interval is [temp-(d-1) / 2, temp+(d-1) / 2].

5. The method of claim 2, wherein, The first interval is [min, max], and the determination of the suboptimal threshold value based on the first step length and the first interval comprises the following steps: in the case that the value of the first step length cannot divide (max-min) evenly, adjusting the upper limit of the first interval to obtain a third interval [min, max+m], the m being smaller than the value of the first step length, and (max+m-min) being an integer multiple of the value of the first step length; traversing the third interval by using the first step length to obtain the suboptimal threshold value.

6. An image processing apparatus characterized by comprising: The method comprises the following steps: The splitting unit is configured to split an initial image to obtain an upper region image and a lower region image, the upper region image including an upper boundary of the checkerboard, and the lower region image including a lower boundary of the checkerboard. The first processing unit is configured to process the upper region image by using the Otsu method to obtain a first image. The second processing unit is configured to reduce the gray value of all pixel points in a first pixel point set by a first threshold value, and reduce the gray value of all pixel points in a second pixel point set to 0, the pixel points in the lower region including the first pixel point set and the second pixel point set, the gray value of any pixel point in the first pixel point set being greater than or equal to the first threshold value, and the gray value of any pixel point in the second pixel point set being less than the first threshold value. The second processing unit is further configured to process the lower region image by using an adaptive local mean threshold method to obtain a second image. The fusion unit is configured to fuse the first image and the second image to obtain a third image. The segmentation unit is configured to, when the checkerboard in the third image is occluded, determine a slope of a boundary of the checkerboard of the occluded part in the third image based on a first internal corner point and a second internal corner point of the checkerboard in the third image, and determine a first straight line and a second straight line according to the slope of the boundary of the checkerboard of the occluded part in the third image, the first straight line and the second straight line being used to divide the third image into a target image and a fourth image, the fourth image including a region in which the checkerboard in the third image is occluded.

7. A vehicle characterized by comprising: The vehicle includes at least one processor, a memory and a display screen; The memory is coupled with the processor, and the memory is configured to store computer program code including computer instructions, and the processor is configured to invoke the computer instructions to enable the vehicle to perform the method of any one of claims 1-5.

8. A computer-readable storage medium comprising instructions, characterized in that, The instructions, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-5.

Images