Image processing method, device and computer readable storage medium

Abstract

The present invention relates to the field of digital image processing technology, and in particular to an image processing method, device and computer-readable storage medium, wherein the method comprises: determining the pixel coordinates of each target connected area in the original image, and determining the convex hull point according to the pixel coordinates; determining the first vector according to the coordinates of the first convex hull point and the second convex hull point, and determining the second vector according to the first convex hull point and the third convex hull point; determining the card corner point according to the angle between the first vector and the second vector; determining the affine transformation inverse matrix of the original image based on the coordinates of the card corner point, and generating a target image based on the affine transformation inverse matrix and the original image. By extracting the convex hull of the original image, filtering the card corner points from the convex hull, and performing an inverse affine transformation on the original image based on the card corner points, a target image with the affine transformation eliminated is generated, thereby improving the image processing efficiency and solving the problem of how to restore the calibrated card image that has undergone affine transformation.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and in particular to an image processing method, device and computer-readable storage medium. Background Art

[0002] Camera calibration is a common method in image processing technology. Camera calibration is usually used to establish the correspondence between the coordinate points of an object from the three-dimensional world to the imaging plane.

[0003] During the camera calibration process, since there may be a certain angle between the camera and the image card, the image card will undergo affine transformation during the acquisition process. That is, the image card, which was originally a standard rectangle, becomes an irregular quadrilateral in the image captured by the camera. This transformation will affect the accuracy of subsequent camera calibration.

[0004] The traditional affine transformation restoration method is to match each point of the calibration card image with the actual point on the calibration card one by one, and then perform an inverse affine transformation on the image to restore an image that is consistent with the shape of the actual calibration card. In actual industrial applications, this method has the problems of cumbersome calculation process and high time consumption.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present invention is to provide an image processing method, aiming to solve the problem of how to restore a calibration chart image that has undergone an affine transformation.

[0007] To achieve the above object, the present invention provides an image processing method, which comprises:

[0008] Determine the pixel coordinates of each target connected area in the original image, and determine the convex hull points according to the pixel coordinates;

[0009] Determine a first vector according to coordinates of the first convex hull point and the second convex hull point, and determine a second vector according to the first convex hull point and the third convex hull point, wherein the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point;

[0010] Determine a corner point of the chart according to an angle between the first vector and the second vector;

[0011] Based on the coordinates of the corner points of the chart, an inverse affine transformation matrix of the original image is determined, and a target image is generated according to the inverse affine transformation matrix and the original image.

[0012] Optionally, determining a convex hull point according to the pixel coordinates includes:

[0013] Determine the pixel coordinates corresponding to the pixel points corresponding to the target connected area;

[0014] Determine a coordinate mean corresponding to the pixel coordinates in the target connected area, and use the coordinate mean as the center coordinate corresponding to the target connected area;

[0015] The pixel point corresponding to the center coordinates of the target connected area is used as the convex hull point.

[0016] Optionally, the step of determining a corner point of the chart according to an angle between the first vector and the second vector comprises:

[0017] Get the preset angle threshold;

[0018] Determine the angle of the convex hull point according to the angle between the first vector and the second vector;

[0019] The card corner point is determined according to the angle of the convex hull point and the preset angle threshold.

[0020] Optionally, after the step of determining the corner point of the chart according to the angle between the first vector and the second vector, the method further comprises:

[0021] Determine the sorting rule of the corner points of the chart according to the coordinates of the corner points of the chart;

[0022] The corner points of the image card are sorted according to the image card corner point sorting rule.

[0023] Optionally, before the step of determining the inverse affine transformation matrix of the original image based on the coordinates of the corner points of the chart, and generating the target image according to the inverse affine transformation matrix and the original image, the method further includes:

[0024] Acquire the coordinates of the calibration chart, where the coordinates of the calibration chart are the coordinates of the calibration chart in the actual scene;

[0025] Calculate the homogeneous coordinate matrix corresponding to the coordinates of the corner points of the card;

[0026] Normalizing the homogeneous coordinate matrix to obtain the horizontal axis coordinate equation and the vertical axis coordinate equation of the corner point of the chart;

[0027] The affine transformation matrix is ​​determined according to the coordinates of each corner point of the chart, the coordinates of the calibration chart, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each corner point of the chart.

[0028] Optionally, the step of determining the inverse affine transformation matrix of the original image based on the coordinates of the corner points of the chart, and generating the target image according to the inverse affine transformation matrix and the original image comprises:

[0029] Determine the affine transformation inverse matrix corresponding to the affine transformation matrix through inverse matrix transformation;

[0030] Determine the coordinates of the corner points of the target chart according to the inverse affine transformation matrix and the coordinates of the corner points of the chart, wherein the coordinates of the corner points of the target chart are the mapping coordinates of the coordinates of the calibration chart in the original image;

[0031] The target image is determined based on the coordinates of the corner points of the target card.

[0032] Optionally, determining the pixel coordinates of each target connected area in the original image includes:

[0033] Obtaining a pixel value of each pixel in the original image and obtaining a preset pixel threshold;

[0034] Determining the target connected area according to the pixel value of each pixel point and the preset pixel threshold;

[0035] The pixel coordinates of the target connected area are obtained.

[0036] Optionally, before the step of determining the pixel coordinates of each target connected area in the original image and determining the convex hull points according to the pixel coordinates, the step further includes:

[0037] Binarizing the collected calibration card image to obtain a binary image corresponding to the calibration card image;

[0038] Extracting coordinates, aspect ratio and / or area of ​​connected regions in the binary image;

[0039] Obtaining reference coordinates, reference aspect ratio and / or reference area;

[0040] The connected regions in the binary image that do not match the reference coordinates, the reference aspect ratio and / or the reference area are regarded as connected regions with irregular shapes;

[0041] The connected regions with irregular shapes are eliminated to obtain the original image.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides an image processing device, which includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, and the image processing program implements the steps of the image processing method described above when executed by the processor.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, which includes a memory, a processor, and an image processing program stored on the memory and executable on the processor, and the image processing program implements the steps of the image processing method described above when executed by the processor.

[0044] The embodiment of the present invention provides an image processing method, device and computer-readable storage medium, the method comprising: determining the pixel coordinates of each target connected area in the original image, and determining the convex hull point according to the pixel coordinates; determining the first vector according to the coordinates of the first convex hull point and the second convex hull point, and determining the second vector according to the first convex hull point and the third convex hull point, wherein the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point; determining the corner point of the chart according to the angle between the first vector and the second vector; determining the inverse affine transformation matrix of the original image based on the coordinates of the corner point of the chart, and generating the target image according to the inverse affine transformation matrix and the original image. By extracting the convex hull of the target connected area in the original image, filtering out the required corner points of the chart from the convex hull, performing an inverse affine transformation on the corner points of the chart, obtaining the target corner points of the chart with the affine transformation eliminated, and finally generating the target image with the affine transformation eliminated according to the coordinates of the corner points of the target chart, the image processing efficiency is improved, and the problem of how to restore the calibrated chart image that has undergone affine transformation is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the hardware architecture of an image processing device involved in an embodiment of the present invention;

[0046] Figure 2 is a schematic flow chart of a first embodiment of an image processing method of the present invention;

[0047] Figure 3 A schematic diagram of an original image in a specific embodiment of the present invention;

[0048] Figure 4 A schematic diagram of extracting the convex hull of a calibration chart image in a specific embodiment of the present invention;

[0049] Figure 5 A schematic diagram of a target image in a specific embodiment of the present invention;

[0050] Figure 6 FIG. 4 is a flow chart of a second embodiment of an image processing method according to the present invention.

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the exemplary embodiments of the present invention are shown in the accompanying drawings of the present invention, and the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0053] As an implementation scheme, the image processing device can be as follows Figure 1 shown.

[0054] The embodiment of the present invention relates to an image processing device, which includes: a processor 101, such as a CPU, a memory 102, and a communication bus 103. The communication bus 103 is used to realize connection and communication between these components.

[0055] The memory 102 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Figure 1 As shown, the memory 102 as a computer-readable storage medium may include an image processing program; and the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0056] Determine the pixel coordinates of each target connected area in the original image, and determine the convex hull points according to the pixel coordinates;

[0057] Determine a first vector according to coordinates of the first convex hull point and the second convex hull point, and determine a second vector according to the first convex hull point and the third convex hull point, wherein the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point;

[0058] Determine a corner point of the chart according to an angle between the first vector and the second vector;

[0059] Based on the coordinates of the corner points of the chart, an inverse affine transformation matrix of the original image is determined, and a target image is generated according to the inverse affine transformation matrix and the original image.

[0060] In one embodiment, the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0061] Determine the pixel coordinates corresponding to the pixel points corresponding to the target connected area;

[0062] Determine a coordinate mean corresponding to the pixel coordinates in the target connected area, and use the coordinate mean as the center coordinate corresponding to the target connected area;

[0063] The pixel point corresponding to the center coordinates of the target connected area is used as the convex hull point.

[0064] In one embodiment, the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0065] Get the preset angle threshold;

[0066] Determine the angle of the convex hull point according to the angle between the first vector and the second vector;

[0067] The card corner point is determined according to the angle of the convex hull point and the preset angle threshold.

[0068] In one embodiment, the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0069] Determine the sorting rule of the corner points of the chart according to the coordinates of the corner points of the chart;

[0070] The corner points of the image card are sorted according to the image card corner point sorting rule.

[0071] In one embodiment, the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0072] Acquire the coordinates of the calibration chart, where the coordinates of the calibration chart are the coordinates of the calibration chart in the actual scene;

[0073] Calculate the homogeneous coordinate matrix corresponding to the coordinates of the corner points of the card;

[0074] Normalizing the homogeneous coordinate matrix to obtain the horizontal axis coordinate equation and the vertical axis coordinate equation of the corner point of the chart;

[0075] The affine transformation matrix is ​​determined according to the coordinates of each corner point of the chart, the coordinates of the calibration chart, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each corner point of the chart.

[0076] In one embodiment, the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0077] Determine the affine transformation inverse matrix corresponding to the affine transformation matrix through inverse matrix transformation;

[0078] Determine the coordinates of the corner points of the target chart according to the inverse affine transformation matrix and the coordinates of the corner points of the chart, wherein the coordinates of the corner points of the target chart are the mapping coordinates of the coordinates of the calibration chart in the original image;

[0079] The target image is determined based on the coordinates of the corner points of the target card.

[0080] In one embodiment, the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0081] Obtaining a pixel value of each pixel in the original image and obtaining a preset pixel threshold;

[0082] Determining the target connected area according to the pixel value of each pixel point and the preset pixel threshold;

[0083] The pixel coordinates of the target connected area are obtained.

[0084] In one embodiment, the processor 101 may be used to call the image processing program stored in the memory 102 and perform the following operations:

[0085] Binarizing the collected calibration card image to obtain a binary image corresponding to the calibration card image;

[0086] Extracting coordinates, aspect ratio and / or area of ​​connected regions in the binary image;

[0087] Obtaining reference coordinates, reference aspect ratio and / or reference area;

[0088] The connected regions in the binary image that do not match the reference coordinates, the reference aspect ratio and / or the reference area are regarded as connected regions with irregular shapes;

[0089] The connected regions with irregular shapes are eliminated to obtain the original image.

[0090] Based on the hardware architecture of the above-mentioned image processing device based on digital image processing technology, an embodiment of the image processing method of the present invention is proposed.

[0091] During the camera calibration process, it is necessary to identify the calibration card image on the image. However, due to a certain offset angle between the captured calibration card image and the actual calibration card, the card will undergo an affine transformation after shooting, causing the calibration card, which was originally a standard rectangle, to be deformed in the calibration card image. Therefore, it is necessary to perform batch affine transformation restoration on the captured images.

[0092] Reference Figure 2 In a first embodiment, the image processing method comprises the following steps:

[0093] Step S10, determining the pixel coordinates of each target connected area in the original image, and determining the convex hull points according to the pixel coordinates;

[0094] In this embodiment, the pixel coordinates of each target connected area in the original image are first determined, and the convex hull points are determined according to the pixel coordinates. A connected component generally refers to an image area composed of foreground pixels with the same pixel value and adjacent positions in an image. The original image is an image obtained by preprocessing the collected calibration card image and removing the interference area.

[0095] Reference Figure 3 , Figure 3 This is a schematic diagram of an original image in a specific implementation manner. The connected areas in the original image include white dots and black areas. For the machine, not only each white dot will be recognized as a connected area by the machine, but the black area will also be recognized as a connected area. However, in this embodiment, the part that needs to be processed is a dot matrix composed of 9*12 white dots, and the connected areas outside the dot matrix are not processed. Therefore, in this step, it is also necessary to select the connected area to be processed from the original image as the target connected area.

[0096] Since the original image is a binary image, the pixel values ​​in the image only contain two values: 255 (i.e., white) and 0 (black). Therefore, a pixel threshold can be set to select only the areas in the connected area that meet the pixel threshold as the target connected area. For example, the pixel threshold can be set to 255, and the areas in the connected area with a pixel value of 255 are selected as the target connected area. Figure 3 As shown, each white dot is a target connected area.

[0097] After determining the target connected area, call the relevant function of obtaining coordinates in the OpenCV library to obtain the pixel coordinates of each pixel in the target connected area, and determine the convex hull points in the original image based on the pixel coordinates. The convex hull points are points on the minimum convex hull formed by each target connected area in the original image. The convex hull (Convex Hull) is a concept of geometric figures, which is defined as a convex polygon formed by connecting the outermost points of the figure, which can contain all the points in the point set. The convex hull points are located at the outermost periphery of the convex polygon, and the overall shape of the calibration chart is usually a convex polygon (such as a rectangle). Therefore, in this embodiment, by determining the convex hull points, the machine can identify the boundary of the minimum convex polygon formed by all target connected areas, that is, the boundary of the calibration chart image.

[0098] Optionally, the horizontal coordinates and the vertical coordinates of all pixel coordinates in each target connected area are added up respectively and then the average is calculated to obtain the center coordinates of a target connected area, and the center coordinates can be used as a convex hull point.

[0099] For example, the cv2.connectedComponentsWithStats function in the OpenCV library is called to obtain the center coordinates of each target connected area, and then the cv2.convexHull function is called to take the center coordinates of each target connected area as input, and the minimum convex hull formed by each target connected area is output, that is, the boundary of the calibration card image. Figure 4 , Figure 4 Based on Figure 3 Schematic diagram of the boundary of the calibration card image obtained after convex hull extraction.

[0100] Step S20, determining a first vector according to coordinates of the first convex hull point and the second convex hull point, and determining a second vector according to the first convex hull point and the third convex hull point, wherein the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point;

[0101] Step S30, determining a corner point of the chart according to an angle between the first vector and the second vector;

[0102] After the convex hull is determined, it is necessary to screen the convex hull points on the convex hull. For the calibration card image with affine transformation (i.e., the calibration card image in this embodiment), by extracting the four card corner points in the calibration card image, the four card corner points are used as the feature information of the calibration card image collected by the camera during the camera calibration process, and the feature information of the actual calibration card is associated with each other to obtain a mapping relationship that can reflect the calibration card image and the calibration card, thereby completing the camera calibration based on the mapping relationship. Affine transformation is a two-dimensional projection that maps points in a plane to another plane. Affine transformation maintains the "flatness" of two-dimensional graphics, that is, the place that was originally a straight line is still a straight line. Affine transformation has strong practicality, such as image registration, image correction and texture correction, and creation of panoramic images. It should be noted that in the traditional camera calibration method, for images with affine transformation, it is necessary to match each point of the calibration card image with the actual point on the calibration card one by one, and then perform an inverse affine transformation on the image to restore the image that is consistent with the shape of the actual calibration card. Compared with the traditional affine transformation restoration method, in this embodiment, only 4 pairs of points before and after the transformation need to be found to complete the affine transformation restoration, and it is also applicable to other scenarios that require affine transformation restoration.

[0103] For the convex hull on the original image, a convex hull point is set as the first convex hull point, and its two adjacent points are the second convex hull point and the third convex hull point. The first vector is determined according to the coordinates of the first convex hull point and the second convex hull point, and the second vector is determined according to the first convex hull point and the third convex hull point. Then, according to the angle between the first vector and the second vector, the card corner points in the original image are screened out.

[0104] Exemplarily, the first convex hull point is set to P0, the second convex hull point is set to P1, the third convex hull point is set to P2, the first vector V1 = P1-P0, the second vector V2 = P2-P0, and the angle d0 between V1 and V2 is calculated;

[0105]

[0106] d0 is the angle of P0.

[0107] Based on this, calculate the angle d of each convex hull point in the convex hull in turn n＝1，2，3，4... ;

[0108] Compare the angle of each convex hull point with the preset angle range. Take a rectangular chart as an example. The chart has four corners, each of which is 90 degrees. Therefore, among the points on the convex hull, only four points have angles close to 90 degrees, and the remaining points are on the edge with angles close to 180 degrees, that is, the angle threshold is set to 90 degrees.

[0109] Taking into account the error value in the actual image processing process, a K value can be added to the angle threshold to eliminate the convex hull points with angle values ​​less than 90-K and greater than 90+K. The obtained points are the four points closest to 90 degrees, which are used as the corner points of the chart.

[0110] Optionally, after determining the card corner points, the card corner points need to be sorted. After obtaining the card corner points, the image that is consistent with the shape of the calibration card cannot be directly restored based on the coordinates of the card corner points. Therefore, the sorting rules of the card corner points can be set according to the shape of the calibration card, and the card corner points can be sorted according to the sorting rules, so that the machine can connect and generate an image that is consistent with the shape of the calibration card according to the sorting order of the card corner points and the coordinates of the card corner points.

[0111] For example, assuming that the calibration card is a rectangle, the coordinates of the obtained card corner points are [400, 1600], [400, 1000], [1600, 1000], [1600, 1600], and the sorting order of the four coordinates is 1, 2, 3, and 4. In the subsequent image generation, the machine connects the four card corner points counterclockwise in the order of 1, 2, 3, and 4, and the resulting image is also a rectangle.

[0112] Step S40, determining an inverse affine transformation matrix of the original image based on the coordinates of the corner points of the chart, and generating a target image according to the affine transformation matrix and the original image;

[0113] Furthermore, an inverse affine transformation matrix of the original image is determined based on the coordinates of the corner points of the chart, and a target image is generated according to the inverse affine transformation matrix and the original image.

[0114] The inverse affine transformation matrix is ​​the inverse matrix corresponding to the affine transformation matrix obtained by inversely solving the affine transformation matrix. The affine transformation matrix transforms the coordinates of the corner points of the chart through linear transformation to generate a homogeneous coordinate matrix, and then obtains the horizontal axis coordinate equation and the vertical axis coordinate equation of the corner points of the chart by normalization; then, the coordinates of each corner point of the chart, the coordinates of the calibration chart, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each corner point of the chart are used to determine the affine transformation matrix H between the calibration chart and the calibration chart image, and then the affine transformation matrix H is subjected to an inverse matrix transformation to obtain the affine transformation inverse matrix H -1 , according to H -1 Establish the mapping function between the calibration chart and the original image:

[0115] X=H -1 *X′

[0116] Wherein, X′ is the coordinate of the corner point of the chart on the original image, and X is the coordinate restored by affine transformation (that is, the mapping coordinate of the coordinate of the calibration chart in the actual scene in the coordinate system of the original image).

[0117] According to the coordinate X restored by affine transformation, the target image is generated. The target image is the image obtained by restoring the initial image by affine transformation. Figure 5 , Figure 5 In a specific implementation, based on Figure 4 Schematic diagram of the target image obtained by using the image in .

[0118] In the technical solution provided in this embodiment, the convex hull is extracted for the target connected area in the original image, and the required card corner points are screened out from the convex hull. The card corner points are subjected to an inverse affine transformation to obtain the target card corner points with the affine transformation eliminated. Finally, the target image with the affine transformation eliminated is generated according to the coordinates of the target card corner points. In the entire image processing process, only relatively simple linear operations and geometric operations are required, thereby improving the image processing efficiency.

[0119] Reference Figure 6 In the second embodiment, based on the first embodiment, before step S10, the following steps are further included:

[0120] Step S50, binarizing the collected calibration card image to obtain a binarized image corresponding to the calibration card image;

[0121] Step S60, extracting the coordinates, aspect ratio and / or area of ​​the connected regions in the binary image;

[0122] Step S70, obtaining reference coordinates, reference aspect ratio and / or reference area;

[0123] Step S80, treating the connected regions in the binary image that do not match the reference coordinates, the reference aspect ratio and / or the reference area as connected regions with irregular shapes;

[0124] Step S90, eliminating the connected regions with irregular shapes to obtain the original image.

[0125] Optionally, this embodiment provides a method for determining the original image. In this embodiment, the captured calibration card image is preprocessed to obtain feature information in the calibration card image, and the feature information includes but is not limited to connected area information in the calibration card image. A connected area (Connected Component) generally refers to an image area composed of foreground pixels with the same pixel value and adjacent positions in the image. The connected area information includes but is not limited to the coordinates, aspect ratio and area of ​​the connected area. Optionally, the calibration card image can be a color image or a grayscale image after pixel processing. It should be noted that the captured calibration card image is an image that has a certain angle deviation from the actual calibration card.

[0126] Preprocessing may be to remove irregular areas in the calibration card image. In some embodiments, the calibration card image is first binarized. For example, the binarization may be performed by using the cv2.threshold function in OpenCV (an open source computer vision framework software) to binarize the image and obtain a binarized image corresponding to the calibration card image; then, the connected area of ​​the binary image is extracted. For example, the connected area may be determined by a connected domain analysis algorithm and may be obtained by a common algorithm such as a two-pass method or a seed-filling method; then, the connected area coordinates, aspect ratio and / or area are extracted from the connected area. For example, the connected area is determined by inputting the image into OpenCV and using the API software package provided by OpenCV, and the feature information of the connected area is obtained, including but not limited to information such as coordinates, length, width and area; then, reference feature information such as reference coordinates, reference aspect ratio and / or reference area as feature information of the regular area is obtained. It should be noted that the regular area can be an area determined in advance by the developer, or it can be a selection interface for a connected area that is output, and a specific connected area in the calibration card image is selected as the target connected area through the selection interface; in the next step, a traversal matching operation is performed on each connected area in the binary image with the reference coordinates, reference aspect ratio and / or reference area, and the connected areas that do not match the aforementioned reference feature information are regarded as connected areas with irregular shapes; finally, the connected areas with irregular shapes are eliminated to obtain the original image of the connected areas with regular shapes retained.

[0127] In the technical solution provided in this embodiment, by preprocessing the collected calibration card image, an original image that only retains the connected areas of regular shapes is obtained, which facilitates subsequent affine transformation restoration processing and improves image processing efficiency.

[0128] In addition, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method described in the above embodiment are implemented.

[0129] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0130] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0131] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises: Determine the pixel coordinates of each target connected area in the original image, and determine the convex hull points according to the pixel coordinates; Determine a first vector according to coordinates of the first convex hull point and the second convex hull point, and determine a second vector according to the first convex hull point and the third convex hull point, wherein the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point; Get the preset angle threshold; Determine the angle of the convex hull point according to the angle between the first vector and the second vector; According to the comparison between the angle of each convex hull point and the preset angle threshold, the convex hull point closest to the preset angle threshold is selected as the corner point of the chart; Based on the coordinates of the corner points of the chart, an inverse affine transformation matrix of the original image is determined, and a target image is generated according to the inverse affine transformation matrix and the original image.

2. The image processing method according to claim 1, characterized in that: Determining the convex hull point according to the pixel coordinates comprises: Determine the pixel coordinates corresponding to the pixel points corresponding to the target connected area; Determine a coordinate mean corresponding to the pixel coordinates in the target connected area, and use the coordinate mean as the center coordinate corresponding to the target connected area; The pixel point corresponding to the center coordinates of the target connected area is used as the convex hull point.

3. The image processing method according to claim 1, characterized in that: After the step of determining the corner point of the chart according to the angle of the convex hull point and the preset angle threshold, the method further includes: Determine the sorting rule of the corner points of the chart according to the coordinates of the corner points of the chart; The corner points of the image card are sorted according to the image card corner point sorting rule.

4. The image processing method according to claim 1, wherein: Before the step of determining the inverse affine transformation matrix of the original image based on the coordinates of the corner points of the card, and generating the target image according to the inverse affine transformation matrix and the original image, the step further includes: Obtaining coordinates of a calibration chart, where the coordinates of the calibration chart are the coordinates of the calibration chart in the actual scene; Calculate the homogeneous coordinate matrix corresponding to the coordinates of the corner points of the card; Normalizing the homogeneous coordinate matrix to obtain the horizontal axis coordinate equation and the vertical axis coordinate equation of the corner point of the chart; An affine transformation matrix is ​​determined according to the coordinates of each corner point of the chart, the coordinates of the calibration chart, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each corner point of the chart.

5. The image processing method according to claim 4, characterized in that: The step of determining the inverse affine transformation matrix of the original image based on the coordinates of the corner points of the card, and generating the target image according to the inverse affine transformation matrix and the original image comprises: Determine the affine transformation inverse matrix corresponding to the affine transformation matrix through inverse matrix transformation; Determine the coordinates of the corner points of the target chart according to the inverse affine transformation matrix and the coordinates of the corner points of the chart, wherein the coordinates of the corner points of the target chart are the mapping coordinates of the coordinates of the calibration chart in the original image; The target image is determined based on the coordinates of the corner points of the target card.

6. The image processing method according to claim 1, characterized in that: Determining the pixel coordinates of each target connected area in the original image includes: Obtaining a pixel value of each pixel in the original image and obtaining a preset pixel threshold; Determining the target connected area according to the pixel value of each pixel point and the preset pixel threshold; The pixel coordinates of the target connected area are obtained.

7. The image processing method according to claim 1, characterized in that: Before the step of determining the pixel coordinates of each target connected area in the original image and determining the convex hull points according to the pixel coordinates, the method further includes: Binarizing the collected calibration card image to obtain a binary image corresponding to the calibration card image; Extracting coordinates, aspect ratio and / or area of ​​connected regions in the binary image; Obtaining reference coordinates, reference aspect ratio and / or reference area; The connected regions in the binary image that do not match the reference coordinates, the reference aspect ratio and / or the reference area are regarded as connected regions with irregular shapes; The connected regions with irregular shapes are eliminated to obtain the original image.

8. An image processing device, characterized in that: The device comprises: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program implements the steps of the image processing method according to any one of claims 1 to 7 when executed by the processor.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

Images