Image Coordinate Matching Method, Terminal, and Computer-Readable Storage Medium

Abstract

The present invention relates to the field of digital image processing technology, and in particular, to an image coordinate matching method, a terminal, and a computer-readable storage medium. Among them, the method includes: determining the corner points of the chart in the two-dimensional image, and determining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the corner points of the chart; determining the connected region number corresponding to the target connected region according to the center coordinates of each target connected region corresponding to the two-dimensional image; based on the coordinate affine transformation matrix, determining the mapping coordinates of the pixel points corresponding to the connected region number in the three-dimensional image; according to the mapping coordinates, determining the matching coordinates corresponding to the connected region number in the connected region coordinate array. It realizes the rapid matching between the coordinates of the three-dimensional image and the coordinates of the two-dimensional image collected by the camera, improves the matching speed in the image coordinate matching process, and thus improves the image processing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and particularly relates to an image coordinate matching method, a terminal, and a computer-readable storage medium. Background Art

[0002] Camera calibration is a common means in image processing technology. During the camera calibration process, it is necessary to match the two-dimensional coordinates of the captured calibration chart image with the three-dimensional coordinates of the chart in the real scene, and then use the matching relationship between the three-dimensional coordinates and the two-dimensional coordinates as the algorithm input for camera calibration, so as to achieve camera calibration.

[0003] However, for the calibration chart, since there are intervals between the calibration points on the chart, when the machine identifies the calibration points, the obtained three-dimensional coordinates are also discrete. Therefore, when placing the three-dimensional coordinates of the calibration points into the camera two-dimensional coordinate system for matching, it cannot directly correspond to the scale order in the two-dimensional coordinate system for fast matching.

[0004] In the traditional matching scheme, usually after determining the matching relationship between the two-dimensional coordinates and the three-dimensional coordinates, each point in the two-dimensional coordinate axis is traversed for matching. This way of traversal matching has a greater matching difficulty and a longer matching process, and there are problems such as low matching efficiency in actual industrial applications.

[0005] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of the present invention is to provide an image coordinate matching method, aiming to solve the problem of how to quickly match the coordinates of a three-dimensional image with the coordinates of a two-dimensional image collected by a camera.

[0007] To achieve the above object, an image coordinate matching method provided by the present invention includes:

[0008] Determine the corner points of the chart in the two-dimensional image, and determine the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the corner points of the chart;

[0009] Determine the connected region number corresponding to the target connected region according to the center coordinates of each target connected region corresponding to the two-dimensional image;

[0010] Based on the coordinate affine transformation matrix, determine the mapping coordinates of the pixel points corresponding to the connected region number in the three-dimensional image;

[0011] Determine, according to the mapping coordinates, the matching coordinates in the connected region coordinate array corresponding to the connected region number, where the matching coordinates are the central coordinates of the region where the target connected region of the two-dimensional image matches the corresponding region of the three-dimensional image.

[0012] Optionally, before the step of determining the connected region number corresponding to the target connected region according to the central coordinates of each target connected region corresponding to the two-dimensional image, the method further includes:

[0013] Obtain the image coordinate system in the two-dimensional image;

[0014] Determine the numbering order of the target connected regions according to the coordinate origin of the image coordinate system;

[0015] Determine the connected region coordinate array according to the numbering order and the central coordinates.

[0016] Optionally, the step of determining the connected region number corresponding to the target connected region according to the central coordinates of each target connected region corresponding to the two-dimensional image includes:

[0017] Determine the pixel mean value of the target connected region corresponding to the central coordinates according to the central coordinates and the pixel value corresponding to the central coordinates;

[0018] Determine the numbered region associated with the target connected region in the two-dimensional image according to the pixel mean value, and determine the numbered position associated with the target connected region in the two-dimensional image according to the central coordinates;

[0019] Determine the connected region number corresponding to the target connected region according to the numbered position and the numbered region.

[0020] Optionally, the step of determining, according to the mapping coordinates, the matching coordinates in the connected region coordinate array corresponding to the connected region number includes:

[0021] Determine the central coordinates of the region in the connected region coordinate array according to the mapping coordinates and the connected region number, where the central coordinates of the region are the central coordinates of the corresponding region of the target connected region in the three-dimensional image;

[0022] Determine the matching coordinates corresponding to each connected region number according to the central coordinates of the region and the connected region number.

[0023] Optionally, determining the card corner points in the two-dimensional image includes:

[0024] Determine the convex hull points according to the central coordinates of each target connected region corresponding to the two-dimensional image;

[0025] Determine a first vector based on the coordinates of a first convex hull point and a second convex hull point, and determine a second vector based on the first convex hull point and a third convex hull point, where the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point;

[0026] Determine the graphic card corner point according to the angle between the first vector and the second vector.

[0027] Optionally, the determining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the graphic card corner point includes:

[0028] Obtain the coordinates of the three-dimensional image, where the coordinates of the three-dimensional image are the coordinates of the three-dimensional image in the actual scene;

[0029] Calculate the homogeneous coordinate matrix corresponding to the coordinates of the graphic card corner point;

[0030] Normalize the homogeneous coordinate matrix to obtain the horizontal axis coordinate equation and the vertical axis coordinate equation of the graphic card corner point;

[0031] Determine the coordinate affine transformation matrix according to the coordinates of each graphic card corner point, the coordinates of the three-dimensional image, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each graphic card corner point.

[0032] Optionally, before the step of determining the graphic card corner point in the two-dimensional image and determining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the graphic card corner point, it further includes:

[0033] Obtain the coordinates of the pixel points corresponding to the connected region numbers;

[0034] Perform an affine transformation on the coordinates of the pixel points corresponding to the connected region numbers based on the coordinate affine transformation matrix to obtain the mapped coordinates.

[0035] Optionally, before the step of determining the graphic card corner point in the two-dimensional image and determining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the graphic card corner point, it further includes:

[0036] Binarize the collected calibration graphic card image to obtain the binarized image corresponding to the calibration graphic card image;

[0037] Extract the coordinates, aspect ratio and / or area of the connected regions in the binarized image;

[0038] Obtain the reference coordinates, reference aspect ratio and / or reference area;

[0039] The connected regions in the binarized image that do not match the reference coordinates, the reference aspect ratio, and / or the reference area are regarded as irregularly shaped connected regions;

[0040] Remove the irregularly shaped connected regions to obtain the two-dimensional image.

[0041] In addition, to achieve the above object, the present invention also provides an image coordinate matching terminal, which includes: a memory, a processor, and an image coordinate matching program stored on the memory and executable on the processor. When the image coordinate matching program is executed by the processor, each step of the above-described image coordinate matching method is implemented.

[0042] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium storing an image coordinate matching program. When the image coordinate matching program is executed by a processor, each step of the image coordinate matching method described in the above embodiments is implemented.

[0043] An embodiment of the present invention provides an image coordinate matching method, a terminal, and a computer-readable storage medium. By determining the card corner points in a two-dimensional image, determining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image based on the card corner points, then determining the connected region numbers corresponding to the target connected regions according to the central coordinates of the respective target connected regions in the two-dimensional image, and then calling the coordinate affine transformation matrix to determine the mapping coordinates of the pixel points corresponding to the connected region numbers in the three-dimensional image, and finally determining the central coordinates of the regions in the connected region coordinate array that match the target connected regions in the two-dimensional image and the corresponding regions in the three-dimensional image according to the mapping coordinates, the rapid matching between the coordinates of the three-dimensional image and the coordinates of the two-dimensional image collected by the camera is realized, the matching speed in the image coordinate matching process is improved, and thus the image processing efficiency is improved. Description of the Drawings

[0044] Figure 1 It is a schematic diagram of the terminal architecture of the hardware operating environment of the image coordinate matching method according to the embodiment of the present invention;

[0045] Figure 2 It is a schematic flowchart of the first embodiment of the image coordinate matching method of the present invention;

[0046] Figure 3 A schematic diagram of the two-dimensional image in a specific embodiment of the image coordinate matching method of the present invention;

[0047] Figure 4 A schematic diagram of the connected region labeling in a specific embodiment of the image coordinate matching method of the present invention;

[0048] Figure 5 Schematic flowchart of the second embodiment of the image coordinate matching method of the present invention.

[0049] The realization, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0050] It should be understood that the accompanying drawings of the present invention show exemplary embodiments of the present invention, and the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0051] As an implementation solution, refer to Figure 1 , Figure 1 which is a schematic diagram of the terminal architecture of the hardware operating environment involved in the embodiment solution of the present invention.

[0052] As shown in Figure 1 , the control terminal may include: a processor 1001, such as a CPU, a network interface 1003, a memory 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The network interface 1003 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1004 may be a high-speed RAM memory or a stable memory. The memory 1004 may optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art can understand that Figure 1 the terminal architecture shown in

[0053] does not limit the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or different component arrangements. Figure 1 the terminal architecture shown in

[0054] does not limit the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or different component arrangements. Figure 1 As shown in

[0055] In the terminal shown in Figure 1 , the processor 1001 may be used to call the image coordinate matching program stored in the memory 1004 and perform the following operations:

[0056] Determine the card corner points in the two-dimensional image, and determine the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the card corner points;

[0057] Determine the connected region numbers corresponding to the target connected regions according to the central coordinates of the respective target connected regions corresponding to the two-dimensional image;

[0058] Based on the coordinate affine transformation matrix, determine the mapping coordinates of the pixel points corresponding to the connected region numbers in the three-dimensional image;

[0059] According to the mapping coordinates, determine the matching coordinates corresponding to the connected region numbers in the connected region coordinate array, where the matching coordinates are the central coordinates of the region where the target connected region of the two-dimensional image matches the corresponding region of the three-dimensional image.

[0060] Optionally, the processor 101 may be used to call the image coordinate matching program stored in the memory 102 and perform the following operations:

[0061] Obtain the image coordinate system in the two-dimensional image;

[0062] Determine the numbering order of the target connected regions according to the coordinate origin of the image coordinate system;

[0063] Determine the connected region coordinate array according to the numbering order and the central coordinates.

[0064] Optionally, the processor 101 may be used to call the image coordinate matching program stored in the memory 102 and perform the following operations:

[0065] Determine the pixel mean value of the target connected region corresponding to the central coordinate according to the central coordinate and the pixel value corresponding to the central coordinate;

[0066] Determine the numbered region associated with the target connected region in the two-dimensional image according to the pixel mean value, and determine the numbered position associated with the target connected region in the two-dimensional image according to the central coordinate;

[0067] Determine the connected region number corresponding to the target connected region according to the numbered position and the numbered region.

[0068] Optionally, the processor 101 may be used to call the image coordinate matching program stored in the memory 102 and perform the following operations:

[0069] Determine the regional central coordinates in the connected region coordinate array according to the mapping coordinates and the connected region numbers, where the regional central coordinates are the central coordinates of the corresponding region of the target connected region in the three-dimensional image;

[0070] Determine the matching coordinates corresponding to each of the connected region numbers according to the region center coordinates and the connected region numbers.

[0071] Optionally, the processor 101 may be configured to call an image coordinate matching program stored in the memory 102 and perform the following operations:

[0072] Determine convex hull points according to the center coordinates of the respective target connected regions corresponding to the two-dimensional image;

[0073] Determine a first vector according to the coordinates of a first convex hull point and a second convex hull point, and determine a second vector according to the first convex hull point and a third convex hull point, where the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point;

[0074] Determine the graphics card corner points according to the angle between the first vector and the second vector.

[0075] Optionally, the processor 101 may be configured to call an image coordinate matching program stored in the memory 102 and perform the following operations:

[0076] Obtain the coordinates of the three-dimensional image, where the coordinates of the three-dimensional image are the coordinates of the three-dimensional image in the actual scene;

[0077] Calculate the homogeneous coordinate matrix corresponding to the coordinates of the graphics card corner points;

[0078] Perform a normalization process on the homogeneous coordinate matrix to obtain the horizontal axis coordinate equation and the vertical axis coordinate equation of the graphics card corner points;

[0079] Determine the coordinate affine transformation matrix according to the coordinates of each graphics card corner point, the coordinates of the three-dimensional image, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each graphics card corner point.

[0080] Optionally, the processor 101 may be configured to call an image coordinate matching program stored in the memory 102 and perform the following operations:

[0081] Obtain the coordinates of the pixel points corresponding to the connected region numbers;

[0082] Perform an affine transformation on the coordinates of the pixel points corresponding to the connected region numbers based on the coordinate affine transformation matrix to obtain the mapped coordinates.

[0083] Optionally, the processor 101 may be configured to call an image coordinate matching program stored in the memory 102 and perform the following operations:

[0084] Perform a binarization process on the collected calibration graphics card image to obtain the binarized image corresponding to the calibration graphics card image;

[0085] Extract the coordinates, aspect ratio, and / or area of the connected regions in the binary image;

[0086] Obtain the reference coordinates, reference aspect ratio, and / or reference area;

[0087] Take the connected regions in the binary image that do not match the reference coordinates, the reference aspect ratio, and / or the reference area as the connected regions with irregular shapes;

[0088] Remove the connected regions with irregular shapes to obtain the two-dimensional image.

[0089] Based on the terminal architecture of the hardware operating environment of the image coordinate matching terminal based on digital image processing technology above, embodiments of the image coordinate matching method of the present invention are proposed.

[0090] During the camera calibration process, it is necessary to identify the calibration card on the image, and it is necessary to match the coordinates of the captured two-dimensional image with the coordinates of the three-dimensional image in the real scene. Then, the mapping relationship between the coordinates of the three-dimensional image and the coordinates of the two-dimensional image is used as the input of the camera calibration algorithm. However, due to the lack of corner point information on the calibration card and the disconnection between adjacent calibration points, the coordinates of the identified calibration points cannot be directly arranged in order. During matching, it is necessary to traverse the coordinates of each calibration point in the two-dimensional image with the coordinates of all calibration points in the three-dimensional image, and then screen out the calibration points corresponding to the calibration points in the two-dimensional image from the matching results. This matching method is slow and difficult to meet the requirement of quickly realizing camera calibration in actual industrial applications.

[0091] Therefore, through the image coordinate matching method in the present invention, by numbering the coordinates of the target connected regions in the two-dimensional image, after obtaining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image, the coordinates of the transformed three-dimensional image are obtained through the transformation matrix, and then the coordinates of the three-dimensional image are matched through the numbering in the connected region coordinate array of the target connected region to obtain the coordinates corresponding to the coordinates of the three-dimensional image in the two-dimensional image, thereby completing the matching of the calibration card coordinates and the picture coordinates.

[0092] Refer to Figure 2 , in the first embodiment, the image coordinate matching method includes the following steps:

[0093] Step S10, determine the calibration card corners in the two-dimensional image, and determine the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the calibration card corners;

[0094] In this embodiment, the corner points of the calibration card are pixel points located at the corners of the two-dimensional image. The corner points of the calibration card are used as the feature information of the two-dimensional image of the calibration card collected by the camera during the camera calibration process, and are associated with the feature information of the three-dimensional image of the actual calibration card, so as to obtain a coordinate affine transformation matrix that can reflect the mapping relationship between the two-dimensional image and the three-dimensional image.

[0095] Optionally, the method for determining the corner points of the two-dimensional image can be to perform convex hull extraction on the two-dimensional image, and screen out the convex hull points with orthogonal vector angles between adjacent convex hull points as the corner points of the calibration card. The convex hull is a concept of geometric figures, defined as a convex polygon formed by connecting the outermost points of the figure, which can contain all the points in the point set. The convex hull points are located on the outermost periphery of the convex polygon, and the overall shape of the two-dimensional image is usually a convex polygon (such as a rectangle).

[0096] Exemplarily, for the convex hull on the two-dimensional image, set a convex hull point as the first convex hull point, and its two adjacent points are the second convex hull point and the third convex hull point respectively. Determine the first vector according to the coordinates of the first convex hull point and the second convex hull point, and determine the second vector according to the first convex hull point and the third convex hull point. Then, according to the angle between the first vector and the second vector, screen out the corner points of the calibration card in the two-dimensional image. Set the two-dimensional image as a rectangular image, set the first convex hull point as P0, the second convex hull point as P1, and the third convex hull point as P2. The first vector V1 = P1 - P0, the second vector V2 = P2 - P0, and calculate the angle d0 between V1 and V2;

[0097]

[0098] d0 is used as the angle of P0.

[0099] Based on this, calculate the angle d of each convex hull point in the convex hull in turn n＝1，2，3，4... ;

[0100] Compare the angle of each convex hull point with a preset angle interval. Taking a rectangular calibration card as an example, the calibration card has four corners, and each corner is 90 degrees. Therefore, among the points on the convex hull, only the angles of four points are close to 90 degrees, and the remaining points are on the sides, and the angles are close to 180 degrees. That is, the angle threshold is set to 90 degrees.

[0101] Considering the error value in the actual image processing process, a K value can be added to the angle threshold, and the convex hull points with angle values less than 90 - K and greater than 90 + K are removed. The obtained points are the four points closest to 90 degrees and are used as the corner points of the calibration card.

[0102] Optionally, the method for determining the coordinate affine transformation matrix based on the corner points of the calibration card may be to generate a homogeneous coordinate matrix by linearly transforming the coordinates of the corner points of the calibration card, and then obtain the horizontal axis coordinate equation and the vertical axis coordinate equation of the corner points of the calibration card through normalization; then, determine the coordinate affine transformation matrix based on the coordinates of each calibration card corner point, the coordinates of the three-dimensional image, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each calibration card corner point.

[0103] Step S20: Determine the connected region numbers corresponding to the target connected regions according to the central coordinates of the respective target connected regions corresponding to the two-dimensional image.

[0104] Furthermore, number the target connected regions in the two-dimensional image that need to be matched. First, obtain the central coordinates of the target connected regions, and then determine the connected region numbers corresponding to the target connected regions according to the central coordinates. A connected component generally refers to an image region composed of foreground pixel points with the same pixel value and adjacent positions in the image.

[0105] Refer to Figure 3 , Figure 3 is a schematic diagram of a two-dimensional image in a specific embodiment. The connected regions in the two-dimensional image include white dots and black regions. For a machine, not only each white dot will be recognized as a connected region by the machine, but the black regions will also be recognized as connected regions. However, in this embodiment, the part to be processed is a dot matrix composed of 9*12 white dots, and the connected regions outside the dot matrix are not processed. Therefore, in this step, it is also necessary to select the connected regions to be processed from the two-dimensional image as the target connected regions. Since the two-dimensional image is a binary image and the pixel values in the image only include two values, 255 (i.e., white) and 0 (black), a pixel threshold can be set to select only the regions that meet the pixel threshold in the connected regions as the target connected regions. Exemplarily, the pixel threshold can be set to 255, and the regions with pixel value 255 in the connected regions are used as the target connected regions. As Figure 3 shown, each white dot is a target connected region.

[0106] Optionally, the method for determining the central coordinates of the target connected regions may be to add the abscissas and ordinates of all pixel coordinates in each target connected region respectively and then calculate the mean value to obtain the central coordinates of a target connected region.

[0107] Optionally, for the method of numbering the target connected regions according to the central coordinates, since the pixel values in the target connected regions are the same, the pixels in this region can be assigned the same number. For example, the first connected region is numbered 1, the second connected region is numbered 2, and so on.

[0108] For a machine, since the machine cannot simply determine where the target connected region corresponding to the number in the image is based on the numbered digits alone, it is therefore necessary to solve the problem of how to enable the machine to determine the target connected region of the image according to the number.

[0109] Exemplarily, in a specific embodiment, the cv2.connectedComponentsWithStats function in the OpenCV library function is called to obtain the central coordinates of each target connected region, and based on the central coordinates, the pixel values of other pixel points with the same pixel value as the pixel points corresponding to the coordinates around the central coordinates are obtained. Since the pixel values of all pixel points within the target connected region are the same, only the pixel value of the central coordinate needs to be used as the input quantity to traverse the surrounding pixel points. Pixel points with the same pixel value as the central coordinate are considered to be points within the target connected region. That is, each pixel point within the target connected region is bundled and assigned the same number. The range of the target connected region is the numbered region corresponding to the number, and the central coordinate is used as the numbered position corresponding to the number. By identifying the numbered region and the numbered position, the machine can determine the target connected region corresponding to the connected region number.

[0110] Step S30: Based on the coordinate affine transformation matrix, determine the mapping coordinates of the pixel points corresponding to the connected region number in the three-dimensional image;

[0111] Step S40: According to the mapping coordinates, determine the matching coordinates corresponding to the connected region number in the connected region coordinate array. The matching coordinates are the central coordinates of the region where the target connected region of the two-dimensional image matches the corresponding region of the three-dimensional image;

[0112] After determining the connected region numbers, coordinate matching between the three-dimensional image and the two-dimensional image needs to be achieved according to the connected region numbers. At this time, the coordinate affine transformation matrix H generated previously is called, and an affine transformation is performed on the coordinates in the two-dimensional image through H. It should be noted that due to a certain offset angle between the two-dimensional image and the actual three-dimensional image during the acquisition process, there is an affine transformation for the two-dimensional image, that is, the three-dimensional image, which is originally a standard regular shape, is deformed in the two-dimensional image. During the matching process of image coordinate points, this deformation factor also needs to be considered. Therefore, an affine transformation process also needs to be performed on the obtained pixel point coordinates to correspond to the affine transformation between the three-dimensional image and the two-dimensional image. It should be noted that compared with the traditional method of implementing affine transformation by corresponding each point one by one, in this embodiment, the affine transformation can be completed only based on several card corner points in the two-dimensional image, and the mapped coordinate X′ can be obtained. Moreover, this method is also applicable to other scenarios that require affine transformation restoration. Taking the mapped coordinate X′ as the input quantity, it is substituted into the connected region coordinate array for matching to determine the matching coordinates where the target connected region in the two-dimensional image matches the corresponding region in the three-dimensional image.

[0113] Optionally, the determination method of the mapped coordinate X′ can be to obtain the coordinate X of the pixel point corresponding to the obtained connected region number through the coordinate affine transformation matrix H to obtain the mapped coordinate X′ after affine transformation:

[0114] X′ = X * H

[0115] Optionally, the connected region coordinate array can be obtained by setting an image coordinate system in the two-dimensional image, determining the numbering order of the target connected region according to the coordinate origin of the image coordinate system, and numbering the central coordinates according to the numbering order to obtain the connected region coordinate array.

[0116] Exemplarily, assume that the target connected region in the two-dimensional image is a 9 * 12 dot matrix. Taking the upper left corner point as the coordinate origin [0, 0], the right side of the origin is the positive direction of the X-axis, and the lower side of the origin is the positive direction of the Y-axis. Each dot matrix is used as a coordinate number, and the formed connected region coordinate array is: [0, 0], [0, 1], [0, 2], [0, 3]... [8, 10][8, 11]. Refer to Figure 4 , Figure 4 In a specific embodiment, based on Figure 3 the two-dimensional image, the connected region label map obtained by numbering coordinates through the connected region coordinate array.

[0117] In the technical solution provided in this embodiment, by determining the corner points of the calibration card in the two-dimensional image, and determining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the corner points of the calibration card, then according to the center coordinates of each target connected region corresponding to the two-dimensional image, determining the connected region number corresponding to the target connected region, and then calling the coordinate affine transformation matrix to determine the mapping coordinates of the pixel points corresponding to the connected region number in the three-dimensional image, and finally according to the mapping coordinates, determining the center coordinates of the region in the connected region coordinate array where the target connected region of the two-dimensional image corresponding to the connected region number matches the corresponding region of the three-dimensional image, the fast matching between the coordinates of the three-dimensional image and the coordinates of the two-dimensional image collected by the camera is realized, the matching speed in the image coordinate matching process is improved, and thus the image processing efficiency is improved.

[0118] Referring to Figure 5 , in the second embodiment, based on the first embodiment, before the step S10, it further includes:

[0119] Step S50, performing binarization processing on the collected calibration card image to obtain the binarized image corresponding to the calibration card image;

[0120] Step S60, extracting the coordinates, aspect ratio and / or area of the connected regions in the binarized image;

[0121] Step S70, obtaining the reference coordinates, reference aspect ratio and / or reference area;

[0122] Step S80, taking the connected regions in the binarized image that do not match the reference coordinates, the reference aspect ratio and / or the reference area as the connected regions with irregular shapes;

[0123] Step S90, removing the connected regions with irregular shapes to obtain the two-dimensional image.

[0124] Optionally, this embodiment provides a method for determining a two-dimensional image. In this embodiment, the two-dimensional image is an image after binarization processing of the collected calibration card image. The captured calibration card image is preprocessed to obtain the feature information in the calibration card image, and the feature information includes, but is not limited to, the connected region information in the calibration card image. Optionally, the calibration card image can be a color image or a grayscale image after pixel processing.

[0125] In this embodiment, the preprocessing may be to remove the irregular regions in the calibration chart image. In some specific embodiments, first, the calibration chart image is binarized. Exemplarily, the binarization process can be performed on the image through the cv2.threshold function in OpenCV (an open-source computer vision framework software) to obtain the binary image corresponding to the calibration chart image. Then, the connected regions of the binary image are extracted. Exemplarily, the connected regions can be determined through a connected component analysis algorithm, and common algorithms such as the Two-Pass method or the Seed-Filling method can be used to obtain them. Then, the coordinates, aspect ratio, and / or area of the connected regions are extracted from the connected regions. Exemplarily, by inputting the image into OpenCV and using the built-in API package of OpenCV, the connected regions are determined, and the feature information of the connected regions is obtained, including but not limited to information such as coordinates, length, width, and area. Then, the reference coordinates, reference aspect ratio, and / or reference area, etc., which are the feature information of the regular region, are obtained. It should be noted that the regular region can be a region determined in advance by developers, or a specific connected region in the calibration chart image can be selected as the target connected region by outputting a selection interface for connected regions. Next, the binary image is traversed and matched with each connected region in terms of the reference coordinates, reference aspect ratio, and / or reference area. The connected regions that do not match the aforementioned reference feature information are regarded as connected regions with irregular shapes. Finally, the connected regions with irregular shapes are removed, and a two-dimensional image of the target connected region with a regular shape can be obtained.

[0126] In the technical solution provided in this embodiment, by preprocessing the acquired calibration chart image, a two-dimensional image with only connected regions of regular shapes is obtained, which is convenient for subsequent affine transformation reduction processing and improves the image processing efficiency.

[0127] In addition, those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the terminal to implement the process steps of the embodiments of the above methods.

[0128] Therefore, the present invention also provides a computer-readable storage medium, which stores an image coordinate matching program. When the image coordinate matching program is executed by a processor, it implements each step of the image coordinate matching method as described in the above embodiments.

[0129] Among them, the computer-readable storage medium can be various computer-readable storage media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disc, etc., which can store program codes.

[0130] It should be noted that in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium as described above (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions to enable a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network terminal, etc.) to execute the methods described in various embodiments of the present invention.

[0132] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent architecture or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An image coordinate matching method, characterized in that, The method includes: Determine the fiducial card corner points in the two-dimensional image, and determine the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the fiducial card corner points; Determine the connected region numbers corresponding to the target connected regions according to the central coordinates of the respective target connected regions corresponding to the two-dimensional image; Based on the coordinate affine transformation matrix, determine the mapping coordinates of the pixel points corresponding to the connected region numbers in the three-dimensional image; According to the mapping coordinates, determine the matching coordinates corresponding to the connected region numbers in the connected region coordinate array, where the matching coordinates are the region center coordinates where the target connected region of the two-dimensional image matches the corresponding region of the three-dimensional image; wherein, the step of determining the matching coordinates corresponding to the connected region numbers in the connected region coordinate array according to the mapping coordinates includes: Determine the region center coordinates in the connected region coordinate array according to the mapping coordinates and the connected region numbers, where the region center coordinates are the center coordinates of the corresponding region of the target connected region in the three-dimensional image; Determine the matching coordinates corresponding to each connected region number according to the region center coordinates and the connected region numbers.

2. The image coordinate matching method according to claim 1, characterized in that, Before the step of determining the connected region numbers corresponding to the target connected regions according to the central coordinates of the respective target connected regions corresponding to the two-dimensional image, it further includes: Obtain the image coordinate system in the two-dimensional image; Determine the numbering order of the target connected regions according to the coordinate origin of the image coordinate system; Determine the connected region coordinate array according to the numbering order and the central coordinates.

3. The image coordinate matching method according to claim 1, wherein, The step of determining the connected region numbers corresponding to the target connected regions according to the central coordinates of the respective target connected regions corresponding to the two-dimensional image includes: Determine the pixel mean value of the target connected region corresponding to the central coordinate according to the central coordinate and the pixel value corresponding to the central coordinate; Determine the numbered region associated with the target connected region in the two-dimensional image according to the pixel mean value, and determine the numbered position associated with the target connected region in the two-dimensional image according to the central coordinate; Determine the connected region number corresponding to the target connected region according to the numbered position and the numbered region.

4. The image coordinate matching method according to claim 1, characterized in that, The determination of the fiducial card corner points in the two-dimensional image includes: Determine the convex hull points according to the central coordinates of the respective target connected regions corresponding to the two-dimensional image; Determine a first vector according to the coordinates of a first convex hull point and a second convex hull point, and determine a second vector according to the first convex hull point and a third convex hull point, where the second convex hull point and the third convex hull point are adjacent convex hull points of the first convex hull point; Determine the fiducial card corner points according to the included angle between the first vector and the second vector.

5. The image coordinate matching method according to claim 1, wherein The determination of the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the fiducial card corner points includes: Obtain the coordinates of the three-dimensional image, where the coordinates of the three-dimensional image are the coordinates of the three-dimensional image in the actual scene; Calculate the homogeneous coordinate matrix corresponding to the coordinates of the fiducial card corner points; Normalize the homogeneous coordinate matrix to obtain the horizontal axis coordinate equation and the vertical axis coordinate equation of the corner points of the calibration card; Determine the coordinate affine transformation matrix according to the coordinates of each corner point of the calibration card, the coordinates of the three-dimensional image, and the horizontal axis coordinate equation and the vertical axis coordinate equation corresponding to each corner point of the calibration card.

6. The image coordinate matching method according to claim 1, characterized in that, The step of determining the mapped coordinates of the pixel points corresponding to the connected region number in the three-dimensional image based on the coordinate affine transformation matrix includes: Obtain the coordinates of the pixel points corresponding to the connected region number; Based on the coordinate affine transformation matrix, perform an affine transformation on the coordinates of the pixel points corresponding to the connected region number to obtain the mapped coordinates.

7. The image coordinate matching method according to claim 1, wherein Before the step of determining the corner points of the calibration card in the two-dimensional image and determining the coordinate affine transformation matrix between the two-dimensional image and the corresponding three-dimensional image according to the corner points of the calibration card, it further includes; Perform binarization processing on the collected calibration card image to obtain the binarized image corresponding to the calibration card image; Extract the coordinates, aspect ratio and / or area of the connected regions in the binarized image; Obtain the reference coordinates, reference aspect ratio and / or reference area; Use the connected regions in the binarized image that do not match the reference coordinates, the reference aspect ratio and / or the reference area as the connected regions with irregular shapes; Remove the connected regions with irregular shapes to obtain the two-dimensional image.

8. An image coordinate matching terminal, characterized in that, The image coordinate matching terminal includes: a memory, a processor, and an image coordinate matching program stored on the memory and executable on the processor. When the image coordinate matching program is executed by the processor, it implements the steps of the image coordinate matching method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, An image coordinate matching program is stored on the computer-readable storage medium. When the image coordinate matching program is executed by the processor, it implements the steps of the image coordinate matching method according to any one of claims 1 to 7.

Images