Camera calibration method and device, computer device and storage medium

Abstract

The application relates to a camera calibration method, device, computer equipment and storage medium. The method comprises the following steps: acquiring a calibration image; the calibration image comprises mark points; the mark points comprise a sorting reference point; the sorting reference point comprises at least two boundary lines formed by a pixel point area, and the boundary lines intersect with each other; the position of the sorting reference point is determined according to a corner point set extracted from the calibration image; the mark points in the calibration image are sorted according to the position of the sorting reference point, and sorted mark points are obtained; and camera calibration is performed according to the sorted mark points. The method can improve the accuracy of camera calibration.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a camera calibration method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] Handheld 3D scanners incorporate binocular systems and structured light laser systems, with camera calibration being particularly crucial in the binocular system. Camera calibration typically involves capturing images of a calibration board and processing them to calculate the camera's intrinsic and extrinsic parameters. This process requires acquiring a large number of images and rapidly sorting and recognizing them. Manual sorting is extremely time-consuming; therefore, automated image recognition and sorting methods are commonly used. Currently, circular markers (comprising three large and several small circular markers) are frequently used as markers on calibration boards due to their ease of detection. However, projective transformations and distortions occur during camera capture. The key large circular markers in the image are often proportionally similar to the surrounding small circular markers, making it difficult to identify their features and leading to inaccurate camera calibration. Summary of the Invention

[0003] Therefore, it is necessary to provide a camera calibration method, apparatus, computer equipment, and storage medium that can improve the accuracy of camera calibration in order to address the above-mentioned technical problems.

[0004] A camera calibration method, the method comprising:

[0005] Obtain a calibration image; the calibration image includes marker points; the marker points include sorting reference points; the sorting reference points contain at least two boundary lines formed by pixel regions, and there are intersecting boundary lines;

[0006] The position of the sorting reference point is determined based on the set of corner points extracted from the calibration image;

[0007] The marker points in the calibration image are sorted according to the positions of the sorting reference points to obtain the sorted marker points;

[0008] Camera calibration is performed based on the sorted marker points.

[0009] A camera calibration device, the device comprising:

[0010] An image acquisition module is used to acquire a calibration image; the calibration image includes marker points; the marker points include sorting reference points; the sorting reference points contain at least two boundary lines formed by pixel regions, and there are intersecting boundary lines.

[0011] The position determination module is used to determine the position of the sorting reference point based on the set of corner points extracted from the calibration image;

[0012] The marker sorting module is used to sort the markers in the calibration image according to the position of the sorting reference point to obtain the sorted markers;

[0013] The calibration module is used to calibrate the camera based on the sorted marker points.

[0014] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of various method embodiments.

[0015] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of various method embodiments.

[0016] The aforementioned camera calibration method, apparatus, computer equipment, and storage medium acquire a calibration image containing sorting reference points. Since the sorting reference points contain at least two boundary lines formed by pixel regions and these boundary lines intersect, meaning the sorting reference points contain corner points, the set of corner points can be extracted from the calibration image based on this feature to determine the position of the sorting reference points. Furthermore, the sorting of marker points can be completed based on the position of the sorting reference points, thereby improving the accuracy of camera calibration. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the application environment of the camera calibration method in one embodiment;

[0018] Figure 2 This is a flowchart illustrating a camera calibration method in one embodiment;

[0019] Figure 3 This is a schematic diagram of the sorting reference points in one embodiment;

[0020] Figure 4 This is a schematic diagram of the sorting reference points in another embodiment;

[0021] Figure 5 This is a schematic diagram of the area where the corner point is located in one embodiment;

[0022] Figure 6 This is a schematic diagram of the target corner point in one embodiment;

[0023] Figure 7 This is a schematic diagram showing the annotations of points A, B, C, and M in one embodiment;

[0024] Figure 8 This is a schematic diagram of the sorted marker points in one embodiment;

[0025] Figure 9 This is a structural block diagram of a camera calibration device in one embodiment;

[0026] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0027] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art without creative effort based on the embodiments of the present invention are within the scope of protection of the present invention.

[0029] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly. The connection can be a direct connection or an indirect connection.

[0030] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0031] In one embodiment, such as Figure 1 As shown, Figure 1 This is a schematic diagram illustrating the application environment of a camera calibration method in one embodiment. Figure 1 This includes computer equipment 110. Computer equipment 110 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, servers, etc.

[0032] In one embodiment, such as Figure 2 The diagram shown is a flowchart of a camera calibration method in one embodiment. Taking the application of this method to a computer device as an example, it includes:

[0033] Step 202: Obtain a calibration image; the calibration image includes marker points; the marker points include sorting reference points; the sorting reference points contain at least two boundary lines formed by pixel regions, and there are intersecting boundary lines.

[0034] The calibration image is obtained by photographing the calibration board with the camera to be calibrated. The calibration image includes marker points. Among the marker points is a sorting reference point. This sorting reference point differs from other marker points in that it contains corner features. A corner feature is defined as the intersection of at least two boundary lines formed by pixel regions. Because the sorting reference point possesses this feature, it contains corner points. It can be understood that, due to the boundary lines between pixel regions, this sorting marker point contains at least two pixel regions with different pixel value ranges. The pixel value range can be specifically divided according to different colors. For example, red has one pixel value range, yellow has one pixel value range, black has one pixel value range, white has one pixel value range, etc. The pixel value range can also be directly divided based on a set threshold. For example, 0~100 is one pixel value range, 100~200 is another, and 200~255 is yet another. The sorting reference point can have the same shape as other marker points. For example, if the sorting reference point is circular, all other marker points are also circular; if the sorting reference point is square, all other marker points are also square. The other marker points, besides the sorting reference point, contain pixels within a certain range of pixel values.

[0035] like Figure 3 The diagram shown is a schematic representation of the sorting reference points in one embodiment. Figure 3 The sorting reference points in the data include pixel regions with black pixel values ​​and pixel regions with white pixel values. Figure 3 It contains 4 boundary lines, and some of these boundary lines intersect. Therefore Figure 3 It can be used as a sorting reference point. Understandably, a sorting reference point can also be a pie shape divided into 5, 6, 8, etc. Figure 3 The corner point is the center point.

[0036] like Figure 4 The diagram shown is a schematic representation of the sorting reference points in another embodiment. Figure 4 The sorting reference points include pixel regions with black pixel values ​​and pixel regions with white pixel values. It's understandable that the sorting reference points can also be divided into 5, 6, 8, etc. Figure 4 The corner point is the center point.

[0037] Specifically, the camera to be calibrated can be located within a computer device, and the computer device captures the calibration image using the camera within the computer device. Alternatively, the camera to be calibrated can be independent of the computer device. The camera captures the calibration image of the calibration board. The device containing the camera transmits the calibration image to the computer device via a wired or wireless network. The computer device receives the calibration image.

[0038] Step 204: Determine the position of the sorting reference point based on the set of corner points extracted from the calibration image.

[0039] In this context, corner points refer to extreme points, i.e., points with prominent pixel value attributes. The attributes of corner points can be defined by the user. Corner points can be located at the intersection of pixels with different value ranges. The set of corner points must include at least the corner points in the sorting reference point, and may also include other pixels.

[0040] Specifically, the computer device extracts a set of corner points in the calibration image based on the pixel values ​​of each pixel; and filters the set of corner points based on the corner point features of the sorting reference points to obtain the positions of each sorting reference point.

[0041] Step 206: Sort the marker points in the calibration image according to the position of the sorting reference points to obtain the sorted marker points.

[0042] Here, the sorted marker points refer to the marker points that have sorting identifiers.

[0043] Specifically, the computer device can obtain the reference sorting identifier corresponding to the sorting reference point, and sort the marker points adjacent to the sorting reference point based on the number of each column or row in the calibration image to obtain the sorted marker points. For example, if the reference sorting identifier of the sorting reference point is 42, and there are a total of 9 columns of calibration points, then the sorting identifier of the point above this sorting reference point could be 33. The computer device can then continue sorting based on the sorting reference point, or based on the marker point with the sorting identifier 33, until the marker points in the entire calibration image are sorted.

[0044] Step 208: Perform camera calibration based on the sorted marker points.

[0045] Specifically, the computer equipment determines the camera's calibration parameters based on the sorted marker points, and calibrates the camera based on these calibration parameters.

[0046] In this embodiment, a calibration image is acquired, which contains sorting reference points. Since the sorting reference points contain pixels with at least two pixel value ranges, a set of corner points can be extracted from the calibration image based on this feature to determine the position of the sorting reference points. Then, the marker points are sorted according to the position of the sorting reference points, thereby improving the accuracy of camera calibration.

[0047] In one embodiment, determining the position of the sorting reference point based on the set of corner points extracted from the calibration image includes:

[0048] Step (a1): Extract the set of corner points from the calibration image; the set of corner points includes multiple corner points.

[0049] Specifically, computer equipment can use corner detection and subpixel optimization methods to extract corner points in the calibration image and obtain a set of corner points.

[0050] Step (a2): Based on the corner position, the region where the corner is located is divided into at least two sub-regions according to the pixel region contour related to the pixel region.

[0051] The pixel region outline associated with a pixel region can be a region defined based on the intersection of pixel regions. Pixel values ​​within the same sub-region can have the same range.

[0052] Specifically, the computer device divides the region containing the corner point based on its position and the shape of the sorting reference point; the computer device further divides the region containing the corner point into at least two sub-regions based on the pixel region contour associated with the pixel region. Figure 3 Taking the sorting reference point shown as an example, the outline of the pixel region related to the pixel point region is a ⊕ shape, and the area where the corner point is located can be... Figure 3 The circle in the middle. Figure 3 It contains 4 sub-regions.

[0053] It's understandable that the sub-regions can be divided in other ways. For example, if the sorting reference point is a circle containing alternating black and white stripes, then the outline of the pixel region related to the pixel point region would be an outline with a circular pattern within a circle. The sub-regions would then be different rings.

[0054] Step (a3) ​​determines the pixel similarity of each sub-region in at least two sub-regions.

[0055] The pixel similarity of a sub-region is calculated based on the pixel values ​​of the pixels within that sub-region. The pixel similarity of a sub-region can be calculated using cross-entropy.

[0056] Specifically, the computer device acquires sampling points in each of at least two sub-regions, and determines the pixel similarity of each sub-region based on the pixels of the sampling points in each sub-region.

[0057] Step (a4): When there are target corner points whose pixel similarity in each sub-region meets the preset similarity conditions, the position of the sorting reference point is obtained based on the target corner points.

[0058] The condition of satisfying the preset similarity means that the pixel values ​​within the sub-region are within the same pixel value range. The preset similarity condition can be, for example, that the sum of the pixel similarities of sub-regions with the same pixel value range is greater than the preset similarity, or that the sum of the similarities of sub-regions with the same pixel value range is less than the preset similarity, etc.

[0059] Specifically, the target corner point is obtained when the pixel similarity of each sub-region meets the preset similarity condition. The computer device can then use the location of this target corner point as the location of the sorting reference point.

[0060] In this embodiment, pixel similarity within the same pixel value range is determined based on the pixel similarity of each sub-region; when there are target corner points whose pixel similarity within the same pixel value range all satisfy the corresponding preset similarity conditions, the position of the sorting reference point is obtained based on the target corner point. Figure 3 Taking the sorting reference point as an example, the computer device adds the pixel similarity scores of two black regions to obtain the pixel similarity score for the range of black pixel values. Similarly, the computer device adds the pixel similarity scores of two white regions to obtain the pixel similarity score for the range of white pixel values. Therefore, when the pixel similarity score for the range of black pixel values ​​is less than 1, and the pixel similarity score for the range of white pixel values ​​is greater than 5, it means that the pixel similarity scores for the same range of pixel values ​​all meet the corresponding preset similarity conditions, and the position of the sorting reference point can be obtained based on the target corner point.

[0061] In this embodiment, since the sorting reference point contains at least two pixel value ranges, it is possible to determine whether a corner point is a corner point of the sorting reference point by detecting the pixel value ranges around the corner point. After extracting the corner point, based on the corner point position and the pixel region contour related to the pixel region, the area where the corner point is located is divided, thus creating sub-regions with different pixel value ranges. Then, by determining the pixel similarity of each sub-region, when the pixel similarity meets the similarity condition, it indicates that the corner point is a corner point of the sorting reference point, thereby obtaining the position of the sorting reference point and improving the accuracy of camera calibration.

[0062] In one embodiment, obtaining the position of the sorting reference point based on the target corner point includes: performing binarization processing on the sampling points in the sub-region corresponding to the target corner point to obtain the binarized sampling points; the sampling points are located at the edge of the sub-region; sequentially detecting the binarized sampling points along the edge of the sub-region; and when the binarized sampling points meet the pixel value jump condition, using the position of the target corner point as the position of the sorting reference point.

[0063] Binarization refers to classifying pixel values ​​to either 0 or 255. Pixel value jump conditions refer to the occurrence of a predetermined number of pixel value jumps during the detection process. Target corner points can be the center coordinates of the sorting reference points.

[0064] Specifically, the computer device selects sampling points from the edge of the sub-region corresponding to the target corner point; these sampling points are then binarized to obtain binarized sampling points. The computer device sequentially detects the binarized sampling points along the edge of the sub-region. When a predetermined number of pixel value jumps occur in a binarized sampling point, the position of the target corner point is used as the position of the sorting reference point. Figure 3 Taking the sorting reference point as an example, the sampling point can be at Figure 3 Eight sampling points are selected from the curved edges of the four sub-regions, resulting in a total of 32 sampling points. A transition from 0 to 255 is recorded as 1, and a transition from 255 to 0 is recorded as -1. After binarizing the 32 sampling points, they are sequentially detected. When a transition in the form [-1, 1, -1, 1] or [1, -1, 1, -1] is found, the position of the target corner point is considered the position of the sorting reference point.

[0065] In this embodiment, the sampling points in the sub-region corresponding to the target corner point are binarized, and the sampling points are detected sequentially along the edge of the sub-region. This can quickly detect whether the pixel value has changed. When the binarized sampling point meets the pixel value change condition, it means that the region where the target corner point is located meets the characteristics of the pixel value range of the sorting reference point, so as to accurately determine the position of the sorting reference point.

[0066] In one embodiment, the sorting reference points include a first reference point, a second reference point, and a third reference point; the topological structure of the sorting reference points on the calibration plate is a right triangle.

[0067] The marker points in the calibration image are sorted according to the positions of the sorting reference points to obtain the sorted marker points, including:

[0068] Determine the positions of the first reference point, the second reference point, and the third reference point based on the positions of the sorting reference points;

[0069] The rectangular coordinate system formed by the right triangle is determined based on the positions of the first reference point, the second reference point, and the third reference point.

[0070] Based on the sorting identifier of at least one of the sorting reference points and the rectangular coordinate system, the marker points in the calibration image are sorted to obtain the sorted marker points.

[0071] The first, second, and third reference points are all distinct. The topological structure of the sorting reference points on the calibration board is a right triangle. That is, the sorting reference points are the three points of this right triangle, and the three sides of this right triangle are not of equal length. The first, second, and third reference points have definite meanings. For example, the first reference point is the vertex of the right angle of the right triangle, the second reference point is the endpoint of the longer side of the right triangle excluding the vertex, and the third reference point is the endpoint of the shorter right-angled side of the right triangle excluding the vertex.

[0072] A sorting identifier is used to uniquely identify the marker point. The sorting identifier can consist of at least one of numbers, letters, symbols, and characters. It is understood that, in this embodiment, after determining the positions of the first reference point, the second reference point, and the third reference point, the marker points can be sorted according to at least one of the sorting identifiers of the first reference point, the second reference point, and the third reference point.

[0073] Specifically, the computer equipment determines the positions of the first reference point, the second reference point, and the third reference point based on the positions of the sorting reference points in the calibration image. The computer equipment then determines a rectangular coordinate system formed by the right triangles based on the positions of the first, second, and third reference points. For example, the right-angled vertex of the right triangle is the origin of the rectangular coordinate system, and the other two vertices of the right triangle represent the two directions of the rectangular coordinate system. The computer equipment sorts the markers in the calibration image based on the sorting identifier of at least one sorting reference point, combined with the rectangular coordinate system and the number of markers in each row or column of the calibration image, to obtain the sorted markers.

[0074] In this embodiment, after determining the positions of the sorting reference points, it is necessary to distinguish which specific sorting reference point is being used to determine the rectangular coordinate system formed by the right triangles. Then, the sorting can be performed based on the sorting identifier of at least one of the sorting reference points and the rectangular coordinate system to obtain the sorted marker points. This allows for the preparation and rapid completion of the marker point sorting, thereby improving the accuracy of the calibration.

[0075] In one embodiment, one of the two legs of the right triangle passes through the target marker. That is, this leg is the longer leg of the right triangle. Furthermore, the other leg of the right triangle does not pass through any marker; that is, this leg is the shorter leg of the right triangle. Therefore, the hypotenuse of the right triangle also does not pass through any marker. The number of target markers is unlimited.

[0076] The positions of the first reference point, the second reference point, and the third reference point are determined based on the positions of the sorting reference points, including:

[0077] Step (b1): Based on the position of the sorting reference points, connect each sorting reference point in pairs to obtain line segments.

[0078] The line segment in question is a side of the right triangle.

[0079] Specifically, the computer equipment connects the first reference point and the second reference point, connects the second reference point and the third reference point, and connects the first reference point and the third reference point according to the position of the sorting reference point, thus obtaining three line segments.

[0080] Step (b2): When a target line segment passing through the target marker is detected in the line segment, the sorting reference point outside the target line segment is determined as the third reference point, and the position of the third reference point is obtained.

[0081] The target marker can be any marker in the calibration image. The third reference point refers to the endpoint of the shorter leg of the right triangle, excluding the vertex.

[0082] Specifically, the computer equipment identifies landmarks in the calibration image. The computer equipment performs edge extraction on the image to identify the landmarks. When the landmarks are regular in shape, such as ellipses or squares, ellipse fitting methods can be used to obtain their shape parameters. Taking an ellipse as an example, the computer equipment can obtain the ellipse's center coordinates, major and minor axes, and tilt angle.

[0083] When there is a target line segment among the three line segments that passes through the target marker, it means that the target line segment is the longer side of a right triangle. Therefore, the sorting reference point outside the target line segment can be determined as the third reference point, and the position of the third reference point can be obtained.

[0084] Step (b3): ​​Starting from the third reference point, determine the reference vector formed by the third reference point and the target marker, the first vector formed by the third reference point and the first reference point, and the second vector formed by the third reference point and the second reference point.

[0085] Step (b4) determines the first cross product between the first vector and the reference vector, and the second cross product between the second vector and the reference vector.

[0086] Specifically, the cross product, also known as the vector product, results in a vector. The cross product is used to indicate the orientation of the result.

[0087] Step (b5): Determine the position of the first reference point based on the first cross product, and determine the position of the second reference point based on the second cross product.

[0088] Specifically, the first and second cross products have different signs. For example, the first cross product is greater than zero, while the second cross product is less than zero. Therefore, the computer can determine the position of the first reference point based on the first cross product and the position of the second reference point based on the second cross product.

[0089] In this embodiment, since the calibration plate is set as a right triangle with one of its two legs passing through the target marker point, by connecting each sorting reference point in pairs, the sorting reference point outside the target line segment can be determined as the third reference point. Starting from the third reference point, different vectors are constructed, and the positions of the first and second reference points can be determined by the cross product. Therefore, a rectangular coordinate system can be accurately constructed, improving the sorting efficiency of the marker points and thus improving the calibration efficiency.

[0090] In one embodiment, the detection method for a target line segment passing through a target marker includes: for each line segment, acquiring multiple points on the line segment; searching for the nearest marker point to the line segment and determining the shape equation corresponding to the nearest marker point; when the points on the line segment are within the range represented by the shape equation, determining the line segment as a target line segment passing through the target marker point.

[0091] In this context, the line segment represents the side of the triangle. The shape equation is the equation determined by the shape of the marker point. For example, if the marker point is elliptical, the shape equation is the ellipse equation. If the marker point is square, the shape equation is the square equation.

[0092] Specifically, for each line segment, the computer device can divide the line segment into a preset number of parts, obtaining multiple points on the line segment. The computer device searches for the nearest marker point to the line segment and determines the shape equation corresponding to the nearest marker point. When any point on the line segment is within the range represented by the shape equation, it means that any point on the line segment falls within the marker point, and the computer device determines that the line segment is a target line segment passing through the target marker point. When a point on the line segment is not within the range represented by the shape equation, the line segment is not a target line segment passing through the target marker point.

[0093] In this embodiment, since the target line segment passes through the target marker, the marker closest to the line segment is the target marker. Therefore, the target line segment can be determined by detecting whether the nearest marker is on the line segment. For each line segment, multiple points on the line segment are obtained, and the nearest marker is searched for. The shape equation corresponding to the nearest marker is determined. When a point on the line segment is within the range represented by the shape equation, it means that the line segment passes through the marker, and it also means that the marker is on the line segment. Therefore, the line segment is the target line segment that passes through the target marker, thereby enabling the rapid determination of the Cartesian coordinate system and improving the efficiency of marker sorting.

[0094] In one embodiment, the marker points in the calibration image are sorted according to the sorting identifier of at least one of the sorting reference points and a Cartesian coordinate system to obtain the sorted marker points, including:

[0095] Step (c1): For any reference point in the sorted reference points, search for the neighboring points of the reference point.

[0096] Among them, the neighboring point can be a four-neighbor point or an eight-neighbor point.

[0097] Specifically, for each of at least one of the sorting reference points, search for the four-neighbor or eight-neighbor points of the reference point. Preferably, search for the four-neighbor points.

[0098] Step (c2): For each neighboring point of the reference point, determine the target direction of the neighboring point relative to the reference point in a rectangular coordinate system.

[0099] The target direction can refer to the direction represented by a Cartesian coordinate system. For example, it can be, but is not limited to, the negative x-axis, positive x-axis, positive y-axis, and negative y-axis of the Cartesian coordinate system.

[0100] Specifically, for each neighboring point of the reference point, the target direction of each neighboring point relative to the reference point in the Cartesian coordinate system is determined. For example, if the reference point is point A, the four neighboring points of point A are the points above, below, to the left, and to the right of point A. Then, the point above point A is the point in the positive y-axis direction of the Cartesian coordinate system, the point below point A is the point in the negative y-axis direction, the point to the left of point A is the point in the negative x-axis direction, and the point to the right of point A is the point in the positive x-axis direction.

[0101] In this embodiment, for each neighboring point of the reference point, the computer device determines the inner product between the neighboring point and the reference point. The computer device obtains the inner product of each direction represented by the Cartesian coordinate system; it matches the inner product between the neighboring point and the reference point with the inner product of the directions represented by the Cartesian coordinate system to obtain the target direction of the neighboring point relative to the reference point.

[0102] Step (c3): Obtain the reference sorting identifier of the reference point.

[0103] Specifically, the reference sorting identifier is used to uniquely identify the reference point. The computer device obtains the reference sorting identifier of the reference point based on its location.

[0104] Step (c4): Determine the sorting identifier of the neighboring points based on the reference sorting identifier and the target direction.

[0105] Specifically, there is a sorting rule corresponding to the target direction. For example, if the target direction is the positive x-axis, then the sorting identifier is decremented by 1. The computer device determines the sorting identifier of the neighboring points based on the reference sorting identifier and the sorting rule corresponding to the target direction.

[0106] Step (c5) involves using the existing sorted markers as reference points and performing a search for neighboring points of the reference points until all markers in the calibration image are traversed to obtain the sorted markers in the calibration image.

[0107] Specifically, the existing sorting markers can be sorting reference points or neighboring points of existing sorting markers. The computer device uses the existing sorting markers as reference points and repeatedly performs the step of searching for neighboring points of the reference points until all markers in the calibration image are traversed, thus obtaining the sorted markers in the calibration image.

[0108] In this embodiment, by searching for neighboring points of a reference point and determining the target direction of the neighboring points relative to the reference point in a Cartesian coordinate system, the sorting identifier of the neighboring points is determined, which enables the rapid sorting of the marker points. Using the marker points with existing sorting identifiers as reference points, the search for neighboring points of the reference point is repeated until all marker points in the calibration image are traversed to obtain the sorted marker points. This allows for further sorting using multiple sorted marker points, improving sorting efficiency.

[0109] In one embodiment, a camera calibration method includes:

[0110] Step (d1): Obtain a calibration image; the calibration image includes marker points; the marker points include sorting reference points; the sorting reference points contain pixels with at least two pixel value ranges; the sorting reference points include a first reference point, a second reference point, and a third reference point; the topological structure of the sorting reference points on the calibration board is a right triangle; one of the two legs of the right triangle passes through the target marker point.

[0111] Step (d2): Extract the set of corner points from the calibration image; the set of corner points includes at least two corner points.

[0112] Step (d3) involves dividing the region containing the corner point into at least two sub-regions based on the corner point position and the pixel region contour associated with the pixel point region.

[0113] Step (d4) determines the pixel similarity of each sub-region in at least two sub-regions.

[0114] Step (d5): When there is a target corner point whose pixel similarity in each sub-region meets the preset similarity condition, the sampling points in the sub-region corresponding to the target corner point are binarized to obtain the binarized sampling points; the sampling points are located at the edge of the sub-region.

[0115] Step (d6): Detect the binarized sampling points sequentially along the edge of the sub-region. When the binarized sampling points meet the pixel value jump condition, use the position of the target corner point as the position of the sorting reference point.

[0116] Step (d7): Based on the position of the sorting reference points, connect each sorting reference point in pairs to obtain line segments.

[0117] Step (d8): For each line segment, obtain multiple points on the line segment.

[0118] Step (d9): Search for the nearest marker point to the line segment and determine the shape equation corresponding to the nearest marker point.

[0119] Step (d10): When a point on a line segment is within the range represented by the shape equation, the line segment is determined to be a target line segment that passes through the target marker point.

[0120] Step (d11): When there is a target line segment in the line segment that passes through the target marker, determine the sorting reference point outside the target line segment as the third reference point and obtain the position of the third reference point.

[0121] Step (d12): Starting from the third reference point, determine the reference vector formed by the third reference point and the target marker, the first vector formed by the third reference point and the first reference point, and the second vector formed by the third reference point and the second reference point.

[0122] Step (d13) determines the first cross product between the first vector and the reference vector, and the second cross product between the second vector and the reference vector.

[0123] Step (d14): Determine the position of the first reference point based on the first cross product, and determine the position of the second reference point based on the second cross product.

[0124] Step (d15): Determine the rectangular coordinate system formed by the right triangles based on the positions of the first reference point, the second reference point, and the third reference point.

[0125] Step (d16): For any reference point in the sorted reference points, search for the neighboring points of the reference point.

[0126] Step (d17): For each neighboring point of the reference point, determine the target direction of the neighboring point relative to the reference point in a rectangular coordinate system.

[0127] Step (d18): Obtain the reference sorting identifier of the reference point.

[0128] Step (d19): Determine the sorting identifier of the neighboring points based on the reference sorting identifier and the target direction.

[0129] Step (d20) involves using the existing sorted markers as reference points and performing a search for neighboring points of the reference points until all markers in the calibration image are traversed to obtain the sorted markers in the calibration image.

[0130] Step (d21): Perform camera calibration based on the sorted marker points.

[0131] In this embodiment, since the sorting reference point contains at least two boundary lines formed by pixel regions, and these boundary lines intersect, it is possible to determine whether a corner point is a sorting reference point by detecting the range of pixel values ​​around the corner point. Therefore, after extracting the corner point, based on its position and the pixel region contour related to the pixel region, the region containing the corner point is divided, thus defining regions with different pixel value ranges. The position of the sorting reference point is accurately determined using both pixel similarity and pixel value jump methods. A Cartesian coordinate system is established based on the positions of each reference point, and the points are sorted. This allows for rapid sorting of marker points in the calibration image, thereby improving the accuracy of camera calibration.

[0132] In one embodiment, the example is given by having three sorting reference points on the calibration board: point A as the first reference point, point B as the second reference point, and point C as the third reference point.

[0133] Calibration plate rules:

[0134] 1. The topological relationship of the three sorting reference points on the calibration board is a right triangle.

[0135] 2. Adjacent to the marker points that serve as the shorter right-angled side

[0136] 3. The two markers on the longer right-angled side are not adjacent and are separated by one marker. The centers of the two markers and the center of the circular marker are on a straight line.

[0137] 4. The line connecting the centers of the sorting reference points located at the two endpoints of the hypotenuse will never pass through any marker points.

[0138] 5. The sorting reference point is a circle, and it is a black and white alternating fan-shaped pattern.

[0139] 6. Other markers are arranged in a straight line.

[0140] According to the design rules of the calibration plate, the marker points can be identified in the calibration image. The main idea of ​​identification is to extract corner points from the calibration image, determine the direction of the right triangle represented by the sorting reference point, and sort according to the direction of the right triangle.

[0141] First, corner points are extracted from the calibration image using corner detection and sub-pixel optimization methods to obtain a corner point set. Then, false corner points in the corner point set are removed using cross-entropy and pixel jump detection methods. The specific implementation methods for cross-entropy and pixel jump detection are as follows:

[0142] 1. Using the existing corner coordinates as the center, draw a virtual circle with a radius of 12 pixels. For example... Figure 5 The diagram shown is a schematic representation of the area where the corner point is located in one embodiment. Figure 5 The gray circle represents the area where the corner points are located. The circle is also divided into four quadrants, with 8 sampling points taken from each quadrant, resulting in a total of 32 sampling points.

[0143] 2. Substitute the pixel values ​​of the sampled points in the diagonal quadrants of the four quadrants into the cross-entropy function, defined in pixels. The cross-entropy function represents the pixel similarity between the diagonal quadrants. For example, there is one cross-entropy value for the black region and one for the white region. When the cross-entropy function of the white region is greater than 5 and the cross-entropy function of the black region is less than 1, the corresponding corner points are used in step 3.

[0144] 3. The determination of pixel transitions relies on 32 sampling points. The pixel values ​​of each sampling point on the virtual circle are binarized sequentially. After binarization, the pixels of each sampling point are sequentially judged. A transition from 0 to 255 is recorded as 1, and a transition from 255 to 0 is recorded as -1. This process is repeated for all 32 sampling points. When only transitions in the form [-1, 1, -1, 1] exist, it proves that the center coordinates of this virtual circle are corner points, which are also the coordinates of the sorting reference points. For example... Figure 6 The image shown is a schematic diagram of the target corner point in one embodiment. Figure 6 The center point of the sorting reference point is the target corner point.

[0145] 4. Determine the locations of points A, B, and C using the following method:

[0146] ① Extract edges from the calibration image and use ellipse fitting to obtain the ellipse parameters of the marker points in the calibration image. Ellipse parameters include the center coordinates, major and minor axes, and tilt angle of the ellipse.

[0147] ② The line connecting the centers of the two sorting reference points A and B on the longer right-angled side will always pass through a target marker point M. The other point on the shorter right-angled side can be designated as point C. For example... Figure 7The diagram shown illustrates the labeling of points A, B, C, and M in one embodiment. When determining point C, the line connecting the centers of two sorting reference points is divided into 10 equal parts, resulting in 10 right-angled edge points. Simultaneously, within a range twice the distance between A and B, the nearest marker point is searched, and the equation of an ellipse is used to determine if any right-angled edge point falls within the ellipse. If a right-angled edge point falls within the ellipse, the other point outside the right-angled edge is identified as point C.

[0148] ③ Starting from point C, arbitrarily select a sorting reference point A, and set A as a point on the shorter right-angled side. The cross product of vectors CA and CM is greater than 0; if the sorting reference point is B, the cross product of vector CB and CM will be less than 0. Therefore, the positions of points A and B can be determined.

[0149] 5. After obtaining the major and minor legs of the triangle formed by the sorting reference points, the ellipses can be sorted according to their directions. Assuming that AC is the minor leg and AB is the major leg, the extracted ellipses are sorted according to AC and AB respectively. The specific sorting algorithm is as follows:

[0150] ① Taking the AC direction as an example, find the four neighboring points of point A. Simultaneously, based on the inner product of point A and its four neighboring points, determine the points along the AC and AB directions. Sort the points along the AC and AB directions to obtain the points with existing sorting labels.

[0151] ② Based on the marked points, continue searching outwards in directions AC and AB to obtain the four neighboring points of the marked points. Determine the direction of the points using the method in step ①, and sort the points according to their index.

[0152] ③ Repeat the above steps until all markers are sorted. For example... Figure 8 The image shown is a schematic diagram of the sorted marker points in one embodiment. Figure 8 The sorting reference points are the points labeled 31, 40, and 42.

[0153] It should be understood that, although the above Figure 2 In the flowchart, the steps are shown sequentially according to the arrows. Steps (a1) to (a4), (b1) to (b5), (c1) to (c5), and (d1) to (d21) are shown sequentially according to their numbers. However, these steps are not necessarily executed in the order indicated by the arrows or numbers. Unless explicitly stated herein, there is no strict order requirement for the execution of these steps; they can be executed in other orders. Figure 2At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0154] In one embodiment, such as Figure 9 The diagram shown is a structural block diagram of a camera calibration device in one embodiment. Figure 9 A camera calibration device is provided, which can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: an image acquisition module 902, a position determination module 904, a marker point sorting module 906, and a calibration module 908, wherein:

[0155] Image acquisition module 902 is used to acquire a calibration image; the calibration image includes marker points; the marker points include sorting reference points; the sorting reference points contain at least two boundary lines formed by pixel regions, and there are intersecting boundary lines;

[0156] The position determination module 904 is used to determine the position of the sorting reference point based on the set of corner points extracted from the calibration image;

[0157] The marker sorting module 906 is used to sort the markers in the calibration image according to the position of the sorting reference point to obtain the sorted markers.

[0158] The calibration module 908 is used to calibrate the camera based on the sorted marker points.

[0159] In this embodiment, a calibration image is acquired, which contains sorting reference points. Since the sorting reference points contain at least two boundary lines formed by pixel regions and there are intersecting boundary lines, that is, the sorting reference points contain corner points, the corner point set can be extracted from the calibration image based on this feature to determine the position of the sorting reference points. Then, the marker points are sorted according to the position of the sorting reference points, thereby improving the accuracy of camera calibration.

[0160] In one embodiment, the position determination module 904 is used to extract a set of corner points in the calibration image; the set of corner points includes at least two corner points; based on the corner point position, the region where the corner point is located is divided into at least two sub-regions according to the pixel region contour related to the pixel point region; the pixel similarity of each sub-region in the at least two sub-regions is determined; when there is a target corner point where the pixel similarity of each sub-region meets the preset similarity condition, the position of the sorting reference point is obtained according to the target corner point.

[0161] In this embodiment, since the sorting reference point contains at least two pixel value ranges, it is possible to determine whether a corner point is a corner point of the sorting reference point by detecting the pixel value ranges around the corner point. After extracting the corner point, based on its position and the pixel region contour related to the pixel region, the area where the corner point is located is divided, thus dividing regions with different pixel value ranges. Then, by determining the pixel similarity of each sub-region, when the pixel similarity meets the similarity condition, it indicates that the corner point is a corner point of the sorting reference point, thereby obtaining the position of the sorting reference point and improving the accuracy of camera calibration.

[0162] In one embodiment, the position determination module 904 is further configured to perform binarization processing on the sampling points in the sub-region corresponding to the target corner point to obtain the binarized sampling points; the sampling points are located at the edge of the sub-region; the binarized sampling points are detected sequentially along the edge of the sub-region; when the binarized sampling points meet the pixel value jump condition, the position of the target corner point is used as the position of the sorting reference point.

[0163] In this embodiment, the sampling points in the sub-region corresponding to the target corner point are binarized, and the sampling points are detected sequentially along the edge of the sub-region. This can quickly detect whether the pixel value has changed. When the binarized sampling point meets the pixel value change condition, it means that the region where the target corner point is located meets the characteristics of the pixel value range of the sorting reference point, so as to accurately determine the position of the sorting reference point.

[0164] In one embodiment, the sorting reference points include a first reference point, a second reference point, and a third reference point; the topological structure of the sorting reference points on the calibration board is a right-angled triangle. The marker point sorting module 906 is further configured to: determine the positions of the first reference point, the second reference point, and the third reference point based on the positions of the sorting reference points; determine the Cartesian coordinate system formed by the right-angled triangle based on the positions of the first reference point, the second reference point, and the third reference point; and sort the marker points in the calibration image according to the sorting identifier of at least one of the sorting reference points and the Cartesian coordinate system to obtain the sorted marker points.

[0165] In this embodiment, after determining the positions of the sorting reference points, it is necessary to distinguish which specific sorting reference point is being used to determine the rectangular coordinate system formed by the right triangles. Then, the sorting can be performed based on the sorting identifier of at least one of the sorting reference points and the rectangular coordinate system to obtain the sorted marker points. This allows for the preparation and rapid completion of the marker point sorting, thereby improving the accuracy of the calibration.

[0166] In one embodiment, one of the two legs of the right triangle passes through the target marker point. The marker point sorting module 906 is further configured to connect each sorting reference point pairwise to obtain line segments based on the positions of the sorting reference points; when a target line segment passing through the target marker point is detected in the line segment, a sorting reference point outside the target line segment is determined as a third reference point, and the position of the third reference point is obtained; starting from the third reference point, a reference vector formed by the third reference point and the target marker point, a first vector formed by the third reference point and the first reference point, and a second vector formed by the third reference point and the second reference point are determined; a first cross product between the first vector and the reference vector is determined, and a second cross product between the second vector and the reference vector is determined; the position of the first reference point is determined based on the first cross product, and the position of the second reference point is determined based on the second cross product.

[0167] In this embodiment, since the calibration plate is set as a right triangle with one of its two legs passing through the target marker point, by connecting each sorting reference point in pairs, the sorting reference point outside the target line segment can be determined as the third reference point. Starting from the third reference point, different vectors are constructed, and the positions of the first and second reference points can be determined by the cross product. Therefore, a rectangular coordinate system can be accurately constructed, improving the sorting efficiency of the marker points and thus improving the calibration efficiency.

[0168] In one embodiment, the marker sorting module 906 is further configured to: obtain multiple points on each line segment; search for the nearest marker point to the line segment; determine the shape equation corresponding to the nearest marker point; and determine the line segment as a target line segment that passes through the target marker point when the points on the line segment are within the range represented by the shape equation.

[0169] In this embodiment, since the target line segment passes through the target marker, the marker closest to the line segment is the target marker. Therefore, the target line segment can be determined by detecting whether the nearest marker is on the line segment. For each line segment, multiple points on the line segment are obtained, and the nearest marker is searched for. The shape equation corresponding to the nearest marker is determined. When a point on the line segment is within the range represented by the shape equation, it means that the line segment passes through the marker, and it also means that the marker is on the line segment. Therefore, the line segment is the target line segment that passes through the target marker, thereby enabling the rapid determination of the Cartesian coordinate system and improving the efficiency of marker sorting.

[0170] In one embodiment, the marker sorting module 906 is further configured to: search for neighboring points of any reference point among the sorting reference points; determine the target direction of each neighboring point relative to the reference point in a Cartesian coordinate system; obtain the reference sorting identifier of the reference point; determine the sorting identifier of the neighboring points based on the reference sorting identifier and the target direction; and search for neighboring points of the reference point using marker points with existing sorting identifiers as reference points, until all marker points in the calibration image are traversed to obtain the sorted marker points in the calibration image.

[0171] In this embodiment, by searching for neighboring points of a reference point and determining the target direction of the neighboring points relative to the reference point in a Cartesian coordinate system, the sorting identifier of the neighboring points is determined, which enables the rapid sorting of the marker points. Using the marker points with existing sorting identifiers as reference points, the search for neighboring points of the reference point is repeated until all marker points in the calibration image are traversed to obtain the sorted marker points. This allows for further sorting using multiple sorted marker points, improving sorting efficiency.

[0172] For specific limitations regarding the camera calibration device, please refer to the limitations on the camera calibration method above, which will not be repeated here. Each module in the aforementioned camera calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0173] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a camera calibration method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0174] Those skilled in the art will understand that Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method embodiments.

[0176] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method embodiments.

[0177] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0178] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes described in the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0179] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A camera calibration method, characterized in that, The method includes: Obtain a calibration image; the calibration image includes marker points; the marker points include sorting reference points; the sorting reference points contain at least two boundary lines formed by pixel regions, and there are intersecting boundary lines; The position of the sorting reference point is determined based on the set of corner points extracted from the calibration image; the sorting reference point includes a first reference point, a second reference point, and a third reference point; the topological structure of the sorting reference point on the calibration board is a right triangle. The positions of the first reference point, the second reference point, and the third reference point are determined based on the positions of the sorting reference points. Based on the positions of the first reference point, the second reference point, and the third reference point, the rectangular coordinate system formed by the right triangle is determined; For any reference point among the sorting reference points, search the neighborhood points of the reference point; For each neighboring point of the reference point, the target direction of the neighboring point relative to the reference point is determined in the Cartesian coordinate system. Obtain the reference sorting identifier of the reference point; The sorting identifier of the neighboring points is determined based on the reference sorting identifier and the target direction; Using the existing sorted markers as reference points, the step of searching for the neighboring points of the reference points is performed until all the markers in the calibration image are traversed to obtain the sorted markers in the calibration image. Camera calibration is performed based on the sorted marker points.

2. The method according to claim 1, characterized in that, Determining the position of the sorting reference point based on the set of corner points extracted from the calibration image includes: Extract the set of corner points from the calibration image; the set of corner points includes multiple corner points; Based on the corner point position, and according to the pixel region contour related to the pixel point region, the region where the corner point is located is divided into at least two sub-regions; Determine the pixel similarity of each sub-region in the at least two sub-regions; When there is a target corner point whose pixel similarity in each of the sub-regions meets the preset similarity condition, the position of the sorting reference point is obtained based on the target corner point.

3. The method according to claim 2, characterized in that, The step of obtaining the position of the sorting reference point based on the target corner point includes: The sampling points in the sub-region corresponding to the target corner point are binarized to obtain the binarized sampling points; the sampling points are located at the edge of the sub-region. The binarized sampling points are sequentially detected along the edge of the sub-region. When the binarized sampling points meet the pixel value jump condition, the position of the target corner point is used as the position of the sorting reference point.

4. The method according to claim 1, characterized in that, One of the two legs of the right triangle passes through the target marker point; Determining the positions of the first reference point, the second reference point, and the third reference point based on the positions of the sorting reference points includes: Based on the positions of the sorting reference points, line segments are obtained by connecting each pair of the sorting reference points; When a target line segment passing through the target marker point is detected in the line segment, the sorting reference point outside the target line segment is determined as the third reference point, and the position of the third reference point is obtained; Starting from the third reference point, determine the reference vector formed by the third reference point and the target marker, the first vector formed by the third reference point and the first reference point, and the second vector formed by the third reference point and the second reference point. Determine the first cross product between the first vector and the reference vector, and determine the second cross product between the second vector and the reference vector; The position of the first reference point is determined based on the first cross product, and the position of the second reference point is determined based on the second cross product.

5. The method according to claim 4, characterized in that, The method for detecting target line segments that pass through target markers includes: For each line segment, obtain multiple points on the line segment; Search for the nearest marker point to the line segment and determine the shape equation corresponding to the nearest marker point; When a point on the line segment is within the range represented by the shape equation, the line segment is determined to be a target line segment that passes through the target marker point.

6. A camera calibration device, characterized in that, The apparatus is used to implement the method according to any one of claims 1 to 5, the apparatus comprising: An image acquisition module is used to acquire a calibration image; the calibration image includes marker points; the marker points include sorting reference points; the sorting reference points contain at least two boundary lines formed by pixel regions, and there are intersecting boundary lines. The position determination module is used to determine the position of the sorting reference point based on the set of corner points extracted from the calibration image; the sorting reference point includes a first reference point, a second reference point, and a third reference point; the topological structure of the sorting reference point on the calibration board is a right triangle; The marker sorting module is used for: The positions of the first reference point, the second reference point, and the third reference point are determined based on the positions of the sorting reference points. Based on the positions of the first reference point, the second reference point, and the third reference point, the rectangular coordinate system formed by the right triangle is determined; For any reference point among the sorting reference points, search the neighborhood points of the reference point; For each neighboring point of the reference point, the target direction of the neighboring point relative to the reference point is determined in the Cartesian coordinate system. Obtain the reference sorting identifier of the reference point; The sorting identifier of the neighboring points is determined based on the reference sorting identifier and the target direction; Using the existing sorted markers as reference points, the step of searching for the neighboring points of the reference points is performed until all the markers in the calibration image are traversed to obtain the sorted markers in the calibration image. The calibration module is used to calibrate the camera based on the sorted marker points.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images