Dimension calibration and measurement method and device of pavement detection frame and storage medium

Abstract

The invention discloses a dimension calibration and measurement method and device for a pavement detection frame and a storage medium, and the method comprises the steps: dividing a collected calibration image, and obtaining four sub-calibration images and first vertex coordinates corresponding to the four sub-calibration images; calculating perspective transformation matrixes corresponding to the four sub-calibration images through the first vertex coordinates and the acquired perspective coordinates of the calibration images; acquiring four groups of images of the four sub-calibration images after perspective correction by utilizing the perspective transformation matrix, and acquiring transverse and longitudinal conversion proportions corresponding to the four groups of images respectively; determining a target image according to the distribution condition of the second vertex coordinates of the to-be-detected frame in the four sub-calibration images; the coordinates of the to-be-detected frame in the overlook view angle are obtained based on the perspective transformation matrix corresponding to the target image, area calculation is carried out based on the coordinates and the transverse-longitudinal conversion proportion, the actual size of the to-be-detected frame is obtained, and the accuracy and efficiency of size measurement of the detection frame are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing size measurement, and relates to, but is not limited to, methods, devices, and storage media for calibrating and measuring the size of a road surface detection frame. Background Art

[0002] With the rapid development of computer vision and deep learning technologies, image-based planar size measurement and calibration technologies have gradually attracted attention due to their advantages such as non-contact, high precision, and high efficiency. By training a deep neural network, in the environmental perception system of intelligent connected vehicles, the features of road surface images can be automatically extracted through a camera, and high-precision target detection can be achieved. On this basis, by combining the pixel coordinates of the detected target area in the image, image distortion can be effectively eliminated through calibration and perspective coordinate transformation, and multiple sets of conversion ratios are designed for intelligent judgment and calculation, enabling rapid and accurate acquisition of planar size information.

[0003] In related technologies, the size information of the road surface detection frame is obtained through a complex camera calibration technology to obtain the internal parameters, external parameters, and distortion coefficients of the camera, etc. However, this process is not only cumbersome in operation, but also often requires high-precision measurement equipment and stable environmental conditions, resulting in problems such as low accuracy of the obtained size information of the road surface detection frame, low efficiency, and high cost.

[0004] Therefore, how to quickly and accurately obtain the size information of the road surface detection frame has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a method for calibrating and measuring the size of a road surface detection frame, which at least solves the problems of low accuracy, low efficiency, and high cost of the size information of the road surface detection frame in related technologies.

[0006] According to the first aspect of the embodiment of the present invention, a method for calibrating and measuring the size of a road surface detection frame is provided, including:

[0007] Dividing the collected calibration image to obtain four sub-calibration images and their respective corresponding first vertex coordinates;

[0008] Calculating the perspective transformation matrix corresponding to each of the four sub-calibration images through the first vertex coordinates and the perspective coordinates of the obtained calibration image;

[0009] Using the perspective transformation matrix to obtain four sets of images after perspective correction of the four sub-calibration images, and obtaining the horizontal and vertical conversion ratios corresponding to each of the four sets of images;

[0010] Determining the target image based on the distribution of the second vertex coordinates of the frame to be detected in the four sub-calibration images;

[0011] Obtain the coordinates of the detection box to be detected in the top-down view based on the perspective transformation matrix corresponding to the target image, and calculate the area based on the coordinates and the horizontal and vertical conversion ratios to obtain the actual size of the detection box to be detected.

[0012] According to a second aspect of an embodiment of the present invention, there is provided an electronic device, including: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the method described in the first aspect.

[0013] According to a third aspect of an embodiment of the present invention, there is provided a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method described in the first aspect.

[0014] According to the solution provided by the embodiment of the present invention, the collected calibration image is divided to obtain four sub-calibration images and their respective corresponding first vertex coordinates; the calibration image is used to calibrate the parameters of the first image acquisition device in the vehicle; the perspective transformation matrix corresponding to each of the four sub-calibration images is calculated through the first vertex coordinates and the perspective coordinates of the obtained calibration image; four sets of images after perspective correction of the four sub-calibration images are obtained by using the perspective transformation matrix, and the horizontal and vertical conversion ratios corresponding to the four sets of images are obtained; the target image is determined by the distribution of the second vertex coordinates of the detection box to be detected in the four sub-calibration images; the detection box to be detected is generated based on the first image acquisition device; the coordinates of the detection box to be detected in the top-down view are obtained based on the perspective transformation matrix corresponding to the target image, and the actual size of the detection box to be detected is obtained based on the coordinates and the horizontal and vertical conversion ratios. In this method, the four sub-calibration images can be perspectively corrected through the perspective transformation matrix, that is, the calibration image can be perspectively corrected to obtain four sets of corrected images and the corresponding horizontal and vertical conversion ratios. Then, it is only necessary to determine the target image corresponding to the detection box to be detected in the four sub-calibration images, and the actual size of the detection box to be detected can be determined based on the perspective transformation matrix corresponding to the target image and the corresponding horizontal and vertical conversion ratios, without performing operations such as calibrating the image acquisition device, improving the accuracy and efficiency of measuring the size of the detection box. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings, where:

[0016] Figure 1Schematic flowchart of the method for calibrating and measuring the size of the road surface detection frame provided by the embodiment of the present invention;

[0017] Figure 2 Schematic diagram of the effects of four sub-calibration images after the calibration image is divided provided by the embodiment of the present invention;

[0018] Figure 3 Schematic diagram of the effect of obtaining the calibration image provided by the embodiment of the present invention;

[0019] Figure 4 Schematic diagram of the effect of obtaining the perspective coordinates when the detection frame to be detected straddles the upper left and upper right images provided by the embodiment of the present invention;

[0020] Figure 5 Schematic diagram of the effect of a rectangular frame to be measured to a trapezoidal frame to be detected provided by the embodiment of the present invention;

[0021] Figure 6 Schematic diagram of the structure of an electronic device provided by the embodiment of the present invention. Detailed implementation manners

[0022] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0023] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0024] It should be noted that the terms "first\second\third" involved in the embodiments of the present invention are only used to distinguish similar objects, and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0025] Those skilled in the art can understand that, unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as the general understanding of those of ordinary skill in the art in the field to which the embodiments of the present invention belong. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0026] Figure 1 FIG. 4 is a schematic flowchart of a method for calibrating and measuring the size of a road surface detection frame provided by an embodiment of the present invention. The method for calibrating and measuring the size of the road surface detection frame provided by the embodiment of the present invention can be executed by an electronic device, and the electronic device can be, for example, a computer, a server, etc.

[0027] As Figure 1 shown, the method for calibrating and measuring the size of the road surface detection frame includes:

[0028] S101. Divide the collected calibration image to obtain four sub-calibration images and their respective corresponding first vertex coordinates.

[0029] In the embodiments of the present invention, the calibration image refers to a specific pattern or image used during the calibration process of the image acquisition device, and the imaging system of the image acquisition device is calibrated through known geometric features to eliminate errors caused by factors such as lens distortion. Among them, the image acquisition device can be a road surface detection camera of an intelligent connected vehicle, an in-vehicle camera, etc.

[0030] In the embodiments of the present invention, the center position of the calibration image can be used as a division point to divide the calibration image into four sub-calibration images, and each sub-calibration image corresponds to the first vertex coordinates. Among them, each sub-calibration image can be a rectangular image, and the first vertex coordinates can be the four vertex coordinates of the rectangular image.

[0031] Exemplarily, as Figure 2 shown, Figure 2 FIG. 5 is a schematic diagram of the effects of four sub-calibration images after the calibration image is divided according to an embodiment of the present invention. In Figure 2 , the four vertex coordinates can be marked in order according to the clockwise direction. The coordinates of the calibration image before division are A1(x 11 , y 11 ), B2(x 22 , y 22 ), C3(x 33 , y 33 ) and D4(x 44 , y 44 ). After division, four sub-calibration images are obtained, namely the upper left image, and the first vertex coordinates are: A1(x11 , y 11 ), B1(x0, y 12 ), O(x0, y0) and D1(x 14 , y0), upper right image, the coordinates of the first vertex are: A2(x0, y 12 ), B2(x 22 , y 22 ), C2(x 23 , y0) and O(x0, y0), lower right image, the coordinates of the first vertex are: O(x0, y0), B3(x 23 , y0), C3(x 33 , y 33 ), and D3(x0, y 34 ), lower left image, the coordinates of the first vertex are: A4(x 14 , y0), O(x0, y0), C4(x0, y 34 ), and D4(x 44 , y 44 ).

[0032] S102. Calculate the perspective transformation matrix corresponding to each of the four sub-calibration images through the coordinates of the first vertex and the perspective coordinates of the obtained calibration image.

[0033] In an embodiment of the present invention, the perspective transformation matrix maps points on a two-dimensional plane to another two-dimensional plane while retaining or introducing a perspective effect, and is used for image correction and view transformation, etc. The perspective coordinates of the calibration image can be calculated through the resolution of the calibration image, and finally, using the perspective transformation algorithm, the coordinates of the first vertex and the perspective coordinates are used to calculate the perspective transformation matrix corresponding to each sub-calibration image.

[0034] Exemplarily, if the resolution of the calibration image is (1920×1080), the perspective coordinates are A trans (0, 0), B trans (1920, 0), C trans (1920, 1080) and D trans (0, 1080).

[0035] S103. Use the perspective transformation matrix to obtain four sets of images of the four sub-calibration images after perspective correction, and obtain the horizontal and vertical conversion ratios corresponding to each of the four sets of images.

[0036] In an embodiment of the present invention, perspective correction aims to correct the perspective distortion of an image caused by shooting angle or viewing angle problems. Specifically, by applying a perspective transformation matrix, the perspective distortion in the image is corrected to restore it to a normal planar image. The horizontal conversion ratio refers to the dimensional ratio relationship in the width direction of the image, and the vertical conversion ratio refers to the dimensional ratio relationship in the height direction of the image. Apply the four groups of perspective transformation matrices to the corresponding sub-calibration images respectively for perspective correction to obtain four groups of images after perspective correction. Finally, calculate the horizontal and vertical conversion ratios corresponding to each of the four groups of images to obtain the corresponding horizontal and vertical conversion ratios.

[0037] S104. Determine the target image based on the distribution of the second vertex coordinates of the detection frame in the four sub-calibration images.

[0038] In an embodiment of the present invention, the detection frame is a key component in target detection and object recognition tasks. It is a rectangular frame formed by identifying and positioning an object in an image captured by an image acquisition device, such as a camera, through a target detection algorithm, and is used to mark and locate the object of interest in the image (such as a person, a vehicle, an animal, etc.). The second vertex coordinates are the four vertex coordinates of the detection frame, and the target image is the image containing the detection frame. There can be multiple target images. The second vertex coordinates of the detection frame can be compared with the coordinates of the four sub-calibration images to determine in which sub-calibration image the detection frame is located, and thus obtain the target image.

[0039] Exemplarily, divide the four sub-calibration images into the upper-left image 1, the upper-right image 2, the lower-right image 3, and the lower-left image 4. Determining the target image corresponding to the detection frame in the four sub-calibration images can include the following nine situations:

[0040] (1). all 1: The detection frame is located within the upper-left image;

[0041] (2). all 2: The detection frame is located within the upper-right image;

[0042] (3). all 3: The detection frame is located within the lower-right image;

[0043] (4). all 4: The detection frame is located within the lower-left image;

[0044] (5). 1 + 2: The detection frame spans the upper-left and upper-right images;

[0045] (6). 1 + 4: The detection frame spans the upper-left and lower-left images;

[0046] (7). 2 + 3: The detection frame spans the upper-right and lower-right images;

[0047] (8). 3 + 4: The detection frame spans the lower-right and lower-left images;

[0048] (9), 1 + 2 + 3 + 4: The detection box spans four sub - calibration images.

[0049] The specific judgment method is as follows: First, the upper - left vertex coordinates of the detection box are A(x1, y1), the upper - right vertex coordinates are: B(x2, y1), the lower - right vertex coordinates are: C(x2, y2), and the lower - left vertex coordinates are: D(x1, y2). Among them, the coordinates of the detection box can be input manually or obtained by the vehicle system. Check the vertical coordinate y2 of point C and point D of the detection box. If y2 ≤ y0, first check the horizontal coordinate x1 of point A and point D. If x1 ≤ x0, then continue to check the horizontal coordinate x2 of point B and point C. If x2 ≤ x0, it is determined as case all 1; otherwise, it is determined as case 1 + 2. If x1 > x0, it is determined as case all 2.

[0050] If y2 > y0, first check the vertical coordinate y1 of point A and point B. If y1 ≤ y0, then continue to determine the horizontal coordinate x1 of point A and point D. If x1 ≤ x0, then check the horizontal coordinate x2 of point B and point C. If x2 ≤ x0, it is determined as case 1 + 4; otherwise, it is determined as case 1 + 2 + 3 + 4. If x1 > x0, it is determined as case 2 + 3.

[0051] If y1 > y0, continue to check the horizontal coordinate x1 of point A and point D. If x1 ≤ x0, then check the horizontal coordinate x2 of point B and point C. If x2 ≤ x0, it is determined as case all 4; otherwise, it is determined as case 3 + 4. If x1 > x0, it is determined as case all 3, and so on.

[0052] S105. Obtain the coordinates of the detection box in the top - down view based on the perspective transformation matrix corresponding to the target image, and obtain the actual size of the detection box based on the coordinates and the horizontal - vertical conversion ratio.

[0053] In the embodiment of the present invention, the actual size of the detection box refers to the physical size in the real world, which reflects the size of the target object (such as a person, a vehicle, an animal, etc.) in the real environment. Obtain the coordinates of the detection box in the top - down view based on the perspective transformation matrix corresponding to the target image. At the same time, the detection box is transformed into a trapezoid in the top - down view, that is, the coordinates of the detected trapezoidal box are obtained. Through the coordinates of the detected trapezoidal box and the horizontal - vertical conversion ratio, the physical lengths of each side of the detected trapezoidal box and the height of the detected trapezoidal box can be calculated, and further area calculation is carried out to finally obtain the actual size of the detection box.

[0054] It can be understood that in the embodiments of the present invention, the collected calibration image is divided to obtain four sub-calibration images and their respective corresponding first vertex coordinates; the calibration image is used to calibrate the parameters of the first image acquisition device in the vehicle; the perspective transformation matrix corresponding to each of the four sub-calibration images is calculated through the first vertex coordinates and the perspective coordinates of the obtained calibration image; four sets of images after perspective correction of the four sub-calibration images are obtained by using the perspective transformation matrix, and the horizontal and vertical conversion ratios corresponding to each of the four sets of images are obtained; the target image is determined according to the distribution of the second vertex coordinates of the detection frame in the four sub-calibration images; the detection frame is generated based on the first image acquisition device; the coordinates of the detection frame in the top-down view are obtained based on the perspective transformation matrix corresponding to the target image, and the actual size of the detection frame is obtained based on the coordinates and the horizontal and vertical conversion ratios. In this method, the four sub-calibration images can be perspective-corrected through the perspective transformation matrix, that is, the calibration image can be perspective-corrected to obtain four sets of corrected images and the corresponding horizontal and vertical conversion ratios. Then, it is only necessary to determine the target image corresponding to the detection frame in the four sub-calibration images, and the actual size of the detection frame can be determined based on the perspective transformation matrix corresponding to the target image and the corresponding horizontal and vertical conversion ratios, without performing operations such as calibrating the image acquisition device, improving the accuracy and efficiency of measuring the size of the detection frame.

[0055] In some embodiments of the present invention, before S101, there is also S10, which can be specifically described through the following steps.

[0056] S10: Cover the target site with a black and white checkerboard paper of a preset size to obtain the target site; and use the image acquisition device to acquire the image of the target site as the calibration image.

[0057] In some embodiments of the present invention, the image acquisition device can be a separate acquisition device, that is, it can be a separate camera, or the same as the image acquisition device in the vehicle. When they are the same, the attribute values of the separate image acquisition device and the image acquisition device in the vehicle are the same, and the attribute values can be the manufacturer, model, resolution, etc. The preset site can be an empty site without interfering objects. Select a black and white checkerboard paper of a preset size as the calibration object, cover the preset site with the black and white checkerboard paper to obtain the site covered with the black and white checkerboard paper, that is, the target site. Use the image acquisition device for acquisition, such as using a camera to take pictures of the target site to obtain the image corresponding to the target site, and use this image as the calibration image.

[0058] Exemplarily, as Figure 3 shown, Figure 3 is a schematic diagram of the effect of obtaining the calibration image provided by the embodiments of the present invention. In Figure 3In this case, select an open space and ensure that there are no other interfering objects in the space so that the calibration paper can be clearly photographed. Install the image acquisition device, such as a digital camera, on a tripod to ensure the stability of the camera. Adjust the height of the camera within a reasonable range, such as the installation position of the road detection camera of an intelligent connected vehicle, so that a large enough ground area can be photographed. Use a black and white checkerboard paper with known dimensions as the calibration paper, and cover as much as possible the road area that the camera can photograph with the black and white checkerboard paper. Adjust the camera height according to the actual measurement requirements and collect calibration images. When taking pictures, ensure that the edges of the calibration paper are completely covered in the image.

[0059] It can be understood that in some embodiments of the present invention, by laying a checkerboard calibration paper with known dimensions, black and white, and fixed size on a preset open space to obtain calibration images, the accuracy of obtaining calibration images is improved.

[0060] In some embodiments of the present invention, S101 can be implemented through S1011 to S1012, and the following steps are used for specific description.

[0061] S1011. Divide the calibration image through the midpoint position coordinates of the black and white checkerboard calibration paper in the calibration image to obtain four sub-calibration images.

[0062] In some embodiments of the present invention, the calibration image is composed of a black and white checkerboard paper. Determine the center position of the black and white checkerboard calibration paper, and divide the calibration image according to the center position, so that the calibration image becomes four sub-calibration images.

[0063] S1012. Obtain the first vertex coordinates corresponding to each of the four sub-calibration images based on a preset size.

[0064] In some embodiments of the present invention, since the size of the black and white checkerboard paper is known, that is, the size of the calibration paper in the calibration image is known, the calibration image can be divided into four sub-calibration images through the pixel coordinates of the center point of the calibration paper, and the first vertex coordinates corresponding to each sub-calibration image can be obtained. The first vertex coordinates include four vertex coordinates, and the four vertex coordinates corresponding to each sub-calibration image can be marked in clockwise order.

[0065] It can be understood that in some embodiments of the present invention, the calibration image is divided through the midpoint position coordinates of the black and white checkerboard paper in the calibration image to obtain four sub-calibration images; obtain the first vertex coordinates corresponding to each of the four sub-calibration images. In this method, based on obtaining the sub-calibration images and the corresponding first vertex coordinates, the calculation of the size of the detection frame to be detected is subsequently realized, and rapid and accurate size measurement can be achieved, improving the measurement efficiency.

[0066] In some embodiments of the present invention, before S103, S20 and S21 are further included, which are specifically described through the following steps.

[0067] S20. Determine the central position of each group of images among the four groups of images, and take the two adjacent coordinate points on the left and right of the central position as the first horizontal calibration point and the second horizontal calibration point in the clockwise direction.

[0068] S21. Take the two adjacent coordinate points above and below the central position as the first vertical calibration point and the second vertical calibration point in the clockwise direction.

[0069] In some embodiments of the present invention, the horizontal calibration points in the image generally refer to the points or lines used to calibrate the horizontal direction in image processing. These calibration points are mainly used to determine the horizontal direction of the image and assist in operations such as image correction and feature extraction. The vertical calibration points in the image usually refer to the functional points or lines used to calibrate the up and down directions in the image, which can help determine the vertical lines or directions in the image and are commonly used in tasks such as image correction, object detection, and feature extraction. For each group of images among the four groups of images, determine the central position O of each group of images, and take the two adjacent coordinate points on the left and right of the central position O as the first horizontal calibration point and the second horizontal calibration point in the clockwise direction, and take the two adjacent coordinate points above and below the central position O as the first vertical calibration point and the second vertical calibration point in the clockwise direction.

[0070] Exemplarily, for the upper left calibration image among the four groups of images, select the left and right vertices A stand1 and B stand1 of two adjacent checkerboards centered horizontally as the first group of horizontal calibration points; at the same time, select the upper and lower vertices C stand1 and D stand1 of two adjacent checkerboards centered vertically as the first group of vertical calibration points;

[0071] For the upper right calibration image, select the left and right vertices A stand2 and B stand2 of two adjacent checkerboards centered horizontally as the second group of horizontal calibration points; at the same time, select the upper and lower vertices C stand2 and D stand2 of two adjacent checkerboards centered vertically as the second group of vertical calibration points;

[0072] For the lower right calibration image, select the left and right vertices A stand3 and B stand3 of two adjacent checkerboards centered horizontally as the third group of horizontal calibration points; at the same time, select the upper and lower vertices C stand3 and D stand3 of two adjacent checkerboards centered vertically as the third group of vertical calibration points;

[0073] For the lower left corrected image, select the left and right vertices A stand4 and B stand4 of two adjacent checkerboards centered horizontally as the fourth set of horizontally calibrated points; at the same time, select the upper and lower vertices C stand4 and D stand4 of two adjacent checkerboards centered vertically as the fourth set of vertically calibrated points.

[0074] It can be understood that in some embodiments of the present invention, the central position of each group of images in the four groups of images is determined, and the two coordinate points adjacent to the left and right of the central position are used as the first horizontally calibrated point and the second horizontally calibrated point in the clockwise direction; the two coordinate points adjacent to the upper and lower of the central position are used as the first vertically calibrated point and the second vertically calibrated point in the clockwise direction. Through the calibrated points, the image deformation caused by factors such as shooting angle and lens distortion can be detected and corrected, and the calibrated points can be used as reference points to ensure the correct relative position between different images.

[0075] In some embodiments of the present invention, S103 can be implemented through S1031 and S1032, and the specific description is as follows through the following steps.

[0076] S1031. Obtain the first physical distance and the first pixel distance between the first horizontally calibrated point and the second horizontally calibrated point, and obtain the second physical distance and the second pixel distance between the first vertically calibrated point and the second vertically calibrated point.

[0077] In some embodiments of the present invention, the size of the black and white checkerboard paper is known, and the size between each black checkerboard and white checkerboard in the black and white checkerboard paper is also known. The first physical distance between the first horizontally calibrated point and the second horizontally calibrated point is calculated through all known sizes, and the first pixel distance between the first horizontally calibrated point and the second horizontally calibrated point is calculated through the Euclidean formula. At the same time, the second physical distance between the first vertically calibrated point and the second vertically calibrated point is calculated through all known sizes, and the second pixel distance between the first vertically calibrated point and the second vertically calibrated point is calculated through the Euclidean calculation formula.

[0078] S1032. Calculate the horizontal conversion ratio through the first physical distance and the first pixel distance, and calculate the vertical conversion ratio through the second physical distance and the second pixel distance.

[0079] In some embodiments of the present invention, the horizontal and vertical conversion ratio formulas are as follows:

[0080]

[0081] In the above formula (2), L t is the first physical distance or the second physical distance, L pis the first pixel distance or the second pixel distance, and scale is the horizontal conversion ratio or the vertical conversion ratio.

[0082] It can be understood that in some embodiments of the present invention, the first physical distance and the first pixel distance between the first horizontal calibration point and the second horizontal calibration point are obtained, and the second physical distance and the second pixel distance between the first vertical calibration point and the second vertical calibration point are obtained; the horizontal conversion ratio is calculated through the first physical distance and the first pixel distance, and the vertical conversion ratio is calculated through the second physical distance and the second pixel distance. In this method, the horizontal and vertical conversion ratios are respectively calculated through the physical distance and the pixel distance between the calibration points, so that the pixel coordinates in the image can be accurately converted into the physical dimensions in the real world.

[0083] In some embodiments of the present invention, obtaining the coordinates of the to-be-detected frame in the top-down view based on the perspective transformation matrix corresponding to the target image in S105 can be implemented through S1051, and the following steps are used for illustration.

[0084] S1051: Obtain the number of target images, and calculate the coordinates in the top-down view according to the number, the perspective transformation matrix, and the coordinate conversion formula.

[0085] In some embodiments of the present invention, the number of target images is at least one, that is, the to-be-detected frame can be completely located in one sub-calibration image, or two sub-calibration images, or simultaneously located in four sub-calibration images. Therefore, obtain the number of target images, and calculate the coordinates of the to-be-detected frame in the top-down view according to the perspective transformation matrix and the coordinate conversion formula of each target image. Among them, the to-be-detected frame becomes a to-be-detected trapezoidal frame in the top-down view, and the coordinate conversion formula is:

[0086]

[0087] In the above formula (1), m is the perspective transformation matrix, x and y are the coordinates of the second vertex, x trans and y trans are the coordinates in the top-down view.

[0088] It can be understood that in some embodiments of the present invention, the number of target images is obtained, and the coordinates are calculated according to the number, the perspective transformation matrix, and the coordinate conversion formula. In this method, through the top-down view, it is easier to identify information such as feature points, lines, and textures on the ground. In addition, the top-down view can also eliminate the perspective distortion caused by the tilt of the camera, thereby ensuring the accuracy of the extracted ground information.

[0089] In some embodiments of the present invention, S1051 can be implemented through S201 to S203, and the following steps are used for illustration.

[0090] S201. When there is one target image, calculate the coordinates through the perspective transformation matrix, the second vertex coordinates, and the coordinate transformation formula.

[0091] S202. When there are multiple target images, divide the detection box to be detected through the target images, obtaining multiple sub-detection boxes and the second vertex coordinates corresponding to each of the multiple sub-detection boxes; each target image contains one sub-detection box to be detected.

[0092] S203. Calculate the coordinates through the perspective transformation matrix, the second vertex coordinates, and the coordinate transformation formula corresponding to each target image.

[0093] In some embodiments of the present invention, when there is one target image, that is, when the detection box to be detected is completely located in one sub-calibration image, only need to calculate the coordinates through the perspective transformation matrix, the second vertex coordinates of the detection box to be detected, and the coordinate transformation formula. When there are multiple target images, that is, when the detection box to be detected is located in multiple target images at the same time, divide the detection box to be detected through the target images, obtaining multiple sub-detection boxes, and obtaining the second vertex coordinates of each sub-detection box. Finally, each target image contains one sub-detection box to be detected. Calculate the coordinates through the perspective transformation matrix corresponding to each target image, the second vertex coordinates of the sub-detection box to be detected included in each target image, and the coordinate transformation formula.

[0094] Exemplarily, for the detection rectangle coordinate points A, B, C, and D in cases all 1, all 2, all 3, and all 4, substitute the corresponding perspective matrix into Equation (1) to calculate the coordinates, obtaining the set of four perspective coordinates of the vertices of the detection box arranged in a clockwise direction: {A'(x1', y1'), B'(x2', y2'), C'(x3', y3'), D'(x4', y4')}.

[0095] For the coordinate points A, B, C, and D of the detection box in case 1+2, the intersection points of the detection box and the abscissa x0 boundary line are obtained as E(x0, y1) and F(x0, y2) from top to bottom. For the vertices and intersection points A, E, F, and D to the left of the boundary abscissa x0, substitute the perspective transformation matrix corresponding to the upper left target image into Equation (1) to calculate the coordinates, obtaining the four perspective vertices A', E', F', D' of the rectangle arranged in a clockwise direction; for the vertices and intersection points E, B, C, and F to the right of the boundary abscissa x0, substitute the upper right target image into Equation (1) to calculate the coordinates, obtaining the four vertices E”, B”, C”, and F” of the rectangle arranged in a clockwise direction. As Figure 4 shown, Figure 4 is a schematic diagram of the perspective coordinate acquisition effect when the detection box to be detected in the embodiment of the present invention straddles the upper left and upper right images, that is Figure 4The effect schematic diagram for Case 1 + 2 is given. The calculation logic of the coordinates for Cases 1 + 4, 2 + 3, and 3 + 4 is the same as that of the above coordinates.

[0096] For the coordinates of the detection box points A, B, C, and D for Case 1 + 2 + 3 + 4, the intersection points of the detection box and the x0 boundary line of the abscissa are obtained as E(x0, y1) and F(x0, y2) from top to bottom in sequence, and the intersection points of the detection box and the y0 boundary line of the ordinate are obtained as G(x1, y0) and H(x2, y0) from left to right in sequence, as well as the demarcation midpoint O(x0, y0). For the four sub-detection boxes divided by the x0 boundary line and the y0 boundary line, their vertex coordinates are in sequence: upper left sub-detection box: A, E, O, G; upper right sub-detection box: E, B, H, O; lower right sub-detection box: O, H, C, F; lower left sub-detection box: G, O, F, D, and substitute them into formula (1) with the corresponding perspective matrices to calculate the coordinates.

[0097] Exemplarily, as Figure 5 shown, Figure 5 is a schematic diagram of the effect of a detection rectangle box to a detection trapezoid box provided by an embodiment of the present invention. In Figure 5 , the detection box is a rectangle box. From a top-down view, the detection rectangle box becomes a detection trapezoid box, and there are four target images. The four target images divide the detection trapezoid box into four sub-detection trapezoid boxes. O is the central position, x0 and y0 are the horizontal and vertical demarcation lines respectively. Taking the upper left image as an example, A stand1 and B stand1 are two horizontal calibration points; C stand1 and D stand1 are two vertical calibration points.

[0098] It can be understood that in some embodiments of the present invention, when there is one target image, the coordinates are calculated through the perspective transformation matrix, the second vertex coordinates, and the coordinate conversion formula; when there are multiple target images, the detection box is divided by the target images to obtain multiple detection boxes and the second vertex coordinates corresponding to each detection box; each target image contains a detection box; the coordinates are calculated through the perspective transformation matrix, the second vertex coordinates, and the coordinate conversion formula corresponding to each target image. By judging the number of target images, the number of divisions of the detection box is determined, and further the coordinates of the divided number are summed, which improves the accuracy of the actual size of the detection box.

[0099] In some embodiments of the present invention, the area calculation based on the coordinates and the horizontal and vertical conversion ratios in S105 to obtain the actual size of the detection box can be implemented through S105a to S105d, and the following steps are used for explanation.

[0100] S105a. Calculate the third pixel distance through the first coordinate and the second coordinate in the coordinates, and calculate the fourth pixel distance through the third coordinate and the fourth coordinate in the coordinates; the detection box becomes a detection trapezoid in the top-down view.

[0101] In some embodiments of the present invention, the detection box becomes a detection trapezoid in the top-down view. The four vertex coordinates of the detection trapezoid are called the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate in the clockwise direction. Calculate the third pixel distance between the first coordinate and the second coordinate, and the fourth pixel distance between the third coordinate and the fourth coordinate respectively through the Euclidean formula.

[0102] S105b. Calculate the upper base size of the detection trapezoid based on the third pixel distance and the horizontal conversion ratio, and calculate the lower base size of the detection trapezoid based on the fourth pixel distance and the horizontal conversion ratio.

[0103] In some embodiments of the present invention, substitute the third pixel distance and the horizontal conversion ratio into formula (2) to calculate the upper base size of the detection trapezoid, and substitute the fourth pixel distance and the horizontal conversion ratio into formula (2) to calculate the lower base size of the detection trapezoid.

[0104] S105c. Calculate the height size of the detection trapezoid through the second coordinate, the third coordinate, the fourth coordinate in the coordinates and the vertical conversion ratio.

[0105] In some embodiments of the present invention, the line segment connecting the third coordinate and the fourth coordinate is the lower base of the detection trapezoid, and the vertical distance from the second coordinate to the line segment between the third coordinate and the fourth coordinate is the height of the detection trapezoid. Calculate the value corresponding to the height size of the detection trapezoid through the values corresponding to the second coordinate, the third coordinate, the fourth coordinate and the vertical conversion ratio.

[0106] Further, the second coordinate, the third coordinate, and the fourth coordinate are P2(X2, Y2), P3(X3, Y3), and P4(X4, Y4) respectively. The calculation formula (3) for the height of the detection trapezoid calculated according to the second coordinate, the third coordinate, the fourth coordinate and the vertical conversion ratio is as follows:

[0107]

[0108] In the above formula (3), h is the height size of the detection trapezoid.

[0109] S105d. Calculate the actual size of the detection trapezoid based on the upper base size, the lower base size, and the height size.

[0110] In some embodiments of the present invention, after obtaining the upper base size, lower base size, and height size, the area of the trapezoidal frame to be detected is calculated according to the trapezoidal area calculation formula.

[0111] In some embodiments of the present invention, when there is only one target image, the upper base size, lower base size, and height size of the trapezoidal frame to be detected are obtained according to the steps from S105a to S105d, and finally the actual size of the trapezoidal frame to be detected is obtained according to the trapezoidal area calculation formula. When there are multiple target images, the areas of one trapezoidal frame to be detected located in each target image are calculated respectively according to the steps from S105a to S105d, and finally the areas of one trapezoidal frame to be detected in all each target image are added together to obtain the actual size of the trapezoidal frame to be detected.

[0112] It can be understood that, in some embodiments of the present invention, the third pixel distance is calculated through the first coordinate and the second coordinate in the coordinates, and the fourth pixel distance is calculated through the third coordinate and the fourth coordinate in the coordinates. Based on the third pixel distance and the horizontal conversion ratio, the upper base size of the trapezoidal frame to be detected is calculated, and based on the fourth pixel distance and the horizontal conversion ratio, the lower base size of the trapezoidal frame to be detected is calculated. The height size of the trapezoidal frame to be detected is calculated through the second coordinate, the third coordinate, the fourth coordinate in the coordinates and the vertical conversion ratio; based on the upper base size, the lower base size, and the height size, the actual size of the trapezoidal frame to be detected is calculated. In this method, the actual size of the trapezoidal frame to be detected is calculated through the calculated upper base size, lower base size, and height size of the trapezoidal frame to be detected, improving the calculation accuracy of the actual size of the trapezoidal frame to be detected.

[0113] Refer to Figure 6 , which shows a schematic structural diagram of an electronic device according to an embodiment of the present invention. The specific implementation of the electronic device is not limited in the specific embodiments of the present invention.

[0114] As Figure 6 shown, the electronic device may include: a processor 502, a communication interface 504, a memory 506, and a communication bus 508.

[0115] Wherein:

[0116] The processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508.

[0117] The communication interface 504 is used to communicate with other electronic devices or servers.

[0118] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above method embodiments.

[0119] Specifically, the program 510 may include program code, which includes computer operation instructions.

[0120] The processor 502 may be a central processing unit (CPU), or a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0121] The memory 506 is used to store the program 510. The memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0122] The program 510 may specifically be used to cause the processor 502 to perform the operations corresponding to the methods described in the above method embodiments.

[0123] For the specific implementation of each step in the program 510, reference may be made to the corresponding steps and descriptions in the above method embodiments, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be elaborated here.

[0124] It should be noted that according to the needs of implementation, the various components / steps described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the objectives of the embodiments of the present invention.

[0125] The method according to an embodiment of the present invention can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or be implemented as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or an FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as a RAM, a ROM, a flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0126] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present invention.

[0127] The above embodiments are only used to illustrate the embodiments of the present invention, rather than to limit the embodiments of the present invention. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention. The patent protection scope of the embodiments of the present invention shall be defined by the claims.

Claims

1. A method for calibrating and measuring the size of a road surface detection frame, characterized in that: include: Divide the collected calibration image to obtain four sub-calibration images and the first vertex coordinates corresponding to each sub-calibration image; Calculating the perspective transformation matrix corresponding to each of the four sub-calibration images through the first vertex coordinates and the obtained perspective coordinates of the calibration image; Using the perspective transformation matrix, four groups of images of the four sub-calibration images are obtained after perspective correction, and horizontal and vertical conversion ratios corresponding to the four groups of images are obtained; Determine the target image according to the distribution of the second vertex coordinates of the to-be-detected box in the four sub-calibration images; The coordinates of the to-be-detected frame in a bird's-eye view are acquired based on the perspective transformation matrix corresponding to the target image, and the area is calculated based on the coordinates and the horizontal and vertical conversion ratios to obtain the actual size of the to-be-detected frame.

2. The method according to claim 1, characterized in that Before dividing the acquired calibration image to obtain four sub-calibration images and the first vertex coordinates corresponding to each sub-calibration image, the method further includes: Black and white checkerboard paper of a preset size is spread over a preset field to obtain a target field; and an image acquisition device is used to capture an image of the target field as the calibration image.

3. The method according to claim 2, characterized in that The collected calibration image is divided to obtain four sub-calibration images and the first vertex coordinates corresponding to each sub-calibration image, including: Dividing the calibration image according to the midpoint position coordinates of the black and white checkerboard paper in the calibration image to obtain the four sub-calibration images; The first vertex coordinates corresponding to each of the four sub-calibration images are acquired based on the calibration image.

4. The method according to claim 1, characterized in that Before obtaining the horizontal and vertical conversion ratios corresponding to the four groups of images, the method further includes: Determine the center position of each of the four groups of images, and use the left and right adjacent coordinates of the center position in a clockwise direction as a first horizontal calibration point and a second horizontal calibration point; The two upper and lower adjacent coordinates of the center position are used as the first longitudinal calibration point and the second longitudinal calibration point in the clockwise direction.

5. The method according to claim 4, characterized in that The obtaining of the horizontal and vertical conversion ratios corresponding to the four groups of images respectively includes: Acquire a first physical distance and a first pixel distance between the first horizontal calibration point and the second horizontal calibration point, and acquire a second physical distance and a second pixel distance between the first vertical calibration point and the second vertical calibration point; A horizontal conversion ratio is calculated by using the first physical distance and the first pixel distance, and a vertical conversion ratio is calculated by using the second physical distance and the second pixel distance.

6. The method according to claim 1, characterized in that The obtaining the coordinates of the to-be-detected frame in a bird's-eye view based on the perspective transformation matrix corresponding to the target image includes: The number of the target images is obtained, and the coordinates are calculated according to the number, the perspective transformation matrix and the coordinate transformation formula; wherein the coordinate transformation formula (1) is: In the above formula (1), m is the perspective transformation matrix, x and y are the second vertex coordinates, x trans and trans is the coordinate under the top-down perspective.

7. The method according to claim 6, characterized in that The calculating the coordinates according to the quantity, the perspective transformation matrix and the coordinate conversion formula includes: When the target image is one, the coordinates are calculated by using the perspective transformation matrix, the second vertex coordinates and the coordinate conversion formula; When there are multiple target images, the to-be-detected frame is divided by the target image to obtain multiple sub-detection frames and second vertex coordinates corresponding to each of the multiple sub-detection frames; each target image contains one sub-detection frame; The coordinates are calculated using the perspective transformation matrix corresponding to each target image, the second vertex coordinates and the coordinate conversion formula.

8. The method according to claim 5, characterized in that The area calculation based on the coordinates and the horizontal and vertical conversion ratios to obtain the actual size of the to-be-detected frame includes: A third pixel distance is calculated by using the first coordinate and the second coordinate in the coordinates, and a fourth pixel distance is calculated by using the third coordinate and the fourth coordinate in the coordinates; the frame to be detected is changed into the trapezoidal frame to be detected under the top-down viewing angle; Calculating the upper base size of the trapezoidal frame to be detected based on the third pixel distance and the horizontal conversion ratio, and calculating the lower base size of the trapezoidal frame to be detected based on the fourth pixel distance and the horizontal conversion ratio; Calculate the height of the trapezoidal frame to be detected by using the second coordinate, the third coordinate, the fourth coordinate and the vertical conversion ratio; The actual size of the trapezoidal frame to be detected is calculated based on the upper base size, the lower base size and the height size.

9. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method as described in any one of claims 1-8.

10. A computer storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images