Floor calibration method and related device based on distorted image correction

Abstract

Floor calibration method and related device based on distorted image correction, including: detecting straight lines in the input building image and performing image radial correction based on the polynomial model of radial distortion; detecting edges in the input building image and obtaining image perspective correction based on the optimal vanishing point pair in the image; detecting the contour of the building image that has undergone image radial correction and image perspective correction to obtain a set of clear floor structure contours of the building image; calculating the centroid of each contour based on the set of floor contour structures of the building image, clustering the ordinate of the contour centroid, and finally the number of generated clusters is the number of floors, and the mean value of the contour centroids within each category is the ordinate position where the floor is located. The present invention realizes comprehensive correction for a single building image, without additional information, can make full use of the image content, and more lightly realizes floor structure capture, and has the advantages of robust performance and full utilization of image information.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a floor calibration method and related device based on distortion image correction. Background Art

[0002] In recent years, the frequent occurrence of high-altitude parabolic events has caused serious social harm and even triggered people's concerns about "safety above the head". To curb this phenomenon, many high-altitude parabolic behavior detection systems have emerged, aiming to capture parabolic behaviors and obtain parabolic trajectories through object detection and object tracking. However, existing methods generally lack the positioning link for the parabolic floor, only providing detection functions but lacking the floor positioning function, thus unable to accurately warn of parabolic behaviors and provide detailed evidence of parabolic behaviors. To fill this gap, it is necessary to achieve standard positioning of floors for buildings in natural scenes. However, due to the complex environment where buildings are located and the different floor structures, there is no general floor calibration method. At the same time, due to reasons such as different camera lens parameters and different shooting perspectives, building images usually have various distortion problems, which damage the standard floor structure and further increase the difficulty of floor calibration.

[0003] Existing floor positioning methods based on monitoring hardware facilities generally have a strong dependence on the hardware installation conditions of monitoring devices. It is necessary to install and deploy the monitoring system according to strict rules (such as specifying the height of the monitoring camera, the distance from the floor height, etc.) to complete parabolic detection, and the installation cost is relatively high, which is not convenient for large-scale promotion. In addition, most high-altitude parabolic behavior detection systems can only vaguely push parabolic position information and cannot accurately locate the specific floor.

[0004] The prior art constructs a floor training network based on the yolo network. Before the training starts, at fixed time intervals, several floor images at different times, lights, and weathers are manually marked, and these floor images are used to train the yolo network to detect the floors in the images. However, it is necessary to build a model through manual annotation and training, and the trained network has poor scalability.

[0005] The prior art locates the parabolic floor by fitting the parabolic position. This method fits and completes the parabolic motion trajectory to obtain the final fitting landing point, fitting parabolic point, fitting landing area, and fitting parabolic point area, thereby determining the floor where the high-altitude parabolic occurs. However, it is necessary to rely on the parabolic trajectory to locate the floor, and the limitations are relatively large. Summary of the Invention

[0006] The purpose of the present invention is to provide a floor calibration method and related device based on distortion image correction to solve the problems that the network trained by the prior art has poor scalability and the floor can only be located relying on the parabolic trajectory, with relatively large limitations.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A floor calibration method based on distorted image correction, comprising:

[0009] Performing line detection on the input building image, and performing image radial correction based on the polynomial model of radial distortion;

[0010] Performing edge detection on the input building image, and obtaining image perspective correction based on the optimal vanishing point pair in the image;

[0011] Performing contour detection on the building image that has undergone image radial correction and image perspective correction, and obtaining a set of clear floor structure contours of the building image;

[0012] Based on the set of floor contour structures of the building image, calculating the centroid of each contour, clustering the ordinates of the contour centroids, and the finally generated number of clusters is the number of floors, and the mean value of the contour centroids within each class is the ordinate position where the floor is located.

[0013] Further, the image radial correction is specifically as follows:

[0014] Performing line detection on the input building image, dividing the line segments extracted from the same edge into a group, and traversing all line segment groups, filtering out the line segments and line segment groups that cannot provide correction information; inspired by the idea that the line segments within the same line segment group should be collinear, using the included angle between two consecutive line segments within the same line segment group as a measurement index to evaluate the correction effect; based on the polynomial model of radial distortion, using the Newton iteration method to estimate the distortion parameters, and applying the optimal distortion parameters for image correction.

[0015] Further, the EDlines algorithm is used to detect the image edges, and line segments are extracted on the detected edges. The line segments extracted on the same edge are divided into a group, and the corresponding initial line segment group is L init = L1 + L2 + … + L p ; Traversing the line segment groups, if a line segment group contains only 1 line segment, then delete this line segment group, if there is a line segment within a line segment group whose length is less than the threshold or approximately passes through the center of the image, then delete this line segment;

[0016] Using the included angle between two consecutive line segments within the same line segment group as a measurement index to evaluate the correction effect, and the calculation process of the loss function is as follows:

[0017]

[0018]

[0019]

[0020] In the formula, β j represents the angle between two consecutive line segments l j and l j+1 in the line segment group i, n is the total number of line segments in the line segment group i, and N is the number of line segment groups;

[0021] Then, on the obtained line segment groups, a radial distortion polynomial model with two parameters k1 and k2 is adopted, and the head and tail coordinates (x u1 , y u1 ) and (x u2 , y u2 ) of the corresponding line segments on the standard image and the distortion parameters k1 and k2 are estimated by the Newton iteration method;

[0022] Suppose the loss function obtained by estimating the distortion parameters for N line segment groups is Loss N . Next, N - 1 line segment groups are extracted from the N line segment groups as sub-line segment groups, and the distortion parameters and their corresponding Loss N-1 are calculated again; a total of sub-line segment groups can be extracted, corresponding to N - 1 Loss N-1 respectively. The minimum value Loss min is selected; if Loss min ≥Loss N , select Loss N to enter the final correction module; otherwise, retain the sub-line segment group corresponding to Loss min ; until removing any group of line segments cannot reduce the loss function. At this time, the corresponding distortion parameters k1 and k2 are the optimal parameters, and these parameters are applied to the entire radially distorted image to complete the radial distortion image correction.

[0023] Furthermore, for image perspective correction:

[0024] Perform edge detection on the input building image to obtain the edge set of the image; assume that the image vanishing point is the intersection point of the edge lines corresponding to two randomly selected edges, and estimate the optimal vanishing point pair in the image based on the RANSAC algorithm; calculate the image homography matrix H and rotation matrix R using the estimated horizontal vanishing point and vertical vanishing point, and adopt the transformation T = HR to perform image perspective correction.

[0025] Furthermore, obtain the edges on the image through the Canny edge detection operator, and extract the edge lines using the probabilistic Hough transform; each edge E contains 3 attributes, namely edge position p, edge direction d, and edge intensity s, and the elements in the edge set are sorted in descending order of intensity;

[0026] Assume that the vanishing point of the image is the intersection point of the edge lines corresponding to two randomly selected edges, and the vanishing point is iteratively selected through the RANSAC (Random Sample Consensus) algorithm; let the edge randomly selected in a certain round of iteration be E i , E j , and the corresponding edge segments are l i , l j . The model fitted by this pair of edges is denoted as M(E i , E j ), and the corresponding vanishing point is denoted as v M = l i × l j ;

[0027] For the fitted model M(E i , E j ), traverse all the remaining edges E k in the edge set, (k ≠ i, j) is the score of this model, and the score value is as follows:

[0028]

[0029] In the formula, the line connecting the vanishing point and the midpoint of the edge endpoint pair is denoted as l vp , θ is the included angle between l vp and l k ;

[0030] In one round of iteration, the result of accumulating all the voting values is the score of the model M(E i , E j ). Through a sufficient number of rounds of iteration, the best-fitted model M best is obtained;

[0031] Subsequently, based on the definition in the RANSAC algorithm that the edge lines conforming to M best are called "inliers", use S = {E k | vote(E k , M best ) > 0} to represent the "inliers"; ideally, l k will pass through that is Let w k = vote(E k , M best ), increase the proportion of the edges with large edge intensity. In the three-dimensional coordinate system, denote the edge line l k = [a k , b k , c k T , The matrix expression for solving the vanishing point is as follows, and it can be solved by using the matrix singular value decomposition method: ​

[0032]

[0033] After obtaining a vanishing point, delete all the edges in the “inner point” S, and repeat the above steps to obtain the second vanishing point. The horizontal vanishing point v1 and the vertical vanishing point v2 corresponding to the building image are obtained by two vanishing point estimations. The straight line passing through v1 and v2 is recorded as l 12 =v1×v2=[l x , l y , l z ] T ; Use the homography matrix H to make the vanishing points tend to infinity in their respective directions, and use the rotation matrix R to further correct the direction of the vertical vanishing points:

[0034]

[0035] Where α is the angle between the vertical vanishing point v2 and the y-axis;

[0036] Then, using the correction matrix T=HR, parallel straight lines can be restored, the vertical viewing angle can be corrected, and the perspective distortion image correction can be completed.

[0037] Furthermore, a clear set of floor structure outlines of the building image is obtained:

[0038] The interference contours are filtered out by ROI delineation and contour filtering to obtain a clear floor structure contour set C of the building image:

[0039] First, define the image ROI area of interest, that is, the building part, and then use the OpenCV built-in operator findContours to perform contour detection to obtain the initial contour set C init , each contour is represented by its coordinate sequence; the area, length and width of each contour are calculated, all contours are sorted by area size, and the contours with the smallest 30% and the largest 10% are filtered out to obtain the contour set C.

[0040] Furthermore, in the process of floor calibration based on contour detection, the array floor is set to store the ordinates of each floor, and the floor height threshold γ is set; the ordinates y of all contour centroids are traversed. i , if (y i -γ,y i +γ) in the array floor, it means that the centroid coordinates of this floor are already in the final array, and there is no need to add y i into the array floor; otherwise, y i into the array floor; the length of the array floor finally represents the number of floors, and the elements in it represent the coordinates of each floor.

[0041] Further, a floor calibration system based on distorted image correction includes:

[0042] An image radial correction module for performing line detection on the input building image and performing image radial correction based on a polynomial model of radial distortion;

[0043] An image perspective correction module for performing edge detection on the input building image and obtaining image perspective correction based on the optimal vanishing point pair in the image;

[0044] A contour set obtaining module for performing contour detection on the building image that has undergone image radial correction and image perspective correction to obtain a clear floor structure contour set of the building image;

[0045] A floor calibration module for calculating the centroid of each contour based on the floor contour structure set of the building image, clustering the ordinates of the contour centroids, and finally, the number of generated clusters is the number of floors, and the mean value of the contour centroids within each category is the ordinate position where the floor is located.

[0046] Further, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the floor calibration method based on distorted image correction are implemented.

[0047] Further, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the floor calibration method based on distorted image correction are implemented.

[0048] Compared with the prior art, the present invention has the following technical effects:

[0049] The present invention generates a floor standard positioning through distorted image correction, floor contour detection, and floor structure clustering. First, radial distortion correction processing is performed on the image by dividing line segments into groups and straightening the line segments within the same group; then, vanishing point estimation and homography matrix transformation are used to achieve image perspective correction and restore the front parallel feature of the building image; next, contour detection is performed on the undistorted building image to obtain the building structure and filter out interfering contours; finally, a floor structure clustering model is constructed to achieve the standard positioning of the floors. The present invention realizes comprehensive correction for a single building image without additional information, can make full use of the image content, and more lightly realizes floor structure capture, having the advantages of robust performance and full utilization of image information. Description of the Drawings

[0050] Figure 1 is a block diagram of the floor calibration method based on distorted image correction of the present invention.

[0051] Figure 2 It is the flowchart of radial distortion image correction.

[0052] Figure 3 It is the flowchart of perspective distortion image correction.

[0053] Figure 4 It is the flowchart of floor structure extraction and clustering based on contour detection.

[0054] Figure 5 It is the test sample result of the present invention.

[0055] Figure 6 It is the performance test result graph of the present invention. Specific Embodiments

[0056] The following describes the embodiments of the present invention in detail in conjunction with the accompanying drawings and embodiments. It should be noted that the embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, without conflict, the technical features involved in the embodiments of the present invention can be combined with each other.

[0057] The specific implementation process of the present invention includes radial distortion image correction, perspective distortion image correction, and contour detection and clustering. Figure 1 It is the block diagram of the floor calibration method based on distortion image correction of the present invention.

[0058] Radial Distortion Image Correction

[0059] After giving the input building image, the present invention uses the EDlines algorithm to detect the image edges, extracts line segments on the detected edges, and the line segments extracted on the same edge are divided into a group, corresponding to the initial line segment group L init = L1 + L2 + … + L p . Intuitively, the line segments extracted from the same edge should be collinear.

[0060] First, the present invention traverses the line segment group and filters out the line segments and line segment groups that cannot provide correction information. If there is only 1 line segment in a certain line segment group, then delete this line segment group. If there is a line segment with a length less than the threshold or the line segment approximately passes through the center of the image in a certain line segment group, then delete this line segment.

[0061] The goal of radial distortion correction is to adjust the line segments in the same line segment group to collinear line segments, and use the included angle between two consecutive line segments in the same line segment group as a measurement index to evaluate the correction effect. The present invention calculates the loss function using the following formula:

[0062]

[0063]

[0064]

[0065] In the formula, β j represents a group of line segments i and is the included angle between two consecutive line segments l j , l j+1 within the group. n is the total number of line segments within the i-th group of line segments, and N is the number of groups of line segments;

[0066] Then, the present invention estimates the distortion parameters on the obtained group of line segments by using a radial distortion polynomial model with two parameters k1 and k2. The head and tail coordinates (x d1 , y d1 ) and (x d2 , y d2 ) of each line segment are known. Comparing with the polynomial distortion model, the distortion parameters k1 and k2 and the corresponding head and tail coordinates (x u1 , y u1 ) and (x u2 , y u2 ) on the standard image are missing. The present invention sets the initial distortion parameters k1 = 0 and k2 = 0. On the premise of fixing k1 and k2, the undistorted point coordinates (x u1 , y u1 ) and (x u2 , y u2 ) are solved by the Newton iteration method. Based on obtaining a pair of undistorted point coordinates, k1 and k2 that minimize the loss are iteratively searched for, and so on. The actual distortion parameters are estimated within an acceptable error range.

[0067] Subsequently, assuming that the loss function obtained by estimating the distortion parameters for N groups of line segments is Loss N , the present invention extracts N - 1 groups of line segments from these N groups of line segments as sub - groups of line segments, and calculates the distortion parameters and their corresponding Loss N-1 again; a total of sub - groups of line segments can be extracted, corresponding to N - 1 Loss N-1 . The minimum value Loss min is selected; if Loss min ≥Loss N , Loss N is selected to enter the final correction module; otherwise, the sub - group of line segments corresponding to Loss min is retained; until removing any group of line segments cannot reduce the loss function, at this time, the corresponding distortion parameters k1 and k2 are the optimal parameters, and applying these parameters to the entire radially distorted image can complete the radial distortion image correction.

[0068] The above process of radial distortion image correction is as Figure 2 shown.

[0069] 2. Perspective Distortion Image Correction

[0070] The present invention obtains the edges on the image I through the Canny edge detection operator and uses the probabilistic Hough transform to extract the edge lines. Each edge E contains three attributes, namely the edge position p, the edge direction d, and the edge strength s. For the convenience of calculation, the present invention expresses the edge position by the midpoint coordinates of each edge, represents the edge direction by the tangent value corresponding to the inclination angle of the edge segment, and measures the edge strength by the length of the edge. The elements in the edge set are arranged in descending order of strength.

[0071] Assume that the vanishing point of the image is the intersection point of the edge lines corresponding to two randomly selected edges. The vanishing point is iteratively selected through the RANSAC (Random Sample Consensus) algorithm. Let the randomly selected edges in a certain round of iteration be E i , E j , and the corresponding edge segments be l i , l j . The model fitted by this pair of edges is denoted as M(E i , E j ), and the corresponding vanishing point is denoted as v M = l i × l j .

[0072] For the fitted model M(E i , E j ), traverse all the remaining edges E k in the edge set. (k ≠ i, j) is the score of this model. The present invention uses the following formula for scoring:

[0073]

[0074] In the formula, the line connecting the vanishing point and the midpoint of the edge endpoint pair is denoted as l vp , θ is the included angle between l vp and l k ;

[0075] In a round of iteration, the result of accumulating all the voting values is the score of the model M(E i , E j ). Through a sufficient number of rounds of iteration, the best-fitting model M best can be obtained.

[0076] Subsequently, the present invention calculates the vanishing point corresponding to M best through the weighted least squares method. The RANSAC algorithm points out that the edge lines that conform to M best are called "inliers", denoted as S = {E k | vote(E k , Mbest ) > 0} to characterize "inlier points". In an ideal situation, l k would pass through That is to say Let w k = vote(E k , M best ), increasing the proportion of edges with large edge intensities. In a three-dimensional coordinate system, denote the edge line l k = [a k , b k , c k T , The matrix expression for solving the vanishing point is as shown in the following formula, and it can be solved by using the matrix singular value decomposition method:

[0077]

[0078] Similarly, after obtaining a vanishing point, delete all the edges in the "inlier points" S, and then repeat the above steps to obtain the second vanishing point. After obtaining the corresponding horizontal vanishing point v1 and vertical vanishing point v2 in the present invention, denote the line passing through v1 and v2 as l 12 = v1 × v2 = [l x , l y , l z T , use the homography matrix H to make the vanishing points tend to infinity in their respective directions, and use the rotation matrix R to further correct the direction of the vertical vanishing point. The matrices H and R are as shown in the following formulas:

[0079]

[0080] In the formula, α is the angle between the vertical vanishing point v2 and the y-axis.

[0081] Finally, the present invention uses the calibration matrix T = HR to restore the parallel lines of the image and correct the vertical perspective, completing the perspective distortion image correction.

[0082] The above process of perspective distortion image correction is as Figure 3 shown.

[0083] 3. Contour Detection Based on Undistorted Images

[0084] The present invention filters out the interference regions by delimiting the ROI (Region of Interest) of the image to obtain the building part. The present invention delimits the region manually. For the input image, click the left mouse button to draw points, click the right mouse button to cancel the most recently selected point, and finally click the middle mouse button to connect the overall region. Then, the present invention uses the built-in operator findContours in OpenCV for contour detection to obtain the initial contour set C init ​​, each contour is characterized by its coordinate sequence. Then, the area, length, and width of each contour are calculated through the built-in operators of OpenCV. All the contours are sorted by area size, and 30% of the contours with the smallest area and 10% of the contours with the largest area are filtered out to obtain the set C of the extracted floor structure contours.

[0085] 4. Floor Calibration Based on Contour Detection

[0086] The present invention sets an array floor to store the vertical coordinates of each floor and sets a floor height threshold γ. Based on the set C of the extracted floor structure contours, the centroid of each contour is calculated, and the vertical coordinate y of the contour centroid is traversed. i . If (y i - γ, y i + γ) is within the array floor, it means that the centroid coordinates of this floor already exist in the final array and there is no need to add y i to the array floor; otherwise, add y i to the array floor; finally, the length of the array floor represents the number of floors, and the elements in it represent the coordinates of each floor.

[0087] The flow chart of floor structure extraction and clustering based on contour detection is as Figure 4 shown, the test examples of this model are as Figure 5 shown, and the performance test results of this model on different types of data sets are as Figure 6 shown.

[0088] In another embodiment of the present invention, a floor calibration system based on distorted image correction is provided, which can be used to implement the above-mentioned floor calibration method based on distorted image correction. Specifically, the system includes:

[0089] An image radial correction module, which is used to perform line detection on the input building image and perform image radial correction based on the polynomial model of radial distortion;

[0090] An image perspective correction module, which is used to perform edge detection on the input building image and obtain image perspective correction based on the optimal vanishing point pair in the image;

[0091] A contour set obtaining module, which is used to perform contour detection on the building image after image correction and image perspective correction to obtain a clear floor structure contour set of the building image;

[0092] A floor calibration module, which is used to calculate the centroid of each contour based on the floor contour structure set of the building image, cluster the vertical coordinates of the contour centroids, and the final generated number of clusters is the number of floors, and the mean value of the centroid of the contours within each class is the vertical coordinate position where the floor is located.

[0093] The division of modules in the embodiments of the present invention is illustrative and is only a logical function division. In actual implementation, there may be other division methods. In addition, in each embodiment of the present invention, the functional modules can be integrated in a processor, can exist separately physically, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0094] In another embodiment of the present invention, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store a computer program. The computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function. The processor in the embodiments of the present invention can be used for the operation of the floor calibration method based on distorted image correction.

[0095] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, and the operating system of the terminal is stored in this storage space. And, one or more instructions suitable for being loaded and executed by the processor are also stored in this storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The one or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the floor calibration method based on distorted image correction in the above embodiments.

[0096] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0097] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0098] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: still can modify the specific implementation manners of the present invention or make equivalent replacements, and any modification or equivalent replacement without departing from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A floor calibration method based on distortion image correction, characterized in that, Including: Performing line detection on the input building image and performing image radial correction based on the polynomial model of radial distortion; Performing edge detection on the input building image and obtaining image perspective correction based on the optimal vanishing point pair in the image; Performing contour detection on the building image that has undergone image radial correction and image perspective correction to obtain a set of clear floor structure contours of the building image; Based on the set of floor contour structures of the building image, calculating the centroid of each contour, clustering the ordinate of the contour centroid, and the finally generated number of clusters is the number of floors, and the mean value of the contour centroids within each category is the ordinate position where the floor is located; The specific image radial correction is as follows: Performing line detection on the input building image, dividing the line segments extracted from the same edge into a group, and traversing all line segment groups, filtering out the line segments and line segment groups that cannot provide correction information; inspired by the idea that the line segments within the same line segment group should be collinear, using the angle between two consecutive line segments within the same line segment group as a measurement index to evaluate the correction effect; based on the polynomial model of radial distortion, using the Newton iteration method to estimate the distortion parameters, and applying the optimal distortion parameters for image correction; Use the EDlines algorithm to detect image edges and extract line segments on the detected edges. The line segments extracted on the same edge are grouped into a group, and the corresponding initial line segment group is L init =L1+L2+…+L p ; Traverse the line segment groups. If a line segment group contains only one line segment, delete the line segment group. If a line segment in a line segment group has a length less than the threshold or approximately passes through the center of the image, delete the line segment. Image perspective correction: Performing edge detection on the input building image to obtain the edge set of the image; assuming that the image vanishing point is the intersection of the edge lines corresponding to two randomly selected edges, estimating the optimal vanishing point pair in the image based on the RANSAC algorithm; calculating the homography matrix H and rotation matrix R of the image using the estimated horizontal vanishing point and vertical vanishing point, and obtaining image perspective correction using the transformation T = HR; Obtaining the edges on the image through the Canny edge detection operator and extracting the edge lines using the probabilistic Hough transform; each edge E contains 3 attributes, namely the edge position p, edge direction d, and edge intensity s, and the elements in the edge set are sorted in descending order of intensity.

2. The floor calibration method based on distorted image correction according to claim 1, wherein Using the angle between two consecutive line segments within the same line segment group as a measurement index to evaluate the correction effect, and the calculation process of the loss function is as follows: where β j represents the included angle between two consecutive line segments l j , l j+1 in the line segment group i, n is the total number of line segments in the line segment group i, and N is the number of line segment groups; Then, on the obtained line segment group, a radial distortion polynomial model with two parameters k1 and k2 is adopted, and the head and tail coordinates (x u1 , y u1 ), (x u2 , y u2 ) of the corresponding line segments on the standard image and the distortion parameters k1 and k2 are estimated by the Newton iteration method; Suppose the loss function obtained by estimating the distortion parameters for N groups of line segments is Loss N , next, extract N - 1 groups of line segments from the N groups of line segments as sub-line segment groups, and calculate the distortion parameters and their corresponding Loss again N-1 ; a total of sub-line segment groups can be extracted, corresponding to N - 1 Losses N-1 , select the minimum value of Loss min ; if Loss min ≥Loss N , select Loss N to enter the final correction module; otherwise, retain the sub-line segment group corresponding to Loss min ; until removing any group of line segments cannot reduce the loss function, at this time, the corresponding distortion parameters k1, k2 are the optimal parameters, and apply this parameter to the entire radially distorted image to complete the radial distortion image correction.

3. The floor calibration method based on distorted image correction according to claim 1, wherein, Assume that the vanishing point of the image is the intersection point of the edge lines corresponding to two randomly selected edges, and the vanishing point is selected iteratively through the RANSAC (Random Sample Consensus) algorithm; let the edge randomly selected during a certain round of iteration be E i , E j , and the corresponding edge segments are l i , l j . The model fitted by this pair of edges is denoted as M(E i , E j ), and the corresponding vanishing point is denoted as v M = l i × l j ; For the fitted model M(E i ,E j ), traverse all the remaining edges E in the edge set k , (k≠i,j) is the score of this model, and the score values are as follows: In the formula, the line connecting the vanishing point and the midpoint of the edge endpoint pair is denoted as l vp , and θ is the included angle between l vp and l k ; In one iteration, the result of accumulating all voting values is the score of the model M(E i ,E j ). Through a sufficient number of iterations, the best fitting model M best ; Subsequently, based on the definition that the edge lines conforming to M in the RANSAC algorithm are called "inliers", use S = {E best | vote(E k , M k ) > 0} to represent "inliers"; Ideally, l best will pass through k , that is Let w k = vote(E k , M best ) to increase the proportion of edges with large edge intensity. In a three-dimensional coordinate system, denote the edge line l k = [a k , b k , c k T , The matrix expression for solving the vanishing point is as follows, and it can be solved by using the matrix singular value decomposition method:​​ After obtaining one vanishing point, all the edges in the "inlier" S are deleted, and then the above steps are repeated to obtain the second vanishing point; the horizontal vanishing point v1 and the vertical vanishing point v2 corresponding to the building image are obtained through two vanishing point estimations, and the line passing through v1 and v2 is denoted as l 12 = v1 × v2 = [l x , l y , l z T ; The homography matrix H is used to make the vanishing points tend to infinity in their respective directions, and the rotation matrix R is used to further correct the direction of the vertical vanishing point:​ In the formula, α is the angle between the vertical vanishing point v2 and the y-axis; Subsequently, using the correction matrix T = HR, parallel lines can be restored, the vertical perspective can be corrected, and the perspective distortion image correction can be completed.

4. The floor calibration method based on distorted image correction according to claim 1, wherein Obtaining a set of clear floor structure contours of the building image: Filtering out interfering contours by means of ROI delineation and contour filtering to obtain a set of clear floor structure contours C of the building image: First, delimit the ROI (Region of Interest) of the image, that is, the building part. Then, use the built-in operator findContours in OpenCV to perform contour detection to obtain the initial contour set C init , and each contour is characterized by its coordinate sequence; calculate the area, length, and width of each contour, sort all the contours by area size, and filter out the contours with the smallest 30% area and the largest 10% area to obtain the contour set C.

5. The floor calibration method based on distorted image correction according to claim 1, wherein, During the floor calibration process based on contour detection, set an array floor to store the vertical coordinates of each floor and set a floor height threshold γ; traverse the vertical coordinates y of all contour centroids i , if (y i -γ, y i +γ) is within the array floor, it means that the centroid coordinates of this floor already exist in the final array and there is no need to add y i to the array floor; otherwise, add y i to the array floor; finally, the length of the array floor represents the number of floors, and its elements represent the coordinates of each floor.

6. A floor calibration system based on distorted image correction, characterized in that, Including: An image radial correction module for performing line detection on the input building image and performing image radial correction based on the polynomial model of radial distortion; An image perspective correction module for performing edge detection on the input building image and obtaining image perspective correction based on the optimal vanishing point pair in the image; A contour set obtaining module for performing contour detection on the building image that has undergone image radial correction and image perspective correction to obtain a set of clear floor structure contours of the building image; A floor calibration module for calculating the centroid of each contour based on the set of floor contour structures of the building image, clustering the ordinate of the contour centroid, and the finally generated number of clusters is the number of floors, and the mean value of the contour centroids within each category is the ordinate position where the floor is located; Specifically, the radial correction of the image is as follows: Perform line detection on the input building image, divide the line segments extracted from the same edge into a group, and traverse all line segment groups to filter out the line segments and line segment groups that cannot provide correction information; inspired by the idea that the line segments in the same line segment group should be collinear, use the included angle between two consecutive line segments in the same line segment group as a measurement index to evaluate the correction effect; based on the polynomial model of radial distortion, use the Newton iteration method to estimate the distortion parameters, and apply the optimal distortion parameters for image correction; Use the EDlines algorithm to detect the edges of the image, and extract line segments on the detected edges. The line segments extracted on the same edge are grouped together, and the corresponding initial line segment group is L init = L1 + L2 + … + L p ; Traverse the line segment groups. If a line segment group contains only 1 line segment, then delete this line segment group. If there is a line segment in a line segment group whose length is less than the threshold or approximately passes through the center of the image, then delete this line segment; Perspective correction of the image: Perform edge detection on the input building image to obtain the edge set of the image; assume that the vanishing point of the image is the intersection point of the edge lines corresponding to two randomly selected edges, and estimate the optimal vanishing point pair in the image based on the RANSAC algorithm; calculate the homography matrix H and rotation matrix R of the image using the estimated horizontal vanishing point and vertical vanishing point, and obtain the perspective correction of the image by using the transformation T = HR; Obtain the edges on the image through the Canny edge detection operator, and use the probabilistic Hough transform to extract the edge lines; each edge E contains three attributes, namely the edge position p, edge direction d, and edge intensity s, and the elements in the edge set are arranged in descending order of intensity.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the floor calibration method based on distorted image correction according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the floor calibration method based on distorted image correction according to any one of claims 1 to 5 are implemented.

Images