Three-dimensional point cloud data modeling method and server for road and bridge slope inspection
By combining the three-dimensional point cloud data modeling method of image and lidar, the problem of insufficient data processing during drone inspection is solved, and accurate modeling and data analysis of road and bridge slopes is realized.
Patent Information
- Application Number
- CN202510072630.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-17
- Publication Date
- 2025-04-25
- Estimated Expiration
- 2045-01-17
Smart Images
Figure CN119516141B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a three-dimensional point cloud data modeling method and server for road and bridge slope inspection. Background Art
[0002] The main purpose of road and bridge slope inspection is to promptly discover and deal with safety hazards existing in the slopes, ensure the stability of the slopes and safe passage of the road. Through inspection, we can understand the deformation and damage of the slopes, as well as the integrity of the protective facilities, and provide a basis for subsequent maintenance and reinforcement.
[0003] There are two ways to implement it at present: manual inspection and drone inspection. The most widely used one is the combination of the two methods, because each of them has its own advantages and disadvantages. The advantage of manual inspection is that it can rely on experience to discover safety hazards, but it has obvious shortcomings in frequency and coverage. The advantage of drone inspection is that it can achieve high frequency and full coverage, but it requires targeted design and development in subsequent data processing.
[0004] At present, drone inspections are mostly based on taking images, which can reflect the actual situation of the road and bridge slopes to a certain extent, but the images do not carry height information and are not sensitive to subtle changes in layout and height changes. Of course, laser radar is currently used to make up for the shortcomings of image processing methods, but the massive amount of data generated by laser radar and the data overlap and data position correspondence problems caused by position movement are more, and how to solve them requires further research. Summary of the invention
[0005] The present application provides a three-dimensional point cloud data modeling method and server for road and bridge slope inspection, which combines the respective advantages of images and lidar to model the road and bridge slopes. The established model can reflect subtle changes in the layout and height changes of the road and bridge slopes, and can provide an accurate basis for subsequent data analysis.
[0006] The above-mentioned purpose of the present application is achieved through the following technical solutions:
[0007] In a first aspect, the present application provides a three-dimensional point cloud data modeling method for road and bridge slope inspection, comprising:
[0008] Use satellite positioning to determine the current position coordinates and use ranging to determine the current altitude coordinates;
[0009] Perform a position adjustment according to the current position coordinates and the current height coordinates, and move to the first position;
[0010] Acquire a first captured image at a first position;
[0011] Extracting local features in the first captured image and comparing the local features with local features in a reference captured image corresponding to the first position to obtain a spatial displacement adjustment amount;
[0012] Move according to the spatial displacement adjustment amount, from a first position to a second position;
[0013] acquiring a second captured image at a second position;
[0014] Extracting local detail features in the second captured image and performing three-dimensional scanning on the local detail features to obtain three-dimensional point cloud data belonging to the local detail features;
[0015] A three-dimensional point cloud model belonging to the second position is constructed using the three-dimensional point cloud data.
[0016] In a possible implementation manner of the first aspect, extracting the local feature in the first captured image includes:
[0017] Processing the first captured image using a grayscale processing method to obtain a first grayscale captured image;
[0018] Using the grayscale interval to perform feature extraction in the first grayscale captured image, obtaining a first grayscale feature image, wherein the first grayscale feature image includes local features;
[0019] Determine the center position point of the local feature included in the first grayscale feature image and calculate the grayscale value of the center position point, where the grayscale value of the center position point is the average of the grayscale values of the local feature corresponding to the center position point;
[0020] A feature grid is constructed using the central position points, and the number of edges of the grids in the feature grid is the same.
[0021] In a possible implementation manner of the first aspect, extracting the local feature in the first captured image includes:
[0022] Processing the first captured image using a grayscale processing method to obtain a first grayscale captured image;
[0023] creating at least one reference line on the first grayscale captured image;
[0024] Use the pixel values of the pixels on the reference line to construct an analysis curve or analysis series;
[0025] The analysis curve is decomposed by wavelet decomposition or the characteristic series of the analysis series is calculated to determine multiple position characteristic points on the reference line. The characteristic series includes a difference series or a quadratic difference series.
[0026] In a possible implementation manner of the first aspect, at least two reference lines intersect each other, and an intersection point of the two reference lines is located on a local feature in the first captured image.
[0027] In a possible implementation manner of the first aspect, obtaining the spatial displacement adjustment amount includes:
[0028] Comparing the local features in the first captured image with the local features in the corresponding reference captured image to obtain an adjustment amount, the adjustment amount including an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount;
[0029] A spatial displacement adjustment amount is generated according to the adjustment amount, and the spatial displacement adjustment amount includes at least one of an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount.
[0030] In a possible implementation manner of the first aspect, extracting local detail features in the second captured image and performing three-dimensional scanning on the local detail features includes:
[0031] Performing grayscale processing on the second captured image using a grayscale processing method to obtain a second grayscale captured image;
[0032] Using the grayscale interval to perform feature extraction in the second grayscale captured image, obtaining a second grayscale feature image, wherein the second grayscale feature image includes local detail features;
[0033] Determine the edge contour and position coordinates of local detail features;
[0034] A scanning area is created using position coordinates and edge contours of local detail features, and a three-dimensional scan is performed on the scanning area to obtain a three-dimensional point cloud data set belonging to the scanning area.
[0035] In a possible implementation manner of the first aspect, constructing a three-dimensional point cloud model belonging to the second position using the three-dimensional point cloud data includes:
[0036] Taking the position coordinates of the local detail features as the position coordinates of the corresponding three-dimensional point cloud data set;
[0037] Calculate the data mean of the 3D point cloud data group, and use the data mean as the height coordinate of the corresponding local detail feature;
[0038] A three-dimensional point cloud model belonging to the second position is created using the position coordinates and the height coordinates of the local detail feature.
[0039] In a second aspect, the present application provides a three-dimensional point cloud data modeling device for road and bridge slope inspection, comprising:
[0040] A coordinate calculation unit, used to determine the current position coordinates using a satellite positioning method and determine the current altitude coordinates using a ranging method;
[0041] A first position adjustment unit, used to perform a position adjustment according to the current position coordinates and the current height coordinates, and move to the first position;
[0042] A first shooting unit, used for acquiring a first shot image at a first position;
[0043] Extracting local features in the first captured image and comparing the local features with local features in a reference captured image corresponding to the first position to obtain a spatial displacement adjustment amount;
[0044] A second position adjustment unit, used to move from the first position to the second position according to the spatial displacement adjustment amount;
[0045] A second shooting unit, used for acquiring a second shot image at a second position;
[0046] A feature extraction unit, used to extract local detail features in the second captured image and perform three-dimensional scanning on the local detail features to obtain three-dimensional point cloud data belonging to the local detail features;
[0047] The model creation unit is used to construct a three-dimensional point cloud model belonging to the second position using the three-dimensional point cloud data.
[0048] In a third aspect, the present application provides a server, the server comprising:
[0049] one or more memories for storing instructions; and
[0050] One or more processors, used to call and run the instructions from the memory to execute the method as described in the first aspect and any possible implementation of the first aspect.
[0051] In a fourth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium comprising:
[0052] Program, when the program is executed by a processor, the method described in the first aspect and any possible implementation of the first aspect is executed.
[0053] In a fifth aspect, the present application provides a computer program product, comprising program instructions. When the program instructions are executed by a computing device, the method described in the first aspect and any possible implementation of the first aspect is executed.
[0054] In a sixth aspect, the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above methods.
[0055] The chip system may be composed of chips, or may include chips and other discrete devices.
[0056] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wired or wireless means, or the processor and the memory can also be coupled on the same device. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 It is a schematic diagram of the step flow of a three-dimensional point cloud data modeling method provided in this application.
[0058] Figure 2 This is a schematic diagram of the principle of obtaining the current position coordinates and the current height coordinates provided by the present application.
[0059] Figure 3 This is a schematic diagram provided by the present application for dividing regions and determining a fixed collection location in each region.
[0060] Figure 4 This is a schematic diagram of the principle of extracting local features from a first captured image provided by the present application.
[0061] Figure 5 It is a schematic diagram of a characteristic grid provided in this application.
[0062] Figure 6 This is a schematic diagram of a process for obtaining a spatial displacement adjustment amount as an enlargement amount and / or a reduction amount provided by the present application.
[0063] Figure 7 This is a schematic diagram of a process for obtaining a spatial displacement adjustment amount as a lateral movement amount and / or a longitudinal movement amount provided by the present application. DETAILED DESCRIPTION
[0064] In order to more clearly understand the technical solution in this application, the relevant contents are first introduced.
[0065] Road and bridge slopes are generally divided into two categories. The first category is artificially trimmed road and bridge slopes with a regular appearance, which are achieved with the help of concrete retaining walls and concrete grids. In some areas, steel bars are used to fix the road and bridge slopes. The second category is natural slopes, which are not fixed or use methods such as hanging nets for basic shielding.
[0066] The technical solution in this application is further described in detail below in conjunction with the accompanying drawings.
[0067] This application discloses a 3D point cloud data modeling method for road and bridge slope inspection. Figure 1In some examples, the three-dimensional point cloud data modeling method for road and bridge slope inspection disclosed in this application includes the following steps:
[0068] S101, using satellite positioning to determine the current position coordinates and using ranging to determine the current altitude coordinates;
[0069] S102, performing a position adjustment according to the current position coordinates and the current height coordinates, and moving to the first position;
[0070] S103, acquiring a first captured image at a first position;
[0071] S104, extracting local features in the first captured image and comparing the local features with local features in a reference captured image corresponding to the first position to obtain a spatial displacement adjustment amount;
[0072] S105, moving from the first position to the second position according to the spatial displacement adjustment amount;
[0073] S106, acquiring a second captured image at a second position;
[0074] S107, extracting local detail features in the second captured image and performing three-dimensional scanning on the local detail features to obtain three-dimensional point cloud data belonging to the local detail features;
[0075] S108, constructing a three-dimensional point cloud model belonging to the second position using the three-dimensional point cloud data.
[0076] First of all, it needs to be explained that the executor of this application is the analysis server, and data collection is performed by a drone equipped with an image sensor and a lidar. The analysis server can be deployed separately or directly on the drone. For the sake of convenience of description, the executor will be referred to as a server below.
[0077] In step S101, the server (drone) first determines the current position coordinates using satellite positioning and determines the current height coordinates using ranging. The satellite positioning method provides plane coordinates, and the ranging method provides height coordinates. Figure 2 shown.
[0078] This is because the acquisition range of image sensors and lidar should be limited. Based on this, it is necessary to divide the road and bridge slopes into regions and determine a fixed acquisition position in each region, such as Figure 3 As shown, this method can avoid edge fusion of image data and three-dimensional point cloud data as much as possible.
[0079] It should be understood that the three-dimensional point cloud data can be understood as a combination of distance measurement data. Each distance measurement process will be affected by errors. When the three-dimensional point cloud data at two positions are edge-fused, the errors will cause the edge fusion to be unable to find the alignment reference.
[0080] In order to solve this problem, the present application uses a fixed collection position for processing, and each data collected at a fixed position is compared and analyzed with the benchmark data at the fixed position, thus avoiding the above-mentioned problem.
[0081] At the same time, comparing with baseline data can also reduce the difficulty of discovering problems. This is because if you only compare with the previous data, when the changes are relatively small, problems may go undetected. However, by comparing with baseline data, you can avoid the problem of small changes accumulating and going undetected.
[0082] At the same time, the benchmark data can also be updated. For example, after a maintenance or a period of safety, a new time will be selected for the benchmark data. This is because the slope of the road and bridge will also undergo normal deformation during long-term use, and this normal deformation is not a defect.
[0083] In step S102, a position adjustment is first performed based on the current position coordinates and the current height coordinates, and the position is moved to the first position. At this time, the first position is an imprecise position, which is mainly affected by the positioning error.
[0084] In step 103 and step S104, a first captured image is acquired at the first position and local features in the first captured image are extracted. Meanwhile, the local features are compared with local features in a reference captured image corresponding to the first position to obtain a spatial displacement adjustment amount.
[0085] The spatial displacement adjustment amount obtained at this time is used in step S105. In step S105, the drone moves according to the spatial displacement adjustment amount. At this time, the drone moves from the first position to the second position. The second position is a fixed collection position in each area mentioned above.
[0086] In step S106 and step S107, a second captured image is acquired at the second position, and local detail features in the second captured image are extracted and three-dimensionally scanned to obtain three-dimensional point cloud data belonging to the local detail features. Finally, in step S108, the three-dimensional point cloud data is used to construct a three-dimensional point cloud model belonging to the second position.
[0087] In general, in the above content, the present application uses two position adjustments. The first position adjustment is made with the help of satellite positioning and altitude measurement, and the second position adjustment is made with the help of captured images. The two position adjustment methods are to find the accurate shooting position of the second captured image.
[0088] Finally, a three-dimensional point cloud model belonging to the second position is established with the help of local detail features in the second captured image, rather than establishing a three-dimensional point cloud model for all positions corresponding to the second captured image. The advantage of this three-dimensional point cloud model establishment method is that the amount of three-dimensional point cloud data is greatly compressed, and the three-dimensional point cloud data used in the process of establishing the three-dimensional point cloud model corresponds to the local features at the corresponding position of the second captured image.
[0089] In the data processing process of the procedure, by analyzing the local features, it can be more intuitively determined whether there is a safety hazard at the corresponding position of the second captured image.
[0090] For some examples, see Figure 4 , the method of extracting local features in the first captured image is as follows:
[0091] S201, processing a first captured image using a grayscale processing method to obtain a first grayscale captured image;
[0092] S202, extracting features in the first grayscale captured image using the grayscale interval to obtain a first grayscale feature image, where the first grayscale feature image includes local features;
[0093] S203, determining the center position point of the local features included in the first grayscale feature image and calculating the grayscale value of the center position point, where the grayscale value of the center position point is the average grayscale value of the local features corresponding to the center position point;
[0094] S204, constructing a feature grid using the central position point, wherein the number of edges of the grids in the feature grid are all the same.
[0095] In step S01 to step S204, the first captured image is firstly grayscale processed, and then features are extracted in the first grayscale captured image using multiple grayscale intervals to obtain first grayscale feature images, the number of which is the same as the number of grayscale intervals.
[0096] The grayscale interval here refers to a sequence consisting of multiple continuous grayscale values. The grayscale values of pixels on the first grayscale captured image that are within the grayscale interval will be displayed on the first grayscale feature image corresponding to the grayscale interval.
[0097] The first grayscale feature image includes local features, and the local features here refer to the content presented on the first grayscale feature image.
[0098] Then, the center position point of the local feature included in the first grayscale feature image is determined and the grayscale value of the center position point is calculated. The local feature refers to some pixel points gathered together in the first grayscale feature image.
[0099] At this time, some restriction rules need to be set. For example, the area occupied by the clustered pixels should be larger than a set area value, and the clustered areas smaller than the set area value should be discarded.
[0100] The center point refers to the center point of the filtered clustering area.
[0101] The gray value of the center point is the mean gray value of the local feature corresponding to the center point.
[0102] In step S204, the central position point is used to construct a feature grid, and the number of edges of the grids in the feature grid is the same, such as Figure 5 As shown, Figure 5 The dotted line in the figure represents the mean gray value.
[0103] The local features in the reference captured image corresponding to the first position mentioned above are compared, and the local features in the reference captured image are also generated in the above manner.
[0104] The comparison at this time is to compare the similarity of the two feature grids. The specific method of comparison is to make the center position points on the two feature grids coincide with each other. Usually, it is also necessary to satisfy that the mean grayscale values of the corresponding center position points are the same.
[0105] The coincidence of two center position points means that the positions coincide or the straight-line distance is less than the set distance, and the same grayscale value means of the center position points means that the values of the two grayscale value means are the same or the difference is within an appropriate range.
[0106] Finally, the similarity of the two feature grids is judged based on the ratio of the number of overlapping center position points with the same grayscale value mean to the number of all center position points. Generally, two feature grids with a ratio greater than or equal to 0.95 are considered to be the same.
[0107] Another way to judge is as follows:
[0108] S301, processing a first captured image using a grayscale processing method to obtain a first grayscale captured image;
[0109] S302, creating at least one reference line on the first grayscale captured image;
[0110] S303, constructing an analysis curve or an analysis series using the pixel values of the pixel points on the reference line;
[0111] S304, using wavelet decomposition to decompose the analysis curve or calculate the characteristic series of the analysis series to determine a plurality of position characteristic points on the reference line, wherein the characteristic series includes a difference series or a quadratic difference series.
[0112] In step S301 to step S304, at least one reference line is first created on the first grayscale captured image, and then the pixel values of the pixels on the reference line are used to construct an analysis curve or an analysis series.
[0113] Then, the specific method of using the pixel values of the pixel points on the reference line to construct the analysis curve is to use the position of the pixel point as the horizontal coordinate and the gray value as the vertical coordinate. At this time, a group of discrete points will be obtained. The curve obtained by connecting these discrete points in sequence is the analysis curve.
[0114] The specific method of constructing the analysis series using the pixel values of the pixel points on the reference line is to use the position of the pixel point as the queue position and the gray value as the numerical value corresponding to the queue position.
[0115] When the analysis curve is decomposed by wavelet decomposition, multiple sinusoidal lines (curves) can be obtained. Each sinusoidal line has at least one different frequency, amplitude and phase, and each sinusoidal line has a clear starting position and end position.
[0116] A small number of sinusoidal lines are obtained by selecting using frequency and / or amplitude, and the starting position and the end position of these sinusoidal lines are the position feature points.
[0117] The specific method of calculating the characteristic sequence of the analysis sequence includes calculating the difference sequence or the quadratic difference sequence. The peak point on the curve corresponding to the difference sequence or the quadratic difference sequence is the position characteristic point.
[0118] In some possible implementations, at least two reference lines intersect each other, and an intersection point of the two reference lines is located on a local feature in the first captured image.
[0119] Based on this, the method of obtaining the spatial displacement adjustment amount is further explained, and the specific process is as follows:
[0120] Comparing the local features in the first captured image with the local features in the corresponding reference captured image to obtain an adjustment amount, the adjustment amount including an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount;
[0121] A spatial displacement adjustment amount is generated according to the adjustment amount, and the spatial displacement adjustment amount includes at least one of an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount.
[0122] Enlarge and reduce the amount of Figure 6 As shown, Figure 6The arrow in indicates that the local features in the first captured image need to be enlarged.
[0123] The lateral and longitudinal movement are as follows Figure 7 As shown, Figure 7 The arrow in indicates that the local features in the first captured image need to be moved.
[0124] When the local features in the first captured image are the same as the local features in the corresponding reference captured image, they are processed by an adjustment method, and the adjustment amounts involved in the adjustment method include an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount.
[0125] The amount of enlargement and reduction corresponds to forward movement and backward movement; the amount of lateral movement corresponds to movement in the horizontal direction, without forward or backward movement; the amount of longitudinal movement corresponds to movement in the vertical direction, without forward or backward movement.
[0126] The spatial displacement adjustment amount can be generated by the adjustment amount (the spatial displacement adjustment amount includes at least one of the enlargement amount, the reduction amount, the lateral movement amount and the longitudinal movement amount). This is a simple linear correspondence and will not be elaborated here.
[0127] In some examples, the specific method of extracting local detail features in the second captured image and performing three-dimensional scanning on the local detail features is as follows:
[0128] S401, grayscale processing is performed on the second captured image using a grayscale processing method to obtain a second grayscale captured image;
[0129] S402, extracting features in the second grayscale captured image using the grayscale interval to obtain a second grayscale feature image, where the second grayscale feature image includes local detail features;
[0130] S403, determining the edge contour and position coordinates of the local detail feature;
[0131] S404, creating a scanning area using the position coordinates and the edge contour of the local detail features and performing a three-dimensional scan on the scanning area to obtain a three-dimensional point cloud data set belonging to the scanning area.
[0132] The contents of step S401 and step S402 are the same as those of step S104 and its related contents, and are not repeated here. The edge contour of the local detail feature in step S402 refers to the edge area of the clustered pixels, and the position coordinates refer to the coordinates of the center point of the edge area of the clustered pixels.
[0133] In step S404, the position coordinates and the edge contours of the local detail features are used to create a scanning area and perform a three-dimensional scan on the scanning area to obtain a three-dimensional point cloud data set belonging to the scanning area.
[0134] The specific method of creating a scanning area is to assign coordinates to the edge contour of the local detail feature according to the setting coordinates, and then select in the 3D point cloud data according to the coordinates to obtain some 3D point cloud data points. These 3D point cloud data points are the 3D point cloud data group belonging to the scanning area.
[0135] The specific method of constructing a 3D point cloud model belonging to the second position using the 3D point cloud data is as follows:
[0136] S501, taking the position coordinates of the local detail features as the position coordinates of the corresponding three-dimensional point cloud data set;
[0137] S502, calculating the data mean of the three-dimensional point cloud data group, and using the data mean as the height coordinate of the corresponding local detail feature;
[0138] S503: Create a three-dimensional point cloud model belonging to the second position using the position coordinates and height coordinates of the local detail feature.
[0139] Through step S501 to step S503, a clear point coordinate and a height coordinate belonging to the point coordinate can be obtained, and these point coordinates can form a point coordinate set. Correspondingly, there will be a standard point coordinate set. Comparing the point coordinate set with the standard point coordinate set can provide two methods: position comparison and height comparison.
[0140] Reference here Figure 5 When the coordinates of some points change or the coordinates of some points do not change but the corresponding height coordinates change, it means that there may be problems at the corresponding positions of these point coordinates.
[0141] The present application also provides a three-dimensional point cloud data modeling device for road and bridge slope inspection, comprising:
[0142] A coordinate calculation unit, used to determine the current position coordinates using a satellite positioning method and determine the current altitude coordinates using a ranging method;
[0143] A first position adjustment unit, used to perform a position adjustment according to the current position coordinates and the current height coordinates, and move to the first position;
[0144] A first shooting unit, used for acquiring a first shot image at a first position;
[0145] Extracting local features in the first captured image and comparing the local features with local features in a reference captured image corresponding to the first position to obtain a spatial displacement adjustment amount;
[0146] A second position adjustment unit, used to move from the first position to the second position according to the spatial displacement adjustment amount;
[0147] A second shooting unit, used for acquiring a second shot image at a second position;
[0148] A feature extraction unit, used to extract local detail features in the second captured image and perform three-dimensional scanning on the local detail features to obtain three-dimensional point cloud data belonging to the local detail features;
[0149] The model creation unit is used to construct a three-dimensional point cloud model belonging to the second position using the three-dimensional point cloud data.
[0150] Further, extracting local features from the first captured image includes:
[0151] Processing the first captured image using a grayscale processing method to obtain a first grayscale captured image;
[0152] Using the grayscale interval to perform feature extraction in the first grayscale captured image, obtaining a first grayscale feature image, wherein the first grayscale feature image includes local features;
[0153] Determine the center position point of the local feature included in the first grayscale feature image and calculate the grayscale value of the center position point, where the grayscale value of the center position point is the average of the grayscale values of the local feature corresponding to the center position point;
[0154] A feature grid is constructed using the central position points, and the number of edges of the grids in the feature grid is the same.
[0155] Further, extracting local features from the first captured image includes:
[0156] Processing the first captured image using a grayscale processing method to obtain a first grayscale captured image;
[0157] creating at least one reference line on the first grayscale captured image;
[0158] Use the pixel values of the pixels on the reference line to construct an analysis curve or analysis series;
[0159] The analysis curve is decomposed by wavelet decomposition or the characteristic series of the analysis series is calculated to determine multiple position characteristic points on the reference line. The characteristic series includes a difference series or a quadratic difference series.
[0160] Furthermore, there are at least two reference lines intersecting each other, and the intersection point of the two reference lines is located on a local feature in the first captured image.
[0161] Furthermore, obtaining the spatial displacement adjustment amount includes:
[0162] Comparing the local features in the first captured image with the local features in the corresponding reference captured image to obtain an adjustment amount, the adjustment amount including an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount;
[0163] A spatial displacement adjustment amount is generated according to the adjustment amount, and the spatial displacement adjustment amount includes at least one of an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount.
[0164] Further, extracting local detail features in the second captured image and performing three-dimensional scanning on the local detail features includes:
[0165] Performing grayscale processing on the second captured image using a grayscale processing method to obtain a second grayscale captured image;
[0166] Using the grayscale interval to perform feature extraction in the second grayscale captured image, obtaining a second grayscale feature image, wherein the second grayscale feature image includes local detail features;
[0167] Determine the edge contour and position coordinates of local detail features;
[0168] A scanning area is created using position coordinates and edge contours of local detail features, and a three-dimensional scan is performed on the scanning area to obtain a three-dimensional point cloud data set belonging to the scanning area.
[0169] Further, constructing a three-dimensional point cloud model belonging to the second position using the three-dimensional point cloud data includes:
[0170] Taking the position coordinates of the local detail features as the position coordinates of the corresponding three-dimensional point cloud data set;
[0171] Calculate the data mean of the 3D point cloud data group, and use the data mean as the height coordinate of the corresponding local detail feature;
[0172] A three-dimensional point cloud model belonging to the second position is created using the position coordinates and the height coordinates of the local detail feature.
[0173] In one example, the unit in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.
[0174] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0175] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.
[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0177] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0179] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0180] It should also be understood that in various embodiments of the present application, the first, second, etc. are only used to indicate that multiple objects are different. For example, the first time window and the second time window are only used to indicate different time windows. They should not have any impact on the time window itself, and the first, second, etc. mentioned above should not impose any limitations on the embodiments of the present application.
[0181] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.
[0182] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0183] The present application also provides a server, the server comprising:
[0184] one or more memories for storing instructions; and
[0185] One or more processors are used to call and run the instructions from the memory to execute the method as described above.
[0186] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the terminal device and the network device perform operations of the terminal device and the network device corresponding to the above method.
[0187] The present application also provides a chip system, which includes a processor for implementing the functions involved in the above content, such as generating, receiving, sending, or processing the data and / or information involved in the above method.
[0188] The chip system may be composed of chips, or may include chips and other discrete devices.
[0189] The processor mentioned in any of the above places can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above-mentioned feedback information transmission method.
[0190] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wire or wireless means to support the chip system to implement various functions in the above embodiments. Alternatively, the processor and the memory can also be coupled on the same device.
[0191] Optionally, the computer instructions are stored in a memory.
[0192] Optionally, the memory is a storage unit within the chip, such as a register, a cache, etc. The memory can also be a storage unit within the terminal located outside the chip, such as a ROM or other types of static storage devices that can store static information and instructions, RAM, etc.
[0193] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.
[0194] The non-volatile memory may be a ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory.
[0195] The volatile memory may be a RAM, which is used as an external cache. There are many different types of RAM, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct memory bus RAM.
[0196] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.
Claims
1. A three-dimensional point cloud data modeling method for road and bridge slope inspection, characterized in that: include: Use satellite positioning to determine the current position coordinates and use ranging to determine the current altitude coordinates; Perform a position adjustment according to the current position coordinates and the current height coordinates, and move to the first position; Acquire a first captured image at a first position; Extracting a position reference feature in the first captured image and comparing the local feature with a local feature in a reference captured image corresponding to the first position to obtain a spatial displacement adjustment amount; Move according to the spatial displacement adjustment amount, from a first position to a second position; acquiring a second captured image at a second position; Extracting local detail features in the second captured image and performing three-dimensional scanning on the local detail features to obtain three-dimensional point cloud data belonging to the local detail features; constructing a three-dimensional point cloud model attributable to the second location using the three-dimensional point cloud data; Extracting the position reference feature in the first captured image includes: Processing the first captured image using a grayscale processing method to obtain a first grayscale captured image; creating at least one reference line on the first grayscale captured image; Use the pixel values of the pixels on the reference line to construct an analysis curve or analysis series; The analysis curve is decomposed by wavelet decomposition or the characteristic series of the analysis series is calculated to determine multiple position characteristic points on the reference line. The characteristic series includes a difference series or a quadratic difference series.
2. The three-dimensional point cloud data modeling method for road and bridge slope inspection according to claim 1 is characterized in that: Extracting the position reference feature in the first captured image includes: Processing the first captured image using a grayscale processing method to obtain a first grayscale captured image; Using the grayscale interval to perform feature extraction in the first grayscale captured image, obtaining a first grayscale feature image, wherein the first grayscale feature image includes a position reference feature; Determine the center position point of the position reference feature included in the first grayscale feature image and calculate the grayscale value of the center position point, where the grayscale value of the center position point is the average of the grayscale values of the position reference feature corresponding to the center position point; A feature grid is constructed using the central position points, and the number of edges of the grids in the feature grid is the same.
3. The three-dimensional point cloud data modeling method for road and bridge slope inspection according to claim 1 is characterized in that: At least two reference lines intersect each other, and an intersection point of the two reference lines is located on a position reference feature in the first captured image.
4. The three-dimensional point cloud data modeling method for road and bridge slope inspection according to claim 1 is characterized in that: The spatial displacement adjustment amount includes: Comparing the position reference feature in the first captured image with the local feature in the corresponding reference captured image to obtain an adjustment amount, the adjustment amount including an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount; A spatial displacement adjustment amount is generated according to the adjustment amount, and the spatial displacement adjustment amount includes at least one of an enlargement amount, a reduction amount, a lateral movement amount, and a longitudinal movement amount.
5. The three-dimensional point cloud data modeling method for road and bridge slope inspection according to claim 1 is characterized in that: Extracting local detail features from the second captured image and performing three-dimensional scanning on the local detail features includes: Performing grayscale processing on the second captured image using a grayscale processing method to obtain a second grayscale captured image; Using the grayscale interval to perform feature extraction in the second grayscale captured image, obtaining a second grayscale feature image, wherein the second grayscale feature image includes local detail features; Determine the edge contour and position coordinates of local detail features; A scanning area is created using position coordinates and edge contours of local detail features, and a three-dimensional scan is performed on the scanning area to obtain a three-dimensional point cloud data set belonging to the scanning area.
6. The three-dimensional point cloud data modeling method for road and bridge slope inspection according to claim 5 is characterized in that: Using the three-dimensional point cloud data to construct a three-dimensional point cloud model belonging to the second position includes: Taking the position coordinates of the local detail features as the position coordinates of the corresponding three-dimensional point cloud data set; Calculate the data mean of the 3D point cloud data group, and use the data mean as the height coordinate of the corresponding local detail feature; A three-dimensional point cloud model belonging to the second position is created using the position coordinates and the height coordinates of the local detail feature.
7. A three-dimensional point cloud data modeling device used for road and bridge slope inspection, characterized in that: include: A coordinate calculation unit, used to determine the current position coordinates using a satellite positioning method and determine the current altitude coordinates using a ranging method; A first position adjustment unit, used to perform a position adjustment according to the current position coordinates and the current height coordinates, and move to the first position; A first shooting unit, used for acquiring a first shot image at a first position; Extracting a position reference feature in the first captured image and comparing the local feature with a local feature in a reference captured image corresponding to the first position to obtain a spatial displacement adjustment amount; A second position adjustment unit, used to move from the first position to the second position according to the spatial displacement adjustment amount; A second shooting unit, used for acquiring a second shot image at a second position; A feature extraction unit, used to extract local detail features in the second captured image and perform three-dimensional scanning on the local detail features to obtain three-dimensional point cloud data belonging to the local detail features; A model creation unit, configured to construct a three-dimensional point cloud model belonging to the second position using the three-dimensional point cloud data; Extracting the position reference feature in the first captured image includes: Processing the first captured image using a grayscale processing method to obtain a first grayscale captured image; creating at least one reference line on the first grayscale captured image; Use the pixel values of the pixels on the reference line to construct an analysis curve or analysis series; The analysis curve is decomposed by wavelet decomposition or the characteristic series of the analysis series is calculated to determine multiple position characteristic points on the reference line. The characteristic series includes a difference series or a quadratic difference series.
8. A server, characterized in that: The server comprises: one or more memories for storing instructions; and One or more processors, configured to call and execute the instructions from the memory to perform the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer readable storage medium comprises: The program, when the program is executed by a processor, the method according to any one of claims 1 to 6 is executed.
Citation Information
Patent Citations
Photovoltaic power station unmanned aerial vehicle aerial photography pose adjusting method and device and terminal equipment
CN112184680A