A method and system for unmanned intelligent mapping with synchronous positioning

Abstract

The present invention relates to the technical field of intelligent surveying and mapping, and discloses a synchronous positioning unmanned intelligent surveying and mapping method and system, the method comprising: using sensors on unmanned surveying and mapping equipment to collect environmental data; performing continuous feature extraction on environmental images and environmental point cloud data; performing time-series matching of point cloud data features and key points of visual data to obtain environmental relative motion information of unmanned surveying and mapping equipment, and performing positioning and surveying of unmanned surveying and mapping equipment in combination with equipment motion data. The present invention performs feature matching based on the similarity between continuous features, realizes feature matching under different viewing angles, improves surveying and mapping accuracy, and iteratively calculates environmental relative motion information at different positioning moments based on the matching results of features corresponding to three-dimensional coordinate points, synchronously corrects the inertial measurement unit, improves the measurement accuracy of the measurement results, realizes synchronous positioning of unmanned surveying and mapping equipment, and obtains plane surveying and mapping modeling results of different unmanned surveying and mapping equipment positions.

Description

Technical Field

[0001] The present invention relates to the field of intelligent surveying and mapping, and in particular to an unmanned intelligent surveying and mapping method and system with synchronous positioning. Background Art

[0002] With the acceleration of global urbanization, urban planning, management and infrastructure construction are becoming increasingly complex, and traditional urban mapping methods are less efficient in dealing with these changes. Traditional mapping methods rely on manual operations, which are costly and time-consuming. Especially in large-scale or complex terrain environments, they are difficult to meet the rapid needs of contemporary urban construction. Improving the accuracy, efficiency and automation of urban mapping has become an important issue that needs to be solved urgently. SLAM technology has become an important technical approach to solving the problem of urban mapping. SLAM is a technology that uses sensors to perceive the surrounding environment in real time, build an environmental map and determine its own position while unmanned equipment is moving. It can effectively realize intelligent urban mapping. However, the environmental structure in the city is complex, with a large number of high-rise buildings, narrow alleys and dynamic obstacles (such as vehicles and pedestrians). These factors pose severe challenges to the real-time positioning and map construction of SLAM technology. Under the urban canyon effect (due to signal reflection and occlusion caused by high-rise buildings), GPS signals are often severely interfered, resulting in inaccurate or even failed positioning. Summary of the invention

[0003] In view of this, the present invention provides an unmanned intelligent surveying and mapping method with synchronous positioning, which realizes stable and accurate autonomous positioning and map construction by fusing multiple data such as visual information and laser scanning.

[0004] To achieve the above object, the present invention provides an unmanned intelligent surveying and mapping method with synchronous positioning, comprising the following steps:

[0005] S1: Collect environmental data using sensors deployed on unmanned mapping equipment, where the environmental data includes environmental images, environmental point cloud data, and equipment motion data;

[0006] S2: Continuously extract features from the environment image and the environment point cloud data to obtain key points of the environment image and features of the environment point cloud data;

[0007] S3: Match the point cloud data features and the key points of the visual data in time sequence to obtain the relative motion information of the unmanned mapping equipment, and locate the unmanned mapping equipment in combination with the equipment motion data to obtain the position coordinate data of the unmanned mapping equipment;

[0008] S4: Surveying and mapping the city three-dimensional map according to the location coordinate data and environmental data of the unmanned surveying equipment.

[0009] As a further improvement method of the present invention:

[0010] Optionally, collecting environmental data using sensors deployed on unmanned mapping equipment in step S1 includes:

[0011] A variety of sensors are deployed on the unmanned mapping equipment, including cameras, lidars, and inertial measurement units. The cameras are used to collect environmental images of the environment in which the unmanned mapping equipment is located. The lidars are used to emit laser pulses to the environment in which the unmanned mapping equipment is located, and the environmental point cloud data of the environment is calculated based on the returned results. The inertial measurement unit includes an accelerometer and a magnetometer, which are used to measure the acceleration and azimuth of the unmanned mapping equipment, forming the equipment motion data of the unmanned mapping equipment.

[0012] The unmanned mapping equipment moves along the preset driving route, and uses the sensors deployed on the unmanned mapping equipment to collect environmental data at different positioning times during the movement. The environmental data at the nth positioning time is , The environmental image, environmental point cloud data and device motion data at the nth positioning moment are respectively: , N represents the total number of positioning moments, and the time interval between adjacent positioning moments is .

[0013] Optionally, the environmental data The collection process is:

[0014] S11: Extract the acceleration and magnetometer reading of the unmanned surveying and mapping equipment at the nth positioning moment from the inertial measurement unit, and calculate the pitch angle of the unmanned surveying and mapping equipment at the nth positioning moment and azimuth :

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] in:

[0022] Indicates the rolling angle of the unmanned surveying and mapping equipment at the nth positioning moment;

[0023] It represents the acceleration components of the unmanned surveying and mapping equipment at the nth positioning moment, which are the acceleration along the X-axis, the acceleration along the Y-axis, and the acceleration along the Z-axis respectively;

[0024] Indicates the magnetometer reading of the unmanned surveying and mapping equipment at the nth positioning moment, which is the magnetometer reading along the X-axis, the magnetometer reading along the Y-axis, and the magnetometer reading along the Z-axis in sequence;

[0025] Correction value for magnetometer readings combining roll and pitch angles;

[0026] And the acceleration of the UAV mapping equipment at the nth positioning moment , Pitch angle and azimuth As device motion data ;

[0027] S12: The laser radar emits M groups of laser pulses to the environment where the unmanned mapping equipment is located, and calculates the time difference between the emission and return of different laser pulses. and azimuth The three-dimensional coordinates and pulse intensity of the point cloud data corresponding to each group of laser pulses are calculated to construct the environmental point cloud data at the nth positioning moment. :

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] in:

[0034] represents the three-dimensional coordinates of the point cloud data corresponding to the mth group of laser pulses, represents the pulse intensity of the pulse return result of the mth group of laser pulses;

[0035] Represents the three-dimensional coordinates in sequence The coordinates of the X-axis, Y-axis and Z-axis;

[0036] represents the vertical scanning angle of the laser radar when the mth group of laser pulses is emitted, , Indicates the preset vertical rotation angle;

[0037] It represents the horizontal scanning angle of the laser radar when the mth group of laser pulses is emitted. , Indicates the preset horizontal rotation angle;

[0038] represents the speed of light, represents the time difference from the emission to the return of the mth group of laser pulses;

[0039] S13: The camera takes a picture of the environment where the unmanned mapping equipment is located, and uses the picture result as the environment image , where the camera is a depth camera, and the collected environment image The representation is:

[0040] ;

[0041] ;

[0042] in:

[0043] Representing the environment image In the form of a pixel matrix with P rows and Q columns, Indicates the pixel in the pth row and qth column Pixel information of

[0044] Represents pixels Color values ​​in R, G, B color channels;

[0045] Represents pixels The depth value of .

[0046] Optionally, in step S2, continuous feature extraction is performed on the environment image and the environment point cloud data respectively, including:

[0047] Continuous feature extraction is performed on the environment image and the environment point cloud data respectively, where the environment image The continuous feature extraction process is:

[0048] S21: Environmental image The pixels in the image are filtered at multiple scales to obtain the three-layer filtering results of the pixels. The three-layer filtering results are:

[0049] ;

[0050] ;

[0051] in:

[0052] represents an exponential function with a natural constant as base;

[0053] Represents pixels The filtering result of the sth layer is , represents the scale coefficient of the sth layer;

[0054] Represents pixels Maximum values ​​in multiple color channels;

[0055] S22: If the filtering result of the pixel at any layer is The maximum value among the three-layer filtering results of the pixel neighborhood is marked as the environment image. The feature points of which pixels of The three-layer filtering result of the pixel neighborhood is ;

[0056] S23: Calculate the feature direction of the feature point and construct the feature vector of the feature point, where the environment image The feature vector of the kth feature point in is:

[0057] ;

[0058] ;

[0059] in:

[0060] Represents the SIFT (Scale Invariant Feature Transform) feature of the kth feature point;

[0061] Representing the environment image The feature direction of the kth feature point in ;

[0062] Representing the environment image The kth feature point in Neighborhood pixel coordinate set, Representing the environment image Medium Pixels Maximum values ​​in multiple color channels;

[0063] S24: Combine the depth value of the pixel and map the feature point to the world coordinate system to obtain the three-dimensional coordinates of the feature point. The three-dimensional coordinates, color information and feature vectors of all feature points are used as the environment image. Key points of visual data:

[0064] ;

[0065] in:

[0066] Representing the environment image The three-dimensional coordinates of the kth feature point in , Represents the color information of the kth feature point, where the color information is the color value of the RGB color channel. Representing the environment image The total number of feature points in

[0067] The three-dimensional coordinate calculation process of the feature point is as follows: obtain the pixel coordinates and depth value of the feature point, use the intrinsic parameters of the camera to convert the pixel coordinates to the camera coordinate system, and use the extrinsic parameters of the camera for further conversion to obtain the three-dimensional coordinates of the feature point in the world coordinate system.

[0068] Optionally, continuous feature extraction is performed on the environmental point cloud data, including:

[0069] Calculate the curvature of any point cloud data in the environmental point cloud data, where the environmental point cloud data The curvature of the point cloud data corresponding to the mth group of laser pulses is :

[0070] ;

[0071] in:

[0072] Representing environmental point cloud data Central and three-dimensional coordinates A set of three-dimensional coordinates whose distance is less than a preset threshold. , is a set of three-dimensional coordinates Any three-dimensional coordinates in ;

[0073] Represents a set of three-dimensional coordinates The number of three-dimensional coordinates in ;

[0074] The point cloud data in the area above the preset threshold is taken as the edge point cloud, and the three-dimensional coordinates and pulse intensity corresponding to the edge point cloud are taken as the environmental point cloud data. Point cloud data features :

[0075] ;

[0076] in:

[0077] Represents the 3D coordinates of the hth edge point cloud and pulse intensity , Representing environmental point cloud data The total number of edge point clouds.

[0078] Optionally, in step S3, time-sequential matching is performed between the point cloud data features and the key points of the visual data to obtain the environmental relative motion information of the unmanned mapping equipment, including:

[0079] The point cloud data features and visual data key points are matched in time sequence to obtain the relative motion information of the unmanned mapping equipment. And visual data key points The timing matching process between them is:

[0080] S31: Computing key points of visual data Three-dimensional coordinates of feature points and point cloud data features The coordinate distance between the three-dimensional coordinates of the edge point cloud. If the coordinate distance is less than a preset threshold, the color information of the feature point is added to the edge point cloud;

[0081] S32: Calculate the key points of visual data Key points with visual data If the similarity is higher than the preset threshold, the two feature points are regarded as the same feature point; the similarity between the feature points in the two visual data key points is:

[0082] ;

[0083] in:

[0084] Representing key points of visual data Middle Feature points and key points of visual data Middle The similarity between feature points;

[0085] represents the L2 norm;

[0086] S33: Construct a three-dimensional coordinate set of the same feature point:

[0087] ;

[0088] in:

[0089] represents the three-dimensional coordinate group of the same feature point of the rth group, Representing key points of visual data Key points with visual data The number of the same feature points in ;

[0090] Representing key points of visual data The three-dimensional coordinates of the same feature point in Representing key points of visual data The three-dimensional coordinates of the same feature point in;

[0091] S34: Objective function for constructing relative motion information of the environment based on the three-dimensional coordinate set set:

[0092] ;

[0093] in:

[0094] is the relative motion information of the environment to be solved; They are all three-dimensional vectors, corresponding to the X-axis, Y-axis, and Z-axis;

[0095] represents the constructed objective function, represents the L1 norm;

[0096] S35: Iteratively solve the objective function to obtain the relative motion information of the environment ; The iterative solution process of the relative information of the environment is:

[0097] S351: Initialize the relative information of the generation environment , set the current iteration number of the environment relative information to , the maximum number of iterations is Max, then the relative information of the environment The result of the iteration is ;

[0098] S352: Build and obtain environment relative information The corresponding Jacobian matrix :

[0099] ;

[0100] ;

[0101] ;

[0102] in:

[0103] T stands for transpose;

[0104] Express beg The partial derivative of

[0105] S353: Use the Jacobian matrix to iterate the relative information of the environment:

[0106] ;

[0107] in:

[0108] represents the iteration step length, represents the identity matrix with 6 rows and 6 columns;

[0109] S354: Order , until the maximum number of iterations is reached, and the environmental information obtained by the iteration at this time is used as the iterative solution result ;

[0110] The unmanned surveying and mapping equipment is positioned according to the relative motion information of the environment and combined with the equipment motion data to obtain the position coordinate data of the unmanned surveying and mapping equipment.

[0111] Optionally, the unmanned mapping equipment is positioned in combination with the equipment motion data to obtain the position coordinate data of the unmanned mapping equipment, including:

[0112] The acceleration in the device motion data is corrected using the relative motion information of the environment, where the acceleration data The correction formula is:

[0113] ;

[0114] ;

[0115] in:

[0116] is the acceleration of the unmanned surveying and mapping equipment at the nth positioning moment, is the acceleration The correction result of

[0117] The corrected acceleration is used to calculate the speed of the unmanned surveying and mapping equipment at different positioning times. The speed of the unmanned surveying and mapping equipment at the nth positioning time is: :

[0118] ;

[0119] in:

[0120] Indicates the initial speed of the unmanned mapping equipment, represents a differential operator;

[0121] According to the speed of the unmanned surveying and mapping equipment, the position of the unmanned surveying and mapping equipment at different positioning times is calculated, where the position of the unmanned surveying and mapping equipment at the nth positioning time is :

[0122] ;

[0123] in:

[0124] Indicates the initial position of the unmanned surveying and mapping equipment;

[0125] The position of the unmanned surveying and mapping equipment at different positioning times constitutes the position coordinate data of the unmanned surveying and mapping equipment .

[0126] Optionally, in step S4, mapping the three-dimensional city map according to the position coordinate data and environmental data of the unmanned mapping equipment includes:

[0127] The surveying and mapping is carried out according to the location coordinate data and environmental data of the unmanned surveying and mapping equipment, and the surveying and mapping process is as follows:

[0128] S41: extracting the position of the unmanned surveying and mapping equipment at different positioning times and the corresponding environmental data to form N groups of surveying and mapping data, wherein the nth group of surveying and mapping data includes the position of the unmanned surveying and mapping equipment at the nth positioning time. and environmental data;

[0129] S42: for each set of surveying and mapping data, extract the edge point cloud in the environment data, calculate the distance between any three edge point clouds, if the distance between any two edge point clouds is less than the threshold, it means that the three edge point clouds currently selected constitute a plane, then take the position of the unmanned surveying and mapping equipment as the coordinate origin, construct a plane based on the three-dimensional coordinates of the three edge point clouds, and generate the color and material of the plane based on the color information and pulse intensity of the edge point clouds; in the embodiment of the present invention, the plane color is the average value of the color information of the edge point clouds;

[0130] S43: Repeat step S42 to generate plane surveying and mapping modeling results for each group of surveying and mapping data, and use N groups of continuous plane surveying and mapping modeling results as surveying and mapping results.

[0131] In order to solve the above problems, the present invention provides an unmanned intelligent surveying and mapping system with synchronous positioning, the system comprising:

[0132] A data acquisition module, used to collect environmental data using sensors deployed on unmanned mapping equipment;

[0133] A feature extraction module is used to perform continuous feature extraction on the environment image and the environment point cloud data respectively to obtain key points of the environment image and features of the environment point cloud data;

[0134] The intelligent surveying and mapping device is used to time-sequentially match the point cloud data features and the key points of the visual data to obtain the environmental relative motion information of the unmanned surveying and mapping equipment, locate the unmanned surveying and mapping equipment in combination with the equipment motion data, obtain the position coordinate data of the unmanned surveying and mapping equipment, and survey and map the three-dimensional map of the city based on the position coordinate data of the unmanned surveying and mapping equipment and the environmental data.

[0135] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0136] A memory storing at least one instruction;

[0137] Communication interface, enabling electronic equipment to communicate; and

[0138] The processor executes the instructions stored in the memory to implement the unmanned intelligent surveying and mapping method with synchronous positioning described above.

[0139] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned unmanned intelligent surveying and mapping method with synchronous positioning.

[0140] Compared with the prior art, the present invention proposes an unmanned intelligent surveying and mapping method with synchronous positioning, which has the following advantages:

[0141] First, this scheme proposes a method for collecting environmental data, which uses a variety of sensors to collect environmental data of unmanned surveying and mapping equipment. According to the measurement results of the inertial measurement unit in the unmanned surveying and mapping equipment, the pitch angle and azimuth angle of the unmanned surveying and mapping equipment are calculated. Combined with the rotation angle of the lidar, the three-dimensional coordinates in the surrounding environment point cloud data are calculated, and continuous feature extraction is performed on the environmental image and the environmental point cloud data. Feature matching is performed based on the similarity between continuous features to achieve matching of three-dimensional coordinate points under different perspectives, improve the accuracy of surveying and mapping, and match the environmental image and the environmental point cloud data to obtain color information of different point cloud data, thereby realizing the surveying and mapping of color information.

[0142] At the same time, this scheme proposes an intelligent surveying and mapping method. According to the matching results of the features corresponding to the three-dimensional coordinate points, the relative motion information of the environment at different positioning times is iteratively calculated to achieve the matching of the three-dimensional coordinate points of the same feature, and the measurement results of the inertial measurement unit are synchronously corrected according to the relative motion information of the environment used for matching, so as to improve the measurement accuracy of the measurement results, and then the position of the unmanned surveying and mapping equipment at different positioning times is calculated. According to the extracted features, the three-dimensional map of the city at different locations is modeled, and the plane surveying and mapping modeling results of the positions of different unmanned surveying and mapping equipment are obtained, so as to realize the synchronous positioning and intelligent surveying and mapping of the unmanned surveying and mapping equipment in the scenario where GPS positioning is invalid. BRIEF DESCRIPTION OF THE DRAWINGS

[0143] Figure 1 A schematic diagram of a flow chart of an unmanned intelligent surveying and mapping method with synchronous positioning provided by an embodiment of the present invention;

[0144] Figure 2 A functional module diagram of an unmanned intelligent mapping system with synchronous positioning provided by an embodiment of the present invention;

[0145] Figure 2 In: 100 unmanned intelligent mapping system with synchronous positioning, 101 data acquisition module, 102 feature extraction module, 103 intelligent mapping device;

[0146] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0147] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0148] The embodiment of the present application provides an unmanned intelligent mapping method with synchronous positioning. The execution subject of the unmanned intelligent mapping method with synchronous positioning includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the unmanned intelligent mapping method with synchronous positioning can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0149] Embodiment 1:

[0150] S1: Collect environmental data using sensors deployed on unmanned mapping equipment, where the environmental data includes environmental images, environmental point cloud data, and equipment motion data.

[0151] In step S1, the environmental data is collected by using sensors deployed on the unmanned mapping equipment, including:

[0152] A variety of sensors are deployed on the unmanned mapping equipment, including cameras, lidars, and inertial measurement units. The cameras are used to collect environmental images of the environment in which the unmanned mapping equipment is located. The lidars are used to emit laser pulses to the environment in which the unmanned mapping equipment is located, and the environmental point cloud data of the environment is calculated based on the returned results. The inertial measurement unit includes an accelerometer and a magnetometer, which are used to measure the acceleration and azimuth of the unmanned mapping equipment, forming the equipment motion data of the unmanned mapping equipment.

[0153] The unmanned mapping equipment moves along the preset driving route, and uses the sensors deployed on the unmanned mapping equipment to collect environmental data at different positioning times during the movement. The environmental data at the nth positioning time is , The environmental image, environmental point cloud data and device motion data at the nth positioning moment are respectively: , N represents the total number of positioning moments, and the time interval between adjacent positioning moments is .

[0154] The environmental data The collection process is:

[0155] S11: Extract the acceleration and magnetometer reading of the unmanned surveying and mapping equipment at the nth positioning moment from the inertial measurement unit, and calculate the pitch angle of the unmanned surveying and mapping equipment at the nth positioning moment and azimuth :

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160]

[0161] ;

[0162] in:

[0163] Indicates the rolling angle of the unmanned surveying and mapping equipment at the nth positioning moment;

[0164] It represents the acceleration components of the unmanned surveying and mapping equipment at the nth positioning moment, which are the acceleration along the X-axis, the acceleration along the Y-axis, and the acceleration along the Z-axis respectively;

[0165] Indicates the magnetometer reading of the unmanned surveying and mapping equipment at the nth positioning moment, which is the magnetometer reading along the X-axis, the magnetometer reading along the Y-axis, and the magnetometer reading along the Z-axis in sequence;

[0166] Correction value for magnetometer readings combining roll and pitch angles;

[0167] And the acceleration of the UAV mapping equipment at the nth positioning moment , Pitch angle and azimuth As device motion data ;

[0168] S12: The laser radar emits M groups of laser pulses to the environment where the unmanned mapping equipment is located, and calculates the time difference between the emission and return of different laser pulses. and azimuth The three-dimensional coordinates and pulse intensity of the point cloud data corresponding to each group of laser pulses are calculated to construct the environmental point cloud data at the nth positioning moment. :

[0169] ;

[0170] ;

[0171] ;

[0172] ;

[0173] ;

[0174] in:

[0175] represents the three-dimensional coordinates of the point cloud data corresponding to the mth group of laser pulses, represents the pulse intensity of the pulse return result of the mth group of laser pulses;

[0176] Represents the three-dimensional coordinates in sequence The coordinates of the X-axis, Y-axis and Z-axis;

[0177] represents the vertical scanning angle of the laser radar when the mth group of laser pulses is emitted,

[0178] , Indicates the preset vertical rotation angle;

[0179] It represents the horizontal scanning angle of the laser radar when the mth group of laser pulses is emitted.

[0180] , Indicates the preset horizontal rotation angle;

[0181] represents the speed of light, represents the time difference from the emission to the return of the mth group of laser pulses;

[0182] S13: The camera takes a picture of the environment where the unmanned mapping equipment is located, and uses the picture result as the environment image , where the camera is a depth camera, and the collected environment image The representation is:

[0183] ;

[0184] ;

[0185] in:

[0186] Representing the environment image In the form of a pixel matrix with P rows and Q columns, Indicates the pixel in the pth row and qth column Pixel information of

[0187] Represents pixels Color values ​​in R, G, B color channels;

[0188] Represents pixels The depth value of .

[0189] S2: Continuously extract features from the environment image and the environment point cloud data respectively to obtain key points of the environment image and features of the environment point cloud data.

[0190] In the step S2, continuous feature extraction is performed on the environment image and the environment point cloud data respectively, including:

[0191] Continuous feature extraction is performed on the environment image and the environment point cloud data respectively, where the environment image The continuous feature extraction process is:

[0192] S21: Environmental image The pixels in the image are filtered at multiple scales to obtain the three-layer filtering results of the pixels. The three-layer filtering results are:

[0193] ;

[0194]

[0195] in:

[0196] represents an exponential function with a natural constant as base;

[0197] Represents pixels The filtering result of the sth layer is , represents the scale coefficient of the sth layer;

[0198] Represents pixels Maximum values ​​in multiple color channels;

[0199] S22: If the filtering result of the pixel at any layer is The maximum value among the three-layer filtering results of the pixel neighborhood is marked as the environment image. The feature points of which pixels of The three-layer filtering result of the pixel neighborhood is ;

[0200] S23: Calculate the feature direction of the feature point and construct the feature vector of the feature point, where the environment image The feature vector of the kth feature point in is:

[0201] ;

[0202] ;

[0203] in:

[0204] Represents the SIFT (Scale Invariant Feature Transform) feature of the kth feature point;

[0205] Representing the environment image The feature direction of the kth feature point in ;

[0206] Representing the environment image The kth feature point in Neighborhood pixel coordinate set, Representing the environment image Medium Pixels Maximum values ​​in multiple color channels;

[0207] S24: Combine the depth value of the pixel and map the feature point to the world coordinate system to obtain the three-dimensional coordinates of the feature point. The three-dimensional coordinates, color information and feature vectors of all feature points are used as the environment image. Key points of visual data:

[0208] ;

[0209] in:

[0210] Representing the environment image The three-dimensional coordinates of the kth feature point in , Represents the color information of the kth feature point, where the color information is the color value of the RGB color channel. Representing the environment image The total number of feature points in

[0211] The three-dimensional coordinate calculation process of the feature point is as follows: obtain the pixel coordinates and depth value of the feature point, use the intrinsic parameters of the camera to convert the pixel coordinates to the camera coordinate system, and use the extrinsic parameters of the camera for further conversion to obtain the three-dimensional coordinates of the feature point in the world coordinate system.

[0212] Continuous feature extraction of environmental point cloud data, including:

[0213] Calculate the curvature of any point cloud data in the environmental point cloud data, where the environmental point cloud data The curvature of the point cloud data corresponding to the mth group of laser pulses is :

[0214] ;

[0215] in:

[0216] Representing environmental point cloud data Central and three-dimensional coordinates A set of three-dimensional coordinates whose distance is less than a preset threshold. , is a set of three-dimensional coordinates Any three-dimensional coordinates in ;

[0217] Represents a set of three-dimensional coordinates The number of three-dimensional coordinates in ;

[0218] The point cloud data in the area above the preset threshold is taken as the edge point cloud, and the three-dimensional coordinates and pulse intensity corresponding to the edge point cloud are taken as the environmental point cloud data. Point cloud data features :

[0219] ;

[0220] in:

[0221] Represents the 3D coordinates of the hth edge point cloud and pulse intensity , Representing environmental point cloud data The total number of edge point clouds.

[0222] S3: Time-sequentially match the point cloud data features and the key points of the visual data to obtain the environmental relative motion information of the unmanned mapping equipment, and locate the unmanned mapping equipment in combination with the equipment motion data to obtain the position coordinate data of the unmanned mapping equipment.

[0223] In the step S3, the point cloud data features and the key points of the visual data are matched in time sequence to obtain the relative motion information of the environment of the unmanned mapping equipment, including:

[0224] The point cloud data features and visual data key points are matched in time sequence to obtain the relative motion information of the unmanned mapping equipment. And visual data key points The timing matching process between them is:

[0225] S31: Computing key points of visual data Three-dimensional coordinates of feature points and point cloud data features The coordinate distance between the three-dimensional coordinates of the edge point cloud. If the coordinate distance is less than a preset threshold, the color information of the feature point is added to the edge point cloud;

[0226] S32: Calculate the key points of visual data Key points with visual data If the similarity is higher than the preset threshold, the two feature points are regarded as the same feature point; the similarity between the feature points in the two visual data key points is:

[0227] ;

[0228] in:

[0229] Representing key points of visual data Middle Feature points and key points of visual data Middle The similarity between feature points;

[0230] represents the L2 norm;

[0231] S33: Construct a three-dimensional coordinate set of the same feature point:

[0232]

[0233] in:

[0234] represents the three-dimensional coordinate group of the same feature point of the rth group, Representing key points of visual data Key points with visual data The number of the same feature points in ;

[0235] Representing key points of visual data The three-dimensional coordinates of the same feature point in Representing key points of visual data The three-dimensional coordinates of the same feature point in;

[0236] S34: Objective function for constructing relative motion information of the environment based on the three-dimensional coordinate set set:

[0237] ;

[0238] in:

[0239] is the relative motion information of the environment to be solved; They are all three-dimensional vectors, corresponding to the X-axis, Y-axis, and Z-axis;

[0240] represents the constructed objective function, represents the L1 norm;

[0241] S35: Iteratively solve the objective function to obtain the relative motion information of the environment ; The iterative solution process of the relative information of the environment is:

[0242] S351: Initialize the relative information of the generation environment , set the current iteration number of the environment relative information to , the maximum number of iterations is Max, then the relative information of the environment The result of the iteration is ;

[0243] S352: Build and obtain environment relative information The corresponding Jacobian matrix :

[0244] ;

[0245] ;

[0246] ;

[0247] in:

[0248] T stands for transpose;

[0249] Express beg The partial derivative of

[0250] S353: Use the Jacobian matrix to iterate the relative information of the environment:

[0251] ;

[0252] in:

[0253] represents the iteration step length, represents the identity matrix with 6 rows and 6 columns;

[0254] S354: Order , until the maximum number of iterations is reached, and the environmental information obtained by the iteration at this time is used as the iterative solution result ;

[0255] The unmanned surveying and mapping equipment is positioned according to the relative motion information of the environment and combined with the equipment motion data to obtain the position coordinate data of the unmanned surveying and mapping equipment.

[0256] The unmanned mapping equipment positioning is performed in combination with the equipment motion data to obtain the position coordinate data of the unmanned mapping equipment, including:

[0257] The acceleration in the device motion data is corrected using the relative motion information of the environment, where the acceleration data The correction formula is:

[0258] ;

[0259] ;

[0260] in:

[0261] is the acceleration of the unmanned surveying and mapping equipment at the nth positioning moment, is the acceleration The correction result of

[0262] The corrected acceleration is used to calculate the speed of the unmanned surveying and mapping equipment at different positioning times. The speed of the unmanned surveying and mapping equipment at the nth positioning time is: :

[0263] ;

[0264] in:

[0265] Indicates the initial speed of the unmanned mapping equipment, represents a differential operator;

[0266] According to the speed of the unmanned surveying and mapping equipment, the position of the unmanned surveying and mapping equipment at different positioning times is calculated, where the position of the unmanned surveying and mapping equipment at the nth positioning time is :

[0267] ;

[0268] in:

[0269] Indicates the initial position of the unmanned surveying and mapping equipment;

[0270] The position of the unmanned surveying and mapping equipment at different positioning times constitutes the position coordinate data of the unmanned surveying and mapping equipment .

[0271] S4: Surveying and mapping the city three-dimensional map according to the location coordinate data and environmental data of the unmanned surveying equipment.

[0272] In the step S4, the three-dimensional map of the city is surveyed and mapped according to the position coordinate data and environmental data of the unmanned surveying equipment, including:

[0273] The surveying and mapping is carried out according to the location coordinate data and environmental data of the unmanned surveying and mapping equipment, and the surveying and mapping process is as follows:

[0274] S41: extracting the position of the unmanned surveying and mapping equipment at different positioning times and the corresponding environmental data to form N groups of surveying and mapping data, wherein the nth group of surveying and mapping data includes the position of the unmanned surveying and mapping equipment at the nth positioning time. and environmental data;

[0275] S42: for each set of surveying and mapping data, extract the edge point cloud in the environment data, calculate the distance between any three edge point clouds, if the distance between any two edge point clouds is less than the threshold, it means that the three edge point clouds currently selected constitute a plane, then take the position of the unmanned surveying and mapping equipment as the coordinate origin, construct a plane based on the three-dimensional coordinates of the three edge point clouds, and generate the color and material of the plane based on the color information and pulse intensity of the edge point clouds; in the embodiment of the present invention, the plane color is the average value of the color information of the edge point clouds;

[0276] S43: Repeat step S42 to generate plane surveying and mapping modeling results for each group of surveying and mapping data, and use N groups of continuous plane surveying and mapping modeling results as surveying and mapping results.

[0277] Embodiment 2:

[0278] like Figure 2 , which is a functional module diagram of an unmanned intelligent surveying and mapping system with synchronous positioning provided by an embodiment of the present invention, which can implement the unmanned intelligent surveying and mapping method with synchronous positioning in Example 1.

[0279] The unmanned intelligent mapping system 100 with synchronous positioning of the present invention can be installed in an electronic device. According to the functions to be implemented, the unmanned intelligent mapping system with synchronous positioning can include a data acquisition module 101, a feature extraction module 102 and an intelligent mapping device 103. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0280] The data acquisition module 101 is used to collect environmental data using sensors deployed on unmanned mapping equipment;

[0281] The feature extraction module 102 is used to perform continuous feature extraction on the environment image and the environment point cloud data respectively to obtain key points of the environment image and features of the environment point cloud data;

[0282] The intelligent surveying and mapping device 103 is used to time-sequentially match the point cloud data features and the key points of the visual data to obtain the environmental relative motion information of the unmanned surveying and mapping equipment, locate the unmanned surveying and mapping equipment in combination with the equipment motion data, obtain the position coordinate data of the unmanned surveying and mapping equipment, and survey and map the three-dimensional map of the city based on the position coordinate data of the unmanned surveying and mapping equipment and the environmental data.

[0283] In detail, the modules in the unmanned intelligent mapping system 100 for synchronous positioning described in the embodiment of the present invention are used in the same manner as described above. Figure 1 The same technical means are used as the unmanned intelligent surveying and mapping method with synchronous positioning described in the text, and can produce the same technical effects, so I will not go into details here.

[0284] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0285] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0286] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0287] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A synchronous positioning unmanned intelligent surveying and mapping method, characterized in that: The method comprises: S1: Collect environmental data using sensors deployed on unmanned mapping equipment, where the environmental data includes environmental images, environmental point cloud data, and equipment motion data; S2: Continuously extract features from the environment image and the environment point cloud data to obtain key points of the environment image and features of the environment point cloud data; S3: Match the point cloud data features and the key points of the visual data in time sequence to obtain the relative motion information of the unmanned mapping equipment, and locate the unmanned mapping equipment in combination with the equipment motion data to obtain the position coordinate data of the unmanned mapping equipment; S4: Surveying and mapping the city three-dimensional map according to the location coordinate data and environmental data of the unmanned surveying equipment; In the step S1, the environmental data is collected by using sensors deployed on the unmanned mapping equipment, including: A variety of sensors are deployed on the unmanned surveying and mapping equipment. The unmanned surveying and mapping equipment moves along the preset driving route, and uses the sensors deployed on the unmanned surveying and mapping equipment to collect environmental data at different positioning times during the movement. The environmental data at the nth positioning time is , The environmental image, environmental point cloud data and device motion data at the nth positioning moment are respectively: , N represents the total number of positioning moments, and the time interval between adjacent positioning moments is ; The environmental data The collection process is: S11: Extract the acceleration and magnetometer reading of the unmanned surveying and mapping equipment at the nth positioning moment from the inertial measurement unit, and calculate the pitch angle of the unmanned surveying and mapping equipment at the nth positioning moment and azimuth ; And the acceleration of the UAV mapping equipment at the nth positioning moment , Pitch angle and azimuth As device motion data ; S12: The laser radar emits M groups of laser pulses to the environment where the unmanned mapping equipment is located, and calculates the time difference between the emission and return of different laser pulses. and azimuth The three-dimensional coordinates and pulse intensity of the point cloud data corresponding to each group of laser pulses are calculated to construct the environmental point cloud data at the nth positioning moment. ; S13: The camera takes a picture of the environment where the unmanned mapping equipment is located, and uses the picture result as the environment image ; In the step S2, continuous feature extraction is performed on the environment image and the environment point cloud data respectively, including: Continuous feature extraction is performed on the environment image and the environment point cloud data respectively, where the environment image The continuous feature extraction process is: S21: Environmental image Perform multi-scale filtering on the pixels in to obtain the three-layer filtering results of the pixels; S22: If the filtering result of the pixel at any layer is The maximum value among the three-layer filtering results of the pixel neighborhood is marked as the environment image. The characteristic points of S23: Calculate the feature direction of the feature point and construct a feature vector of the feature point; S24: Combine the depth value of the pixel and map the feature point to the world coordinate system to obtain the three-dimensional coordinates of the feature point. The three-dimensional coordinates, color information and feature vectors of all feature points are used as the environment image. Key points of visual data: ； in: Represents the environment image The three-dimensional coordinates of the kth feature point in , Represents the color information of the kth feature point, where the color information is the color value of the RGB color channel. Represents the environment image The total number of feature points in Continuous feature extraction of environmental point cloud data, including: Calculate the curvature of any point cloud data in the environmental point cloud data, where the environmental point cloud data The curvature of the point cloud data corresponding to the mth group of laser pulses is : ； in: Representing environmental point cloud data Central and three-dimensional coordinates A set of three-dimensional coordinates whose distance is less than a preset threshold. , is a set of three-dimensional coordinates Any three-dimensional coordinates in ; Represents a set of three-dimensional coordinates The number of three-dimensional coordinates in ; The point cloud data in the area above the preset threshold is taken as the edge point cloud, and the three-dimensional coordinates and pulse intensity corresponding to the edge point cloud are taken as the environmental point cloud data. Point cloud data features : ； in: Represents the 3D coordinates of the hth edge point cloud and pulse intensity , Representing environmental point cloud data The total number of edge point clouds; In the step S3, the point cloud data features and the key points of the visual data are matched in time sequence to obtain the relative motion information of the environment of the unmanned mapping equipment, including: The point cloud data features and visual data key points are matched in time sequence to obtain the relative motion information of the unmanned mapping equipment. And visual data key points The timing matching process between them is: S31: Computing key points of visual data 3D coordinates of feature points and point cloud data features The coordinate distance between the three-dimensional coordinates of the edge point cloud. If the coordinate distance is less than a preset threshold, the color information of the feature point is added to the edge point cloud; S32: Calculate the key points of visual data Key points with visual data If the similarity is higher than the preset threshold, the two feature points are regarded as the same feature point. S33: construct a three-dimensional coordinate set set of the same feature point; S34: constructing an objective function of the relative motion information of the environment based on the three-dimensional coordinate set set; S35: Iteratively solve the objective function to obtain the relative motion information of the environment ; The unmanned surveying and mapping equipment is positioned according to the relative motion information of the environment and combined with the equipment motion data to obtain the position coordinate data of the unmanned surveying and mapping equipment.

2. The method for unmanned intelligent surveying and mapping with synchronous positioning according to claim 1, characterized in that: The unmanned mapping equipment positioning is performed in combination with the equipment motion data to obtain the position coordinate data of the unmanned mapping equipment, including: Use the relative motion information of the environment to correct the acceleration in the device motion data; The corrected acceleration is used to calculate the speed of the unmanned surveying and mapping equipment at different positioning times. The speed of the unmanned surveying and mapping equipment at the nth positioning time is: ; According to the speed of the unmanned surveying and mapping equipment, the position of the unmanned surveying and mapping equipment at different positioning times is calculated, where the position of the unmanned surveying and mapping equipment at the nth positioning time is : ； in: Indicates the initial position of the unmanned surveying and mapping equipment; The position of the unmanned surveying and mapping equipment at different positioning times constitutes the position coordinate data of the unmanned surveying and mapping equipment .

3. The method for unmanned intelligent surveying and mapping with synchronous positioning according to claim 1, characterized in that: In the step S4, the three-dimensional map of the city is surveyed and mapped according to the position coordinate data and environmental data of the unmanned surveying equipment, including: The surveying and mapping is carried out according to the location coordinate data and environmental data of the unmanned surveying and mapping equipment, and the surveying and mapping process is as follows: S41: extracting the position of the unmanned surveying and mapping equipment at different positioning times and the corresponding environmental data to form N groups of surveying and mapping data, wherein the nth group of surveying and mapping data includes the position of the unmanned surveying and mapping equipment at the nth positioning time. and environmental data; S42: For each set of surveying and mapping data, edge point clouds in the environment data are extracted, and the distance between any three edge point clouds is calculated. If the distance between any two edge point clouds is less than a threshold, it means that the three edge point clouds currently selected constitute a plane. Then, the position of the unmanned surveying and mapping equipment is used as the coordinate origin, and a plane is constructed based on the three-dimensional coordinates of the three edge point clouds. The color and material of the plane are generated based on the color information and pulse intensity of the edge point clouds. S43: Repeat step S42 to generate plane surveying and mapping modeling results for each group of surveying and mapping data, and use N groups of continuous plane surveying and mapping modeling results as surveying and mapping results.

4. A synchronous positioning unmanned intelligent surveying and mapping system, characterized in that: The system comprises: A data acquisition module, used to collect environmental data using sensors deployed on unmanned mapping equipment; A feature extraction module is used to perform continuous feature extraction on the environment image and the environment point cloud data respectively to obtain key points of the environment image and features of the environment point cloud data; An intelligent surveying and mapping device is used to time-sequentially match point cloud data features and key points of visual data to obtain environmental relative motion information of unmanned surveying and mapping equipment, locate the unmanned surveying and mapping equipment in combination with the equipment motion data, obtain position coordinate data of the unmanned surveying and mapping equipment, and survey and map a three-dimensional city map based on the position coordinate data of the unmanned surveying and mapping equipment and environmental data, so as to realize an unmanned intelligent surveying and mapping method with synchronous positioning as described in any one of claims 1 to 3.

Images