Inspection vehicle positioning system based on NRTK and visual information

By combining NRTK and visual SLAM technology, a high-precision map is constructed, which solves the problem of high equipment investment in the existing inspection vehicle positioning methods, and achieves the effect of high-precision positioning and low operating costs.

CN119022939AActive Publication Date: 2024-11-26TSINGHUA SHENZHEN INTERNATIONAL GRADUATE SCHOOL +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411138371.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-26
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

The existing inspection vehicle positioning methods require high equipment and sensor investment, resulting in high operating costs and improved accuracy requires relying on expensive high-precision sensors, reducing economic benefits.

Method used

Using the inspection vehicle positioning system based on NRTK and visual information, combined with the prior map construction subsystem and the visual SLAM positioning subsystem, the precise global latitude and longitude coordinates and visual SLAM technology provided by NRTK are used to build high-precision maps, and restore and depth through motion structures Learn to position the model to reduce dependence on high-precision sensors.

Benefits of technology

It realizes high-precision positioning while reducing operating costs, reducing the demand for high-precision sensors and equipment, simplifies system configuration and maintenance, and improves economic benefits and flexibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119022939A_ABST
    Figure CN119022939A_ABST
Patent Text Reader

Abstract

The invention discloses an inspection vehicle positioning system based on NRTK and visual information, the inspection vehicle positioning system comprises a prior mapping subsystem and a visual SLAM positioning subsystem, the prior mapping subsystem operates firstly and then operates the visual SLAM positioning subsystem after obtaining a point cloud map, and the visual SLAM positioning subsystem loads all point cloud map information at a time before operation. The invention provides an inspection vehicle positioning system based on NRTK and visual information, which can reduce the operation cost of the inspection vehicle and improve economic benefits while ensuring high-precision positioning of the inspection vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of positioning systems, and in particular to a patrol vehicle positioning system based on NRTK and visual information. Background Art

[0002] Inspection vehicles in on-street parking lots need to patrol the parking area for a long time and take pictures of the parking situation in the parking lot at any time to generate billing information. Since the inspection vehicle needs to record the parking location and duration, its own positioning function is crucial.

[0003] The existing high-precision patrol car positioning methods either rely on special equipment or rely on multiple sensors working together. In terms of special equipment, there are currently methods using Bluetooth beacons, embedded metal parking space signs, RFID electronic tags, and parking space QR codes. These methods bind special equipment with signal transmission or unique identification characteristics to parking spaces in on-street parking lots in advance. When the patrol car passes by the parking space, the device transmits a signal, or the patrol car detects the signal or characteristics of the device to complete the identification of the parking space and estimate its own position. The positioning method that relies on multiple sensors is to synchronously process the information collected by multiple sensors with high precision, such as inertial measurement units, lidars, or high-precision GPS signal receivers, and jointly estimate the position of the patrol car.

[0004] In the patrol vehicle positioning method that relies on special equipment, it is necessary to produce and install a large number of Bluetooth beacons, embedded metal parking space signs, RFID electronic tags, etc., and it may even be necessary to modify the existing road surface to a certain extent, which brings high usage costs. However, if this positioning method wants to improve the accuracy, it is necessary to rely on higher-precision sensors, such as precise inertial measurement units, high-precision lidars, or high-precision GPS signal receivers. These high-precision devices are often expensive or require continuous rental services (such as high-precision GPS positioning services), which greatly increases the manufacturing cost of the patrol vehicle and thus reduces the economic benefits of the patrol vehicle. Summary of the invention

[0005] The purpose of the present invention is to provide a patrol vehicle positioning system based on NRTK and visual information, which can ensure high-precision positioning of the patrol vehicle while reducing the operating cost of the patrol vehicle and improving economic benefits.

[0006] The technical solution adopted by the inspection vehicle positioning system based on NRTK and visual information disclosed in the present invention is:

[0007] A patrol vehicle positioning system based on NRTK and visual information, including a priori mapping subsystem and a visual SLAM positioning subsystem. The priori mapping subsystem runs first, and after obtaining a point cloud map, the visual SLAM positioning subsystem runs again. Before the visual SLAM positioning subsystem runs, all point cloud map information is loaded at one time.

[0008] The input data of the prior mapping subsystem is the image and the NRTK position information corresponding to the image. NRTK provides accurate global longitude and latitude coordinates, which can be converted into spatial three-dimensional coordinates through the Earth-centered Earth-fixed coordinate system. The image and its corresponding NRTK position information can be used together to perform constrained motion structure recovery, thereby obtaining the image's position, posture and spatial point cloud map. At this point, the prior mapping subsystem completes its work;

[0009] The visual SLAM positioning subsystem uses the visual SLAM technology to construct its own continuous position and attitude estimation based on the image information collected in real time, and at the same time determines the position of the current map based on the point cloud map, and provides it to the visual SLAM for common constrained position and attitude estimation. When the visual SLAM subsystem is in operation, it will also periodically transmit images to the cloud server;

[0010] The CLIP model in the cloud server calculates the CLIP spatial representation vector of the image and compares it with the text description of key locations on the path passed to provide a rough geographic location observation.

[0011] As a preferred solution, the construction of the point cloud map needs to use motion structure recovery technology, which is regarded as an optimization task to minimize the reprojection error of all images. The optimization goal is to obtain N spatial points, assuming in A spatial point p i 2D image points mapped to the camera at the initial pose The mapping relationship is:

[0012]

[0013] in is a matrix determined by the physical properties of the camera. i Mapped to a pose Location is The camera's viewing angle Then the mapping relationship is:

[0014]

[0015] in Yes iThe inverse of the z-axis coordinate after the spatial change. When a spatial point is observed by M cameras, the posture of the M cameras is Location The parameters will form a mutual constraint relationship, and finally only one optimization goal needs to be constructed to complete nonlinear optimization:

[0016]

[0017] in Yes i Mapped to the actual coordinates of the jth camera. In addition, since monocular vision cannot restore depth, it is generally not optimized. The exact value of the pivot frame is not used, but rather as a ratio: is regarded as 1, and the depth of other frames is its multiple, because the map containing N map points is to be restored, so the final optimization goal is:

[0018]

[0019] Instead of starting with random parameters, we first use feature point tracking, epipolar geometry, triangulation, etc. to obtain rough initial values, and then start optimizing from the initial values.

[0020] It is necessary to add additional absolute position information. Assume that the precise position information of L cameras has been obtained through NRTK signals or actual measurements. At this point, the optimization objective can be modified to:

[0021]

[0022] The final optimization result Construct a sparse point cloud map, It can also provide positioning reference for some situations when the map is used later.

[0023] As a preferred solution, the location information obtained by the NRTK is expressed in longitude, latitude and altitude, and the mapping method is performed in a Cartesian coordinate system. Therefore, it is necessary to convert the longitude and latitude of the earth and the altitude coordinates into Cartesian coordinates. The selected Cartesian system is the geocentric earth-fixed coordinate system. In the geocentric earth-fixed coordinate system, the center of mass of the earth is located at the origin, the Z axis points to the North Pole direction of the earth's rotation axis, the X axis is located in the equatorial plane, pointing to the prime meridian, and the Y axis is perpendicular to the XZ plane, eastward, forming a right-handed coordinate system. In this coordinate system, the rotation of the earth causes the positions of all ground points to be fixed in three-dimensional space, which is called geo-fixed, that is, the coordinates of any fixed point on the earth in this coordinate system will not change with time;

[0024] In the Earth-centered Earth-fixed coordinate system, the position of any point can be represented by three coordinate values: X, Y, and Z. Specifically, the latitude φ, longitude λ, and altitude h of a point on the Earth's surface or near it can be converted into the Earth-centered Earth-fixed coordinate system through a specific conversion formula to calculate the radius N of the circle where the point is located. The formula is:

[0025]

[0026] Among them, a is the semi-major axis of the earth, and e is the first eccentricity of the earth, which is calculated as follows:

[0027]

[0028] b is the Earth's semi-minor axis, and the geographic coordinates are converted to the Earth-centered Earth-fixed coordinate system using the following formula:

[0029]

[0030] Each t j can be expressed as [X, Y, Z] T .

[0031] As a preferred solution, in the visual SLAM positioning subsystem, features that can describe the global image are added. The global features of the image can be obtained by aggregating local features. One image obtains N descriptors with a dimension of D. We can first cluster these N descriptors to obtain K cluster centers. The overall characteristics of the image are Then V is calculated as:

[0032] Where V(k) represents the kth vector in the overall feature, α k (x i ) is an indicator function, if x i The cluster center is c k , then α k (x i )=1, otherwise α k (x i )=0. V is an aggregated local descriptor vector, which uses the vector learned from the neural network to replace the cluster center obtained by clustering In addition, due to α k (x i ) is a non-differentiable function and cannot be parameterized, so it needs to be modified as follows:

[0033]

[0034] where w k , b k To learn the parameters, this modification actually replaces the original indicator function with a continuous probability function, so the new VLAD calculation method is:

[0035]

[0036] After having both global features and local features, the closest image is first retrieved from the map based on the global features, and then the local features are used for precise positioning.

[0037] As a preferred solution, the CLIP model is mainly used to perform language enhancement functions for map positioning: collect pictures during the driving of the inspection vehicle, and transmit the pictures to the cloud server to calculate their feature vectors using the CLIP model; at the same time, use the CLIP model to calculate the feature vector of each text in the path text database, perform correlation calculation, and judge whether the current inspection vehicle has traveled to a certain landmark location based on the correlation score; if so, provide coordinate information near the location to assist positioning, otherwise perform pure visual positioning normally.

[0038] As a preferred solution, the path text database is a text description database of the inspection route of the inspection vehicle, which contains several natural languages ​​specified by humans to represent key locations on the route, collectively referred to as path texts, and a geographic coordinate is assigned to each path text.

[0039] The beneficial effects of the patrol vehicle positioning system based on NRTK and visual information disclosed by the present invention are as follows: the system mainly includes two main subsystems: a priori mapping subsystem and a visual SLAM positioning subsystem. The priori mapping subsystem generates a high-precision on-street parking lot map by collecting image information and high-precision positioning information of NRTK with the help of motion structure recovery technology; the visual SLAM positioning subsystem determines its own position according to the image and map in real time when the patrol vehicle is patrolling. The priori mapping subsystem only needs to be run once, and its result can be used by the visual SLAM positioning subsystem for a long time, so NRTK related services do not need to be continuously leased, and the multiple sensors involved in the system do not need to be accurately calibrated, which has the characteristics of low cost, simple implementation and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a structural schematic diagram of a patrol vehicle positioning system based on NRTK and visual information of the present invention.

[0041] Figure 2 It is a schematic diagram of a sliding window and key frame in a visual SLAM of a patrol vehicle positioning system based on NRTK and visual information of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described and illustrated below in conjunction with specific embodiments and accompanying drawings:

[0043] Please refer to Figure 1 , a patrol vehicle positioning system based on NRTK and visual information, including a priori mapping subsystem and a visual SLAM positioning subsystem. The priori mapping subsystem runs first, and after obtaining the point cloud map, the visual SLAM positioning subsystem runs. Before the visual SLAM positioning subsystem runs, all the point cloud map information will be loaded at one time;

[0044] The input data of the prior mapping subsystem is the image and the NRTK positioning information corresponding to the image. NRTK provides accurate global longitude and latitude coordinates (with an error of centimeters), which can be converted into three-dimensional spatial coordinates through the Earth-Centered, Earth-Fixed (ECEF) coordinate system. The image and its corresponding NRTK position information can be used together to perform constrained motion structure recovery (SFM) to obtain the image's position, posture, and spatial point cloud map. At this point, the prior mapping subsystem completes its work.

[0045] The construction of point cloud map requires the use of motion structure recovery technology, which is regarded as an optimization task to minimize the reprojection error of all images. The optimization goal is to obtain N spatial points, assuming in A spatial point p i 2D image points mapped to the camera at the initial pose The mapping relationship is:

[0046]

[0047] in is a matrix determined by the physical properties of the camera. i Mapped to a pose Location is The camera's viewing angle Then the mapping relationship is:

[0048]

[0049] in Yes i The inverse of the z-axis coordinate after the spatial change. When a spatial point is observed by M cameras, the posture of the M cameras is Location The parameters will form a mutual constraint relationship, and finally only one optimization goal needs to be constructed to complete nonlinear optimization:

[0051] Recovery depth, so generally not optimized The exact value of the pivot frame is not used, but rather as a ratio: is regarded as 1, and the depth of other frames is its multiple, because the map containing N map points is to be restored, so the final optimization goal is

[0052]

[0053] Instead of starting with random parameters, we first use feature point tracking, epipolar geometry, triangulation, etc. to obtain rough initial values, and then start optimizing from the initial values.

[0054] So far, the restored map does not have the physical scale of the real world and cannot be used directly. Additional absolute position information must be added, such as the t of some cameras. j Assume that the precise position information of L cameras has been obtained through NRTK signals or actual measurements. At this time, the optimization objective can be modified to

[0055]

[0056] The final optimization result Construct a sparse point cloud map, It can also provide positioning reference for some situations when the map is used later.

[0057] The location information obtained by NRTK is expressed in longitude, latitude and altitude. The mapping method is carried out in the Cartesian coordinate system. Therefore, it is necessary to convert the earth's longitude, latitude and altitude coordinates into Cartesian coordinates. The selected Cartesian system is the geocentric earth-fixed coordinate system. In the geocentric earth-fixed coordinate system, the center of mass of the earth is located at the origin, the Z axis points to the North Pole direction of the earth's rotation axis, the X axis is located in the equatorial plane, pointing to the prime meridian, and the Y axis is perpendicular to the XZ plane, eastward, forming a right-handed coordinate system. In this coordinate system, the rotation of the earth causes the positions of all ground points to be fixed in three-dimensional space. Therefore, it is called geo-fixed, that is, the coordinates of any fixed point on the earth in this coordinate system will not change with time.

[0058] In the Earth-centered Earth-fixed coordinate system, the position of any point can be represented by three coordinate values: X, Y, and Z. Specifically, the latitude φ, longitude λ, and altitude h of a point on the Earth's surface or near it can be converted into the Earth-centered Earth-fixed coordinate system through a specific conversion formula to calculate the radius N of the circle where the point is located. The formula is:

[0059]

[0060] Among them, a is the semi-major axis of the earth, which is about 6378137.0 meters, and e is the first eccentricity of the earth, which is calculated as follows:

[0061]

[0062] b is the Earth's semi-minor axis, which is approximately 6356752.3 meters. The following formula is used to convert geographic coordinates to the Earth-centered Earth-fixed coordinate system:

[0063]

[0064] Each t j can be expressed as [X, Y, Z] T .

[0065] The visual SLAM positioning subsystem first uses visual SLAM technology to build its own continuous position and attitude estimation based on the real-time collected image information, and at the same time determines the current location of the map based on the point cloud map, and provides it to the visual SLAM to jointly constrain the position and attitude estimation.

[0066] When the visual SLAM subsystem is in operation, the image is also transmitted to the cloud server at intervals. The CLIP model in the cloud server calculates the CLIP space representation vector of the image and compares it with the text description of the key locations on the path passed to provide a rough geographic location observation.

[0067] Visual SLAM in the visual positioning subsystem is an optimization task with equation (2) as the goal. All observation points are point cloud map points generated in the prior map creation phase. However, for efficiency considerations, keyframes and sliding windows are often used, such as Figure 2 As shown. The concept of keyframes mainly comes from: during the camera movement, some image frames have obvious movement, while some image frames have no obvious difference (for example, when the camera is temporarily still). In order to reduce the number of optimized images in SLAM, it is necessary to select images with obvious movement between frames as keyframes, which will not destroy the continuity of the image and save computing resources for redundant images. However, the keyframe method will still maintain a huge optimized image library in long-distance SLAM, causing the system to eventually stop due to resource exhaustion. Therefore, a sliding window method is needed. This method only maintains a fixed number of keyframes for optimization. New keyframes are added to the back of the window, and the keyframes at the front of the window are discarded, so that the current window always maintains the latest part of the keyframes.

[0068] Map positioning uses image retrieval technology and feature point matching technology based on deep learning. In traditional positioning methods, the map is understood as a codebook containing a large number of map points. When the map is needed, the current image is matched with the codebook content to find the closest codebook information, and then this information is used to solve the current positioning information. The basic elements of information in the codebook are generally map point descriptors, such as ORB, SIFT, etc. These map point descriptors are generally calculated based on local image information, such as brightness gradient, texture features, etc. These local information can be matched quickly and accurately when the scene is consistent. However, when encountering scenes with changing light and surface changes, the accuracy of this local information is greatly reduced. Therefore, it is not enough to use only local image information, and features that can describe the global image must be added.

[0069] The global features of an image can be obtained by aggregating local features. An image obtains N descriptors with a dimension of D. We can first cluster these N descriptors to obtain K cluster centers. The overall characteristics of the image are Then V is calculated as:

[0070] Where V(k) represents the kth vector in the overall feature, α k (x i ) is an indicator function, if x i The cluster center is c k , then α k (x i )=1, otherwise α k (x i )=0. V is an aggregated local descriptor vector, which uses the vector learned from the neural network to replace the cluster center obtained by clustering In addition, due to α k (x i ) is a non-differentiable function and cannot be parameterized, so it needs to be modified as follows:

[0071]

[0072] where w k , b k To learn the parameters, this modification actually replaces the original indicator function with a continuous probability function, so the new VLAD calculation method is:

[0073]

[0074] After having both global features and local features, the closest image is first retrieved from the map based on the global features, and then the local features are used for precise positioning.

[0075] The CLIP model will be fine-tuned in advance according to the inspection vehicle route. The fine-tuning method is: collect images of key locations on the inspection vehicle's fixed route, such as the intersection of two main roads and near iconic buildings, and manually annotate each image with a natural language text, such as "the intersection of Road A and Road B", "the square in front of Building C", etc., and then use supervised training to fine-tune the CLIP model. In addition, it is necessary to build a database to record the geographical coordinates corresponding to each natural language. During the actual operation, if the collected image matches a natural language, the geographical coordinates corresponding to the natural language are provided as an observation of the visual SLAM positioning subsystem.

[0076] The CLIP model is mainly used for language enhancement of map positioning: pictures are collected during the driving of the inspection vehicle, and the pictures are transmitted to the cloud server, and their feature vectors are calculated using the CLIP model; at the same time, the CLIP model is used to calculate the feature vector of each text in the path text database, and correlation calculation is performed. The correlation score is used to determine whether the current inspection vehicle has traveled to a certain landmark location; if so, coordinate information near the location is provided to assist positioning, otherwise pure visual positioning is performed normally.

[0077] The path text database is a text description database of the inspection route of the patrol vehicle. It contains several natural languages ​​specified by humans to represent key locations on the route, collectively referred to as path texts, and a geographic coordinate is assigned to each path text. For example, "the intersection of Road A and Road B", "the stop sign of Bus No. C", etc., are collectively referred to as path texts, and a geographic coordinate is assigned to each path text. For example, according to the prior map construction process, it is determined that the picture taken of the "intersection of Road A and Road B" is located near the position (X1, Y1), then the path text is assigned the geographic coordinates of (X1, Y1). From the above description, it can be seen that these steps are completed in the prior map creation stage.

[0078] The advantage of using the CLIP model for positioning text enhancement is that pure visual camera positioning is susceptible to fluctuations due to dynamic changes in the environment. For example, when a certain intersection is full of cars or empty at a certain moment, its visual description may change dramatically. However, by adding text descriptions, the positioning is anchored to certain static objects. For example, the text "the intersection of Road A and Road B" focuses on the name and topological structure of the road, which makes the camera positioning more robust.

[0079] The present invention proposes a patrol vehicle positioning method that does not require changes to the road surface or the installation of special equipment, thereby eliminating the need to produce and install a large number of Bluetooth beacons, embedded metal parking signs, RFID electronic tags and other equipment. This not only greatly reduces the initial investment cost, but also avoids the high cost of modifying existing road surfaces. Compared with multi-sensor positioning methods, this method does not require a high-precision inertial measurement unit, a high-precision lidar, or continuous leasing of high-precision GPS positioning services, thereby significantly reducing equipment costs. By reducing the need for fixed equipment, patrol service providers can deploy and manage patrol vehicles more flexibly, further saving maintenance and operating costs.

[0080] The inspection vehicle positioning method of the present invention only requires a camera and high-precision GPS signal service in the short term, and only requires a camera in the long term to achieve positioning, which greatly simplifies the configuration requirements of the sensor. This method not only reduces the complexity of the initial installation, but also provides higher flexibility, facilitating future technology upgrades and system expansion. Inspection service providers can quickly adjust equipment configurations as needed to improve operational flexibility and adaptability.

[0081] The present invention simplifies the operation steps and technical threshold by reducing the requirements for sensor external parameters and time alignment. In this way, even operators without professional technical backgrounds can easily use and maintain the inspection vehicle, further reducing labor costs and training costs. This simplified operation design allows the inspection vehicle to be put into use more quickly, improving overall efficiency and economic benefits.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A patrol vehicle positioning system based on NRTK and visual information, characterized in that: It includes a priori mapping subsystem and a visual SLAM positioning subsystem. The priori mapping subsystem runs first, and after obtaining the point cloud map, the visual SLAM positioning subsystem runs again. Before the visual SLAM positioning subsystem runs, all the point cloud map information will be loaded at one time; The input data of the prior mapping subsystem is the image and the NRTK position information corresponding to the image. NRTK provides accurate global longitude and latitude coordinates, which can be converted into spatial three-dimensional coordinates through the Earth-centered Earth-fixed coordinate system. The image and its corresponding NRTK position information can be used together to perform constrained motion structure recovery, thereby obtaining the image's position, posture and spatial point cloud map. At this point, the prior mapping subsystem completes its work; The visual SLAM positioning subsystem uses the visual SLAM technology to construct its own continuous position and attitude estimation based on the image information collected in real time, and at the same time determines the position of the current map based on the point cloud map, and provides it to the visual SLAM for common constrained position and attitude estimation. When the visual SLAM subsystem is in operation, it will also periodically transmit images to the cloud server; The CLIP model in the cloud server calculates the CLIP spatial representation vector of the image and compares it with the text description of key locations on the path passed to provide a rough geographic location observation.

2. The inspection vehicle positioning system based on NRTK and visual information as claimed in claim 1, characterized in that: The construction of the point cloud map requires the use of motion structure recovery technology, which is regarded as an optimization task to minimize the reprojection error of all images. The optimization goal is to obtain N spatial points, assuming that in A spatial point p i 2D image points mapped to the camera at the initial pose The mapping relationship is: in is a matrix determined by the physical properties of the camera. i Mapped to a pose Location is The camera's viewing angle Then the mapping relationship is: in Yes i The inverse of the z-axis coordinate after the spatial change. When a spatial point is observed by M cameras, the posture of the M cameras is Location The parameters will form a mutual constraint relationship, and finally only one optimization goal needs to be constructed to complete nonlinear optimization: in Yes i Mapped to the actual coordinates of the jth camera. In addition, since monocular vision cannot restore depth, it is generally not optimized. The exact value of the pivot frame is not used, but rather as a ratio: is regarded as 1, and the depth of other frames is its multiple, because the map containing N map points is to be restored, so the final optimization goal is: Instead of starting with random parameters, we first use feature point tracking, epipolar geometry, triangulation, etc. to obtain rough initial values, and then start optimizing from the initial values. It is necessary to add additional absolute position information. Assume that the precise position information of L cameras has been obtained through NRTK signals or actual measurements. At this time, the optimization objective can be modified to: The final optimization result Construct a sparse point cloud map, It can also provide positioning reference for some situations when the map is used later.

3. The inspection vehicle positioning system based on NRTK and visual information as claimed in claim 1, characterized in that: The location information obtained by the NRTK is expressed in longitude, latitude and altitude. The mapping method is carried out in the Cartesian coordinate system. Therefore, it is necessary to convert the longitude and latitude of the earth and the altitude coordinates into Cartesian coordinates. The selected Cartesian system is the geocentric earth-fixed coordinate system. In the geocentric earth-fixed coordinate system, the center of mass of the earth is located at the origin, the Z axis points to the North Pole direction of the earth's rotation axis, the X axis is located in the equatorial plane, pointing to the prime meridian, and the Y axis is perpendicular to the XZ plane, forming a right-handed coordinate system. In this coordinate system, the rotation of the earth causes the positions of all ground points to be fixed in three-dimensional space, which is called geo-fixed, that is, the coordinates of any fixed point on the earth in this coordinate system will not change with time; In the Earth-centered Earth-fixed coordinate system, the position of any point can be represented by three coordinate values: X, Y, and Z. Specifically, the latitude φ, longitude λ, and altitude h of a point on the Earth's surface or near it can be converted into the Earth-centered Earth-fixed coordinate system through a specific conversion formula to calculate the radius N of the circle where the point is located. The formula is: Among them, a is the semi-major axis of the earth, and e is the first eccentricity of the earth, which is calculated as: b is the Earth's semi-minor axis, and the geographic coordinates are converted to the Earth-centered Earth-fixed coordinate system using the following formula: Each t j can be expressed as [X, Y, Z] T .

4. The inspection vehicle positioning system based on NRTK and visual information as claimed in claim 1, characterized in that: In the visual SLAM positioning subsystem, features that can describe the global image are added. The global features of the image can be obtained by aggregating local features. One image obtains N descriptors with a dimension of D. We can first cluster these N descriptors to obtain K cluster centers. The overall characteristics of the image are Then V is calculated as: Where V(k) represents the kth vector in the overall feature, α k (x i ) is an indicator function, if x i The cluster center is c k , then α k (x i )=t, otherwise α k (x i )=0. V is an aggregated local descriptor vector, which uses the vector learned from the neural network to replace the cluster center obtained by clustering In addition, due to α k (x i ) is a non-differentiable function and cannot be parameterized, so it needs to be modified as follows: where w k , b k To learn the parameters, this modification actually replaces the original indicator function with a continuous probability function, so the new VLAD calculation method is: After having both global features and local features, the closest image is first retrieved from the map based on the global features, and then the local features are used for precise positioning.

5. The inspection vehicle positioning system based on NRTK and visual information as claimed in claim 1, characterized in that: The CLIP model is mainly used for language enhancement of map positioning: collecting pictures while the inspection vehicle is driving, and transmitting the pictures to the cloud server to calculate its feature vector using the CLIP model; At the same time, the CLIP model is used to calculate the feature vector of each text in the path text database, perform correlation calculation, and determine whether the current inspection vehicle has traveled to a certain landmark location based on the correlation score; if so, coordinate information near the location is provided to assist positioning, otherwise pure visual positioning is performed normally.

6. The inspection vehicle positioning system based on NRTK and visual information as claimed in claim 5, characterized in that: The path text database is a text description database of the inspection route of the inspection vehicle, which contains a number of natural languages ​​specified by humans to represent key locations on the route, collectively referred to as path texts, and a geographic coordinate is assigned to each path text.

Citation Information

Patent Citations

  • Mapping method and system based on GPS, IMU and binocular vision

    CN109991636A

  • Automatic map recovery method based on monocular ORB-SLAM

    CN111141295A

  • Method, apparatus and system for generating and exposing advertisement content for three-dimensional visual effect

    KR102512259B1

  • Map construction method, positioning method, apparatus, device and storage medium

    WO2024104225A1