Off-line data generation method for road field digital acquisition

Abstract

The invention discloses an off-line data generation method for road field digital acquisition. The method comprises the following steps: S1, monitoring an off-line data generation request of a user by using a message queue; s2, starting a data processing flow; s3, performing format conversion and preprocessing on the obtained data resource file; s4, executing topology check and repair, namely detecting polygon overlapping and boundary missing topology errors, and repairing the topology errors to enable the topological relation of the GIS data to be correct; s5, selecting an adaptive output form according to the data scale; s6, using a curl command or other API request modes to configure map publishing parameters, completing map publishing, and obtaining an online preview link; s7, determining the downloading range of the map tile, and downloading the vector tile data to the local; and S8, packaging locally downloaded vector tile data into an MBTiles format for a Web end and a mobile application to use. According to the method and the device, the problems of low data processing efficiency, non-uniform formats and insufficient topology repair capability in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data generation, and in particular to an offline data generation method for digital collection of highway field operations. Background Art

[0002] With the rapid development of highway transportation infrastructure, highway field survey plays a vital role in engineering construction, maintenance and operation management. In the traditional highway field survey process, investigators usually need to carry a large amount of paper drawings and materials for on-site review and record. This method not only increases the workload of field surveys, but also has problems such as inconvenience in carrying drawings and untimely information updates, which cannot meet the requirements of modern engineering management for efficiency and accuracy.

[0003] At present, with the development of digital technology, digital collection of highway field work has gradually become a trend. In the existing technology, the main data sources for field digital collection are design files in CAD format or map data in image format. The data contains rich geographic information, but due to the complexity of the format and the huge amount of data, there are great technical difficulties in directly loading it into mobile devices or portable terminals. In addition, the field survey environment usually faces unstable network or no network connection, which makes it impossible for investigators to rely on online systems to query data in real time. There is an urgent need for a lightweight data format that can be efficiently loaded and used in an offline environment.

[0004] In the existing technology, some optimization solutions for the digital collection of highway field have been applied. For example, some systems try to convert CAD data into GIS data to support geospatial analysis, or share data online through map publishing engines. However, these methods still have many defects: first, the data conversion process usually requires manual intervention and cannot be fully automated, resulting in low efficiency; second, the converted data format is diverse and not unified, and cannot be smoothly loaded and displayed in an offline environment; third, the existing map publishing technology is mostly designed for online use scenarios, and has poor support for offline use, and it is difficult to adapt to the needs of a network-free environment.

[0005] In summary, the existing technology has significant deficiencies in data format uniformity, offline data loading efficiency, and automated data processing capabilities, making it difficult to meet the requirements of modern highway field digital collection for efficiency and practicality. These technical defects not only affect the accuracy and efficiency of survey work, but also restrict the further promotion and application of field digital technology. A new solution is urgently needed to overcome the above problems. Summary of the invention

[0006] One object of the present invention is to propose an offline data generation method for highway field digital collection, which solves the problems of inefficient data processing, inconsistent formats and insufficient topology repair capabilities existing in the prior art.

[0007] According to an embodiment of the present invention, a method for generating offline data for highway field digital collection includes the following steps: S1. Use the message queue to listen to the user's offline data generation request, receive the offline data generation task initiated by the user through the Web terminal, queue multiple task requests, and respond in sequence according to the request order; S2. The user uploads local data to the object storage service, obtains the data resource file in the storage service to the server, and starts the data processing process; S3. Performing format conversion and preprocessing on the acquired data resource files; S4. Perform topology checking and repair, including detecting polygon overlap and boundary missing topology errors, and repairing topology errors to make the topological relationship of GIS data correct; S5. Choose the appropriate output format based on the data size: If the amount of GIS data is small, directly generate JSON format data; If the amount of GIS data is large, generate an online preview map through the map publishing engine; S6. Use the curl command or other API request methods to configure map publishing parameters, complete map publishing, and obtain an online preview link; S7. Determine the download range of the map tile and download the vector tile data to the local computer; S8. Package the locally downloaded vector tile data into MBTiles format for use by the Web client and mobile applications.

[0008] Optionally, S3 includes the following steps: S31. CAD data file Decompose into multiple standard layers ,in, , each standard layer Contains one type of feature, including annotation, points, polygons, multipatches, and lines; S32. Use data conversion tools to convert standard layers Import the feature data into the GIS system to generate the corresponding GIS feature set : ; in, Indicates data conversion operation, converting CAD layer data into GIS format; S33. Perform line feature conversion and feature merging processing on polyline features: For polyline features , apply the line feature conversion function Get line features; For feature collections with the same layer name , apply the feature merge operation , get the merged elements ;in, Merge geometry and attribute information of multiple features; S34. Determine the central meridian based on the project's geographic location , defining the projected coordinate system for all features , and convert the coordinate system to latitude and longitude when necessary : For feature collections , apply the projection transformation function The coordinates of the feature Convert to projected coordinates: ; in, Based on the central meridian Perform projection transformation on coordinates to locate the elements in the plane coordinate system; If you need to display data in latitude and longitude, apply the inverse projection conversion function Projection coordinates Convert to geographic coordinates: ; in, are longitude and latitude coordinates.

[0009] Optionally, S4 includes the following steps: S41. GIS feature collection Perform topological error detection to identify topological errors such as polygon overlap and missing boundaries. , calculate the spatial relationship matrix between the current polygon feature and other polygon features : ; in, , polygon features and polygon features For different polygonal elements, represents the area function; like , then the polygon feature With polygon features There is overlap, which needs to be fixed; For each polygon feature Calculate the degree of closure of its boundary : ; in, is the sum of the lengths of the connected edges, is the sum of the lengths of all sides of the polygon; like ,in is the preset threshold, then the polygon feature is determined There are missing boundaries that need to be repaired; S42. For overlapping polygonal features and polygon features , apply the topology correction function Perform a difference or merge operation: If you need to eliminate overlap, calculate the corrected polygon features: ; To merge into new polygon features: ; For polygon features with missing boundaries , apply the boundary repair algorithm Generate a complete polygon: ; in, is the first Strip edge, To be supplemented Vertices, and are the number of edges and vertices, respectively.

[0010] Optionally, S5 includes the following steps: S51. Compute GIS feature collections The total amount of data : ; in, GIS feature collection The total number of elements in For the The size of the geometric data of each feature is calculated as: ; in, is the geometric complexity factor, For the The number of vertices of a feature, The storage size for each vertex, For the Size of attribute data for each feature: ; in, is the attribute data overhead factor, For the The number of attributes per feature, The storage size for each attribute depends on the data type and length of the attribute field; S52. The total amount of data calculated With the preset threshold Compare and select the appropriate output format: when When directly generating JSON format data ; when When the map is published, the online preview map is generated by the map publishing engine : ; in, For map publishing functions, GIS feature collection Publish as an online map, Publish parameters for the map, including map style, tile level, and other configurations.

[0011] Optionally, the S6 comprises the following steps: S61. Configure map publishing parameter set , including map styles , Tile level , Projected Coordinate System : Map style parameters define the visual presentation of the map; The tile level parameter specifies the zoom level range of the map tile, expressed as {Z}_{level}=\left [ {{z}_{min},{z}_{max}} \right ] ,in and are the minimum and maximum zoom levels, respectively; is the projection coordinate system parameter, which is consistent with the projection coordinate system defined in step S34; S62. Use the curl command or other API request methods to send a map publishing request to the map publishing service , the request content includes a GIS feature collection and map publishing parameter set ; S63. Receive a response from the map publishing service and obtain an online preview link.

[0012] The beneficial effects of the present invention are: (1) The present invention establishes a fully automated offline data generation process by combining message queues, GIS data processing and inspection, map publishing, vector tile generation, and MBTiles data packaging technologies. In this process, a message queue mechanism is used to monitor and queue offline data generation requests in real time to avoid manual intervention and achieve efficient data processing around the clock. Through the automated GIS data inspection and processing module, feature classification, feature merging, topology inspection and repair steps are introduced in the conversion process between CAD and GIS data. In combination with dynamic projection definition and coordinate system conversion technology, it is ensured that the generated data meets the spatial consistency requirements. Compared with the existing processing flow that requires a large amount of manual participation, the present invention significantly improves efficiency and can reduce manual input by more than 70% when generating offline data in batches.

[0013] (2) The present invention solves the problem in the prior art that it is difficult to unify data in different formats. Through a systematic format conversion and optimization method, multiple data sources can be converted into JSON or vector tile format, and finally stored in a lightweight MBTiles file. Specifically, the present invention introduces a complexity factor and a dynamic threshold judgment model in the data volume evaluation process, combined with the data volume calculation formula, and dynamically selects an adaptive output format according to the data scale. When the GIS data volume is small, JSON format data is directly generated, and when the GIS data volume is large, vector tile generation technology is used. Finally, it is packaged and stored in MBTiles format, taking into account both the compactness and efficient loading capabilities of the data. Compared with the traditional method, the offline loading speed is increased by 30%-50%, and the storage space is reduced by about 40%.

[0014] (3) The present invention proposes innovative matrix operations and dynamic boundary repair algorithms in topology checking and repairing. It accurately detects polygon overlap and boundary missing problems by constructing a spatial relationship matrix. Combined with an adaptive repair function, it clips and merges overlapping areas to ensure the spatial integrity of the data. In the boundary repair process, a boundary closure formula is introduced to quickly generate complete polygons by dynamically supplementing missing edges and vertices. Traditional technologies mostly rely on manual or semi-automatic processing for topology repair. The present invention significantly improves the accuracy and efficiency of complex GIS data processing through a fully automated topology repair method. The processing time for high topological complexity data is reduced by more than 50%, and the accuracy of topology integrity checking is increased to 99.5%. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 The present invention provides a flowchart of an offline data generation method for highway field digital collection. DETAILED DESCRIPTION

[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0017] refer to Figure 1 , an offline data generation method for highway field digital collection, comprising the following steps: S1. Use the message queue to listen to the user's offline data generation request, receive the offline data generation task initiated by the user through the Web terminal, queue multiple task requests, and respond in sequence according to the request order; S2. The user uploads local data to the object storage service, obtains the data resource file in the storage service to the server, and starts the data processing process; S3. Performing format conversion and preprocessing on the acquired data resource files; S4. Perform topology checking and repair, including detecting polygon overlap and boundary missing topology errors, and repairing topology errors to make the topological relationship of GIS data correct; S5. Choose the appropriate output format based on the data size: If the amount of GIS data is small, directly generate JSON format data; If the amount of GIS data is large, generate an online preview map through the map publishing engine; S6. Use the curl command or other API request methods to configure map publishing parameters, complete map publishing, and obtain an online preview link; S7. Determine the download range of the map tile and download the vector tile data to the local computer; S8. Package the locally downloaded vector tile data into MBTiles format for use by the Web client and mobile applications.

[0018] In this implementation, S3 includes the following steps: S31. CAD data file Decompose into multiple standard layers ,in, , each standard layer Contains one type of feature, including annotation, points, polygons, multipatches, and lines; S32. Use data conversion tools to convert standard layers Import the feature data into the GIS system to generate the corresponding GIS feature set : ; in, Indicates data conversion operation, converting CAD layer data into GIS format; S33. Perform line feature conversion and feature merging processing on polyline features: For polyline features , apply the line feature conversion function Get line features; For feature collections with the same layer name , apply the feature merge operation , get the merged elements ;in, Merge geometry and attribute information of multiple features; S34. Determine the central meridian based on the project's geographic location , defining the projected coordinate system for all features , and convert the coordinate system to latitude and longitude when necessary : For feature collections , apply the projection transformation function The coordinates of the feature Convert to projected coordinates: ; in, Based on the central meridian Perform projection transformation on coordinates to locate the elements in the plane coordinate system; If you need to display data in latitude and longitude, apply the inverse projection conversion function Projection coordinates Convert to geographic coordinates: ; in, are longitude and latitude coordinates.

[0019] In this implementation, S4 includes the following steps: S41. GIS feature collection Perform topological error detection to identify topological errors such as polygon overlap and missing boundaries. , calculate the spatial relationship matrix between the current polygon feature and other polygon features : ; in, , polygon features and polygon features For different polygonal elements, represents the area function; like , then the polygon feature With polygon features There is overlap, which needs to be fixed; For each polygon feature Calculate the degree of closure of its boundary : ; in, is the sum of the lengths of the connected edges, is the sum of the lengths of all sides of the polygon; like ,in is the preset threshold, then the polygon feature is determined There are missing boundaries that need to be repaired; S42. For overlapping polygonal features and polygon features , apply the topology correction function Perform a difference or merge operation: If you need to eliminate overlap, calculate the corrected polygon features: ; To merge into new polygon features: ; For polygon features with missing boundaries , apply the boundary repair algorithm Generate a complete polygon: ; in, is the first Strip edge, To be supplemented Vertices, and are the number of edges and vertices, respectively.

[0020] In this implementation, S5 includes the following steps: S51. Compute GIS feature collections The total amount of data : ; in, GIS feature collection The total number of elements in For the The size of the geometric data of each feature is calculated as: ; in, is the geometric complexity factor, For the The number of vertices of a feature, The storage size for each vertex, For the Size of attribute data for each feature: ; in, is the attribute data overhead factor, For the The number of attributes per feature, The storage size for each attribute depends on the data type and length of the attribute field; S52. The total amount of data calculated With the preset threshold Compare and select the appropriate output format: when When directly generating JSON format data ; when When the map is published, the online preview map is generated by the map publishing engine : ; in, For map publishing functions, GIS feature collection Publish as an online map, Publish parameters for the map, including map style, tile level, and other configurations.

[0021] In this implementation, S6 includes the following steps: S61. Configure map publishing parameter set , including map styles , Tile level , Projected Coordinate System : Map style parameters define the visual presentation of the map; The tile level parameter specifies the zoom level range of the map tile, expressed as {Z}_{level}=\left [ {{z}_{min},{z}_{max}} \right ] ,in and are the minimum and maximum zoom levels, respectively; is the projection coordinate system parameter, which is consistent with the projection coordinate system defined in step S34; S62. Use the curl command or other API request methods to send a map publishing request to the map publishing service , the request content includes a GIS feature collection and map publishing parameter set ; S63. Receive a response from the map publishing service and obtain an online preview link.

[0022] The present invention establishes a fully automated offline data generation process by combining message queues, GIS data processing and inspection, map publishing, vector tile generation, and MBTiles data packaging technologies. In this process, a message queue mechanism is used to monitor and queue offline data generation requests in real time to avoid manual intervention and achieve efficient data processing around the clock. Through the automated GIS data inspection and processing module, feature classification, feature merging, topology inspection and repair steps are introduced in the conversion process between CAD and GIS data, and combined with dynamic projection definition and coordinate system conversion technology, it is ensured that the generated data meets the spatial consistency requirements. Compared with the existing processing flow that requires a large amount of manual participation, the present invention has significantly improved efficiency and can reduce manual input by more than 70% when generating offline data in batches.

[0023] The present invention solves the problem in the prior art that it is difficult to unify data in different formats. Through a systematic format conversion and optimization method, multiple data sources can be converted into JSON or vector tile format, and finally stored in a lightweight MBTiles file. Specifically, the present invention introduces a complexity factor and a dynamic threshold judgment model in the data volume evaluation process, combines the data volume calculation formula, and dynamically selects an adaptive output format according to the data scale. When the GIS data volume is small, JSON format data is directly generated, and when the GIS data volume is large, vector tile generation technology is used. Finally, it is packaged and stored in MBTiles format, taking into account both the compactness and efficient loading capability of the data. Compared with the traditional method, the offline loading speed is increased by 30%-50%, and the storage space is reduced by about 40%.

[0024] The present invention proposes innovative matrix operations and dynamic boundary repair algorithms in terms of topology checking and repairing. It accurately detects polygon overlap and boundary missing problems by constructing a spatial relationship matrix. Combined with an adaptive repair function, the overlapping areas are clipped and merged to ensure the spatial integrity of the data. The boundary closure formula is introduced in the boundary repair process, and complete polygons are quickly generated by dynamically supplementing missing edges and vertices. Traditional technologies mostly rely on manual or semi-automatic processing for topology repair. The present invention significantly improves the accuracy and efficiency of complex GIS data processing through a fully automated topology repair method, reduces the processing time of high topological complexity data by more than 50%, and increases the accuracy of topology integrity checking to 99.5%.

[0025] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for generating offline data for digital collection of highway field, characterized in that: The steps include: S1. Use the message queue to listen to the user's offline data generation request, receive the offline data generation task initiated by the user through the Web terminal, queue multiple task requests, and respond in sequence according to the request order; S2. The user uploads local data to the object storage service, obtains the data resource file in the storage service to the server, and starts the data processing process; S3. Performing format conversion and preprocessing on the acquired data resource files; S4. Perform topology checking and repair, including detecting polygon overlap and boundary missing topology errors, and repairing topology errors to make the topological relationship of GIS data correct; S5. Choose the appropriate output format based on the data size: If the amount of GIS data is small, directly generate JSON format data; If the amount of GIS data is large, an online preview map can be generated through the map publishing engine; S6. Use the curl command or other API request methods to configure map publishing parameters, complete map publishing, and obtain an online preview link; S7. Determine the download range of the map tile and download the vector tile data to the local computer; S8. Package the locally downloaded vector tile data into MBTiles format for use by the Web client and mobile applications.

2. The method for generating offline data for highway field digital collection according to claim 1, characterized in that: The S3 comprises the following steps: S31. CAD data file Decompose into multiple standard layers ,in, , each standard layer Contains one type of feature, including annotation, points, polygons, multipatches, and lines; S32. Use data conversion tools to convert standard layers Import the feature data into the GIS system to generate the corresponding GIS feature set : ； in, Indicates data conversion operation, converting CAD layer data into GIS format; S33. Perform line feature conversion and feature merging processing on polyline features: For polyline features , apply the line feature conversion function Get line features; For feature collections with the same layer name , apply the feature merge operation , get the merged elements ;in, Merge geometry and attribute information of multiple features; S34. Determine the central meridian based on the project's geographic location , defining the projected coordinate system for all features , and convert the coordinate system to latitude and longitude when necessary : For feature collections , apply the projection transformation function The coordinates of the feature Convert to projected coordinates: ； in, Based on the central meridian Perform projection transformation on coordinates to locate the elements in the plane coordinate system; If you need to display data in latitude and longitude, apply the inverse projection conversion function Projection coordinates Convert to geographic coordinates: ； in, are longitude and latitude coordinates.

3. The method for generating offline data for highway field digital collection according to claim 1, characterized in that: The S4 comprises the following steps: S41. GIS feature collection Perform topological error detection to identify topological errors such as polygon overlap and missing boundaries. , calculate the spatial relationship matrix between the current polygon feature and other polygon features : ； in, , polygon features and polygon features For different polygonal elements, represents the area function; like , then the polygon feature With polygon features There is overlap, which needs to be fixed; For each polygon feature Calculate the degree of closure of its boundary : ； in, is the sum of the lengths of the connected edges, is the sum of the lengths of all sides of the polygon; like ,in is the preset threshold, then the polygon feature is determined There are missing boundaries that need to be repaired; S42. For overlapping polygonal elements and polygon features , apply the topology correction function Perform a difference or merge operation: If you need to eliminate overlap, calculate the corrected polygon features: ； To merge into new polygon features: ； For polygon features with missing boundaries , apply the boundary repair algorithm Generate a complete polygon: ； in, is the first Strip edge, To be supplemented Vertices, and are the number of edges and vertices, respectively.

4. The method for generating offline data for highway field digital collection according to claim 1, characterized in that: The S5 comprises the following steps: S51. Compute GIS feature sets The total amount of data : ； in, GIS feature collection The total number of elements in For the The size of the geometric data of each feature is calculated as: ； in, is the geometric complexity factor, For the The number of vertices of a feature, The storage size for each vertex, For the Size of attribute data for each feature: ； in, is the attribute data overhead factor, For the The number of attributes per feature, The storage size for each attribute depends on the data type and length of the attribute field; S52. The total amount of data calculated With the preset threshold Compare and select the appropriate output format: when When directly generating JSON format data ; when When the map is published, the online preview map is generated by the map publishing engine : ； in, For map publishing functions, GIS feature collection Publish as an online map, Publish parameters for the map, including map style, tile level, and other configurations.

5. The method for generating offline data for highway field digital collection according to claim 1, characterized in that: The S6 comprises the following steps: S61. Configure map publishing parameter set , including map styles , Tile level , Projected Coordinate System : Map style parameters define the visual presentation of the map; The tile level parameter specifies the zoom level range of the map tile, expressed as ,in and are the minimum and maximum zoom levels, respectively; is the projection coordinate system parameter, which is consistent with the projection coordinate system defined in step S34; S62. Use the curl command or other API request methods to send a map publishing request to the map publishing service , the request content includes a GIS feature collection and map publishing parameter set ; S63. Receive a response from the map publishing service and obtain an online preview link.

Images