Crowdsourcing surveying and mapping architecture and method based on cloud side end

Abstract

The invention discloses a crowdsourcing surveying and mapping architecture and method based on a cloud side end, and relates to the technical field of surveying and mapping architectures. The task allocation and collaborative management module is used for performing reasonable allocation according to task calculation and response requirements, low-delay and high-real-time tasks are processed in the edge calculation unit, and high-complexity and high-calculation-amount tasks are processed in the cloud calculation unit. The data hierarchical processing module sends data to the cloud computing unit and the edge computing unit, the cloud computing unit processes storage, complex analysis and long-period tasks of a large amount of data, the edge computing unit processes remaining tasks, and the decision and optimization module is used for conducting primary processing and analysis on the data collected in the edge computing unit. A preliminary decision is made through intelligent algorithms such as machine learning, and a decision result and data are uploaded to the cloud computing module, so that the actual working efficiency can be improved, and the purpose of complicated and high-precision task processing can be met.

Description

Technical Field

[0001] The present invention relates to the technical field of surveying and mapping architectures, and particularly to a crowdsourcing surveying and mapping architecture and method based on cloud-edge-terminal. Background Art

[0002] With the continuous expansion of the application fields of surveying and mapping geographic information, its service fields cover many fields such as urban planning, environmental monitoring, and resource management, which has led to an explosive growth in the demand for various types of geographic information data products. At present, the traditional method for collecting geographic information data mainly relies on the centralized operation of professional technicians and equipment in a specific physical environment. This mode not only requires a large investment in a large number of technicians and professional equipment, but also has a cumbersome management process and low efficiency, and it is difficult to quickly respond to the demand for large-scale and large-volume data products. With the continuous development of technologies such as the Internet of Things and the continuous increase of data, the cloud-based Internet of Things solutions are gradually unable to meet the growing needs of people. More and more enterprises have begun to turn their attention to edge computing and use it as an extension of the cloud to accelerate the rate of data analysis and facilitate enterprises to make decisions faster and better.

[0003] Crowdsourcing surveying and mapping mainly focuses on specific scenarios such as the collection and update of electronic map data, and there are limitations in its application. At the same time, restricted by the bottlenecks of quality control and data confidentiality, currently it mainly processes single and low-precision tasks, and it is difficult to process complex and high-precision data tasks. Therefore, we propose a crowdsourcing surveying and mapping architecture and method based on cloud-edge-terminal. Summary of the Invention

[0004] The purpose of the present invention is to provide a crowdsourcing surveying and mapping architecture and method based on cloud-edge-terminal to solve the problems mentioned in the above background art.

[0005] The present invention specifically adopts the following technical solutions to achieve the above purpose: A crowdsourcing surveying and mapping architecture based on cloud-edge-terminal, comprising: A cloud computing unit, which is responsible for data analysis and storage; An edge computing unit, which is responsible for the preliminary filtering, analysis, and storage of data; A coordination unit, which is used to uniformly manage the edge computing unit and schedule the processing tasks, collect the data of the edge computing unit, and send the data collected by the edge computing unit to the cloud computing unit; A terminal device, which is used for data download, collection, and execution of processing tasks.

[0006] Further, the cloud computing unit includes: a data synchronization module, a data storage module, and a learning and training module. Among them, the data synchronization module synchronizes the data information sent via the cooperation unit, the data storage module is used to store the data of the data synchronization module, the learning and training module is used to perform overall optimization and long-term learning, and the learning and training module is based on the stochastic gradient descent method, and its update rule is carried out according to the following formula: (1) In the formula, m is the size of the mini-batch, x i is the sample in the mini-batch, and η is the learning rate.

[0007] Further, the edge computing unit includes: a command issuing module, a data collection module, a data filtering module, a data cleaning module, and a real-time monitoring module. Among them, the command issuing module is used to send commands to the terminal device, the data collection module is used to collect the data information collected by the terminal device, the data filtering module is used to filter the data information collected by the data collection module, the data cleaning module is used to clean the data filtered by the data filtering module, and the real-time monitoring module is used to monitor the working status of the data collection module in real time.

[0008] Further, the cooperation unit includes: a task allocation and cooperation management module, a data hierarchical processing module, a decision-making and optimization module, and a security encryption module. The task allocation and cooperation management module is used to make a reasonable allocation according to the task calculation and response requirements. Among them, low-latency and high-real-time tasks are processed within the edge computing unit, while high-complexity and high-computation tasks are processed within the cloud computing unit. The data hierarchical processing module sends data to the cloud computing unit and the edge computing unit respectively. Among them, the cloud computing unit processes the storage of a large amount of data, complex analysis, and long-cycle tasks, and the edge computing unit processes the remaining tasks. The decision-making and optimization module is used to perform preliminary processing and analysis on the data collected within the edge computing unit, make a preliminary decision through machine learning, decision trees, neural networks, and support vector machine intelligent algorithms, and upload the decision result and data to the cloud computing module. The security encryption module is used to establish a secure data transmission channel between the edge computing unit and the cloud computing unit. The security encryption module is based on the symmetric encryption algorithm, and it is carried out according to the following formula: (2) (3) In the formula, E(x) is the encrypted letter, x is the ASCII code of the plaintext letter, and n is the offset.

[0009] Further, the terminal device is built with a data acquisition module, a command execution module, a data transmission module, and a data download module. Among them, the data acquisition module is used for acquiring surveying and mapping data, the command execution module is used for executing the commands issued by the cloud computing unit and the edge computing unit, the data transmission module is used for sending the data collected by the data acquisition module to the cloud computing unit and the edge computing unit, and the data download module is used for downloading the data that has been collected.

[0010] A crowdsourcing surveying and mapping method based on cloud-edge-terminal adopts the above-mentioned cloud-edge-terminal crowdsourcing surveying and mapping architecture, and includes the following steps: Step 1, data collection, where data is collected via the data acquisition module; Step 2, data transmission, where the data collected by the data acquisition module is respectively transmitted to the cloud computing unit and the edge computing unit, the data synchronization module performs data synchronization, and the data collection module performs data collection; Step 3, data cleaning, where the data collected by the data collection module is filtered, and the filtered useless information is cleaned up, and the working state of the data collection module is monitored in real time through the real-time monitoring module; Step 4, cloud-edge collaboration, where the data collected by the cloud computing unit and the edge computing unit is collected through the collaboration unit, the data is encrypted during transmission through the security encryption module, and is preliminarily processed and analyzed through the decision-making and optimization module, and tasks are allocated through the task allocation and collaboration management module. Commands are issued to the cloud computing unit and the edge computing unit through the data hierarchical processing module, and then the commands are executed through the command execution module.

[0011] Further, the cloud-edge collaboration includes the following steps: Step 41, establish a message transmission mechanism for the cloud computing unit and the edge computing unit, deploy message transmission agents in the cloud computing unit and the edge computing unit respectively, and realize the message transmission between the cloud computing unit and the edge computing unit through message queues, MQTT protocols, DDS protocols or edge collaboration transmission, ensuring that the cloud computing unit can timely obtain the status information and data streams of the terminal devices in the case of a large number of terminal devices and scattered geographical locations; Step 42, data stream distribution, deploy a data distribution agent on the terminal device, and distribute the data to the cloud computing unit and the edge computing unit according to the characteristics of the data stream. Among them, the cloud computing unit processes the storage of a large amount of data, complex analysis, and long-cycle tasks, and the edge computing unit processes the remaining tasks; Step 43: Terminal autonomy. Deploy an autonomous agent on the terminal device so that it can independently process data streams and tasks. When the edge device is offline or unable to connect to the cloud computing unit and the edge computing unit, the autonomous agent can ensure that the terminal device continues to run and complete data analysis and task processing; Step 44: Design the architecture. Design the architecture of the entire cloud-edge collaboration system, including the cloud server, edge device, and edge server. The cloud server is responsible for storing and processing data, the edge device is responsible for collecting data and sending it to the cloud server, and the edge server is responsible for processing the data sent by the edge device; Step 45: Set up the cloud server. Build a server application on the cloud server to receive and process the data sent by the edge device. Use frameworks such as Flask, Django, FastAPI, or Tornado to build the server and define interfaces to receive data; Step 46: Set up the edge device. Write code to collect data and send it to the cloud server; Step 47: Establish a cloud-edge collaboration data transmission method. Determine the target cloud server that the signaling data to be transmitted needs to reach, perform link planning between the edge server and the target cloud server to obtain multiple communication links, and select a communication link that meets the network quality optimization goal to transmit the signaling data Step 48: Image recognition and processing. Receive the image feature information sent by the terminal device, determine whether there is a matching reference feature information in the local feature library. If it exists, generate a judgment result and send it to the terminal device. If it does not exist, encrypt the image information and send it to the cloud computing unit and the edge computing unit.

[0012] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned crowdsourcing mapping method based on the cloud-edge-terminal.

[0013] A computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is made to execute the above-mentioned crowdsourcing mapping method based on the cloud-edge-terminal.

[0014] The beneficial effects of the present invention are as follows: The task allocation and collaborative management module of the present invention is used for reasonable allocation according to task calculation and response requirements. Among them, low-latency and high-real-time tasks are processed within the edge computing unit, while high-complexity and high-computation tasks are processed within the cloud computing unit. The data hierarchical processing module sends data to the cloud computing unit and the edge computing unit respectively. Among them, the cloud computing unit processes the storage of a large amount of data, complex analysis, and long-cycle tasks, and the edge computing unit processes the remaining tasks. The decision-making and optimization module is used for preliminary processing and analysis of the data collected within the edge computing unit, making preliminary decisions through intelligent algorithms such as machine learning, and uploading the decision results and data to the cloud computing module. The security encryption module is used to establish a secure data transmission channel between the edge computing unit and the cloud computing unit, which can improve the actual work efficiency and meet the purpose of processing complex and high-precision tasks.

[0015] The present invention filters the data collected by the data collection module and clears the useless information after filtering. The real-time monitoring module monitors the working state of the data collection module in real time, which can shorten the workload and improve the work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the working block diagram of the present invention.

[0017] Figure 2 is the working flow chart of the present invention.

[0018] Figure 3 is the working flow chart of cloud-edge collaboration in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0020] Please refer to Figure 1 - Figure 3 , the present invention provides a crowdsourcing mapping architecture based on the cloud-edge-end, including: A cloud computing unit, which is responsible for data analysis and storage; the cloud computing unit includes: a data synchronization module, a data storage module, and a learning and training module. Among them, the data synchronization module synchronizes the data information sent by the collaboration unit, the data storage module is used to store the data of the data synchronization module, the learning and training module is used for overall optimization and long-term learning, and the learning and training module is based on the stochastic gradient descent method, and its update rule is carried out according to the following formula: (1) In the formula, m is the size of the small batch, x i is the sample in the small batch, and η is the learning rate.

[0021] An edge computing unit, which is responsible for the preliminary filtering, analysis, and storage of data; the edge computing unit includes: a command publishing module, a data collection module, a data filtering module, a data cleaning module, and a real-time monitoring module. Among them, the command publishing module is used to send commands to terminal devices, the data collection module is used to collect data information collected by terminal devices, the data filtering module is used to filter the data information collected by the data collection module, the data cleaning module is used to clean the data filtered by the data filtering module, and the real-time monitoring module is used to monitor the working status of the data collection module in real time.

[0022] A coordination unit, which is used to uniformly manage the edge computing unit, schedule processing tasks, and collect data of the edge computing unit, and is used to send the data collected by the edge computing unit to the cloud computing unit; the coordination unit includes: a task allocation and coordination management module, a data hierarchical processing module, a decision-making and optimization module, and a security encryption module. The task allocation and coordination management module is used to make reasonable allocations according to task calculation and response requirements. Among them, low-latency and high-real-time tasks are processed within the edge computing unit, while high-complexity and high-computation tasks are processed within the cloud computing unit. The data hierarchical processing module sends data to the cloud computing unit and the edge computing unit respectively. Among them, the cloud computing unit processes the storage of a large amount of data, complex analysis, and long-cycle tasks, and the edge computing unit processes the remaining tasks. The decision-making and optimization module is used to preliminarily process and analyze the data collected within the edge computing unit, make a preliminary decision through intelligent algorithms such as machine learning, decision trees, neural networks, and support vector machines, and upload the decision results and data to the cloud computing module. The security encryption module is used to establish a secure data transmission channel between the edge computing unit and the cloud computing unit. The security encryption module is based on a symmetric encryption algorithm, and it is carried out according to the following formula: (2) (3) In the formula, E(x) is the encrypted letter, x is the ASCII code of the plaintext letter, and n is the offset.

[0023] Terminal devices, which are used to download and collect data and execute processing tasks; the terminal devices are built-in with a data acquisition module, a command execution module, a data transmission module, and a data download module. Among them, the data acquisition module is used to acquire surveying and mapping data, the command execution module is used to execute commands issued by the cloud computing unit and the edge computing unit, the data transmission module is used to send the data collected by the data acquisition module to the cloud computing unit and the edge computing unit, and the data download module is used to download the completed collected data.

[0024] A crowdsourcing mapping method based on cloud-edge-terminal, adopting the cloud-edge-terminal-based crowdsourcing mapping architecture as described above, includes the following steps: Step 1, data collection, where data is collected via the data acquisition module; it can ensure the real-time and accuracy of data collection.

[0025] Step 2, data transmission, where the data collected by the data acquisition module is respectively transmitted to the cloud computing unit and the edge computing unit, data synchronization is performed by the data synchronization module, and data collection is performed by the data collection module; it can ensure the stability and real-time of data transmission.

[0026] Step 3, data cleaning, where the data collected by the data collection module is filtered, and the filtered useless information is cleared, and the working state of the data collection module is monitored in real time through the real-time monitoring module; it can shorten the workload and improve work efficiency.

[0027] Step 4, cloud-edge collaboration, where the data collected by the cloud computing unit and the edge computing unit is collected through the collaboration unit, encrypted during data transmission through the security encryption module, preliminarily processed and analyzed through the decision-making and optimization module, tasks are assigned through the task assignment and collaboration management module, commands are issued to the cloud computing unit and the edge computing unit through the data hierarchical processing module, and then the commands are executed through the command execution module, including the following steps: Step 41, establish a message transmission mechanism for the cloud computing unit and the edge computing unit, deploy message transmission agents in the cloud computing unit and the edge computing unit respectively, and realize the message transmission between the cloud computing unit and the edge computing unit through message queues, MQTT protocols, DDS protocols or edge collaboration transmissions, ensuring that the cloud computing unit can timely obtain the status information and data streams of terminal devices in the case of a large number of terminal devices and scattered geographical locations; Step 42, data stream distribution, deploy data distribution agents on terminal devices, and distribute data to the cloud computing unit and the edge computing unit according to the characteristics of the data stream. Among them, the cloud computing unit processes the storage of a large amount of data, complex analysis and long-cycle tasks, and the edge computing unit processes the remaining tasks; Step 43, terminal autonomy, deploy autonomous agents on terminal devices so that they can independently process data streams and tasks. When edge devices are offline or unable to connect to the cloud computing unit and the edge computing unit, the autonomous agents can ensure that terminal devices continue to run and complete data analysis and task processing; Step 44, design the architecture, design the architecture of the entire cloud-edge collaboration system, including cloud servers, edge devices and edge servers. The cloud server is responsible for storing and processing data, the edge device is responsible for collecting data and sending it to the cloud server, and the edge server is responsible for processing the data sent by the edge device; Step 45: Set up a cloud server, build a server application on the cloud server to receive and process data sent by edge devices. Use frameworks such as Flask, Django, FastAPI, or Tornado to build the server and define interfaces to receive data. Step 46: Set up edge devices, write code to collect data and send it to the cloud server. Step 47: Establish a cloud-edge collaborative data transmission method. Determine the target cloud server that the signaling data to be transmitted needs to reach, perform link planning between the edge server and the target cloud server to obtain multiple communication links, and select a communication link that meets the network quality optimization goal to transmit the signaling data. Step 48: Image recognition and processing. Receive the image feature information sent by the terminal device, determine whether there is a matching reference feature information in the local feature library. If it exists, generate a judgment result and send it to the terminal device. If it does not exist, encrypt the image information and send it to the cloud computing unit and the edge computing unit.

[0028] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned crowdsourcing mapping method based on the cloud-edge-terminal.

[0029] A computer storage medium stores instructions. When the instructions are executed on a computer, the computer is made to execute the above-mentioned crowdsourcing mapping method based on the cloud-edge-terminal.

[0030] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A crowdsourcing mapping architecture based on cloud-edge-terminal, characterized in that, Including: A cloud computing unit responsible for data analysis and storage; An edge computing unit responsible for preliminary filtering, analysis, and storage of data; A coordination unit used to uniformly manage the edge computing unit and schedule processing tasks, collect data of the edge computing unit, and send the data collected by the edge computing unit to the cloud computing unit; A terminal device used to download and collect data and execute processing tasks.

2. The crowdsourcing mapping architecture based on cloud-edge-terminal according to claim 1, characterized in that, The cloud computing unit includes: a data synchronization module, a data storage module, and a learning and training module. Among them, the data synchronization module synchronizes the data information sent via the coordination unit. The data storage module is used to store the data of the data synchronization module. The learning and training module is used for overall optimization and long-term learning. And the learning and training module is based on the stochastic gradient descent method, and its update rule is carried out according to the following formula: （1） In the formula, the m is the size of the mini-batch, x i are the samples in the mini-batch, η is the learning rate.

3. The crowdsourcing mapping architecture based on cloud-edge-terminal according to claim 1, characterized in that, The edge computing unit includes: a command publishing module, a data collection module, a data filtering module, a data cleaning module, and a real-time monitoring module. Among them, the command publishing module is used to send commands to the terminal device. The data collection module is used to collect the data information collected by the terminal device. The data filtering module is used to filter the data information collected by the data collection module. The data cleaning module is used to clean the data filtered by the data filtering module. The real-time monitoring module is used to monitor the working status of the data collection module in real time.

4. The crowdsourcing mapping architecture based on cloud-edge-terminal according to claim 1, characterized in that, The coordination unit includes: a task allocation and coordination management module, a data hierarchical processing module, a decision-making and optimization module, and a security encryption module. The task allocation and coordination management module is used for reasonable allocation according to task calculation and response requirements. Among them, low-latency and high-real-time tasks are processed within the edge computing unit, while high-complexity and high-computation tasks are processed within the cloud computing unit. The data hierarchical processing module sends data to the cloud computing unit and the edge computing unit respectively. Among them, the cloud computing unit processes the storage of a large amount of data, complex analysis, and long-cycle tasks, and the edge computing unit processes the remaining tasks. The decision-making and optimization module is used to perform preliminary processing and analysis on the data collected within the edge computing unit, make a preliminary decision through intelligent algorithms such as machine learning, decision trees, neural networks, and support vector machines, and upload the decision result and data to the cloud computing module. The security encryption module is used to establish a secure data transmission channel between the edge computing unit and the cloud computing unit. The security encryption module is based on the symmetric encryption algorithm, and it is carried out according to the following formula: （2） （3） In the formula, E(x) is the encrypted letter, x is the ASCII code of the plaintext letter, and n is the offset.

5. The crowdsourcing mapping architecture based on cloud-edge-terminal according to claim 1, wherein, The terminal device is built-in with a data acquisition module, a command execution module, a data transmission module, and a data download module. Among them, the data acquisition module is used to collect surveying and mapping data, the command execution module is used to execute the commands issued by the cloud computing unit and the edge computing unit, the data transmission module is used to send the data collected by the data acquisition module to the cloud computing unit and the edge computing unit, and the data download module is used to download the completed collected data.

6. A crowdsourcing mapping method based on cloud-edge-device, which adopts the cloud-edge-device-based crowdsourcing mapping architecture as described in any one of claims 1-5, characterized in that, It includes the following steps: Step 1, data collection, data is collected via the data acquisition module; Step 2, data transmission, the data collected by the data acquisition module is respectively transmitted to the cloud computing unit and the edge computing unit, data synchronization is performed by the data synchronization module, and data collection is performed by the data collection module; Step 3, data cleaning, the data collected by the data collection module is filtered, and the filtered useless information is cleaned up. The working status of the data collection module is monitored in real time through the real-time monitoring module; Step 4, cloud-edge collaboration, the data collected by the cloud computing unit and the edge computing unit is collected through the collaboration unit, encrypted during data transmission through the security encryption module, preliminarily processed and analyzed through the decision-making and optimization module, tasks are allocated through the task allocation and collaboration management module, commands are issued to the cloud computing unit and the edge computing unit through the data hierarchical processing module, and then the commands are executed through the command execution module.

7. A crowdsourcing mapping method based on cloud-edge-terminal according to claim 6, characterized in that: The cloud-edge collaboration includes the following steps: Step 41, establish a message transmission mechanism for the cloud computing unit and the edge computing unit, deploy message transmission agents in the cloud computing unit and the edge computing unit respectively, and realize the message transmission between the cloud computing unit and the edge computing unit through message queues, MQTT protocols, DDS protocols, or edge collaboration transmission, ensuring that the cloud computing unit can timely obtain the status information and data streams of the terminal devices in the case of a large number of terminal devices and scattered geographical locations; Step 42, data stream distribution, deploy a data distribution agent on the terminal device, and distribute the data to the cloud computing unit and the edge computing unit according to the characteristics of the data stream. Among them, the cloud computing unit processes the storage of a large amount of data, complex analysis, and long-cycle tasks, and the edge computing unit processes the remaining tasks; Step 43, terminal autonomy, deploy an autonomous agent on the terminal device so that it can independently process data streams and tasks. When the edge device is offline or unable to connect to the cloud computing unit and the edge computing unit, the autonomous agent can ensure that the terminal device continues to run and complete data analysis and task processing; Step 44, design the architecture, design the architecture of the entire cloud-edge collaboration system, including the cloud server, edge devices, and edge servers. The cloud server is responsible for storing and processing data, the edge devices are responsible for collecting data and sending it to the cloud server, and the edge servers are responsible for processing the data sent by the edge devices. Step 45: Set up a cloud server, build a server application on the cloud server to receive and process data sent by edge devices, build the server using frameworks such as Flask, Django, FastAPI or Tornado, and define interfaces to receive data; Step 46: Set up edge devices, write code to collect data and send it to the cloud server; Step 47: Establish a data transmission method for cloud-edge collaboration, determine the target cloud server that the signaling data to be transmitted needs to reach, perform link planning between the edge server and the target cloud server to obtain multiple communication links, and select a communication link that meets the network quality optimization goal to transmit the signaling data Step 48: Image recognition and processing, receive the image feature information sent by the terminal device, determine whether there is a matching reference feature information in the local feature library. If it exists, generate a judgment result and send it to the terminal device. If it does not exist, encrypt the image information and send it to the cloud computing unit and the edge computing unit.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the above-mentioned crowdsourcing mapping method based on cloud-edge-terminal is implemented.

9. A computer storage medium, characterized in that, Instructions are stored in the computer storage medium. When the instructions are executed on the computer, the computer is made to execute the above-mentioned crowdsourcing mapping method based on cloud-edge-terminal.

Images