Data processing method and device, electronic equipment and storage medium
By performing clustering processing on high-precision map data, the problem of low display efficiency of high-precision map data is solved, enabling the rapid display of data related to access requests and improving display efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHIDAO NETWORK TECH (BEIJING) CO LTD
- Filing Date
- 2022-10-28
- Publication Date
- 2026-06-05
AI Technical Summary
High-precision map data is inefficient to access in databases and cannot flexibly meet diverse access needs, resulting in poor display efficiency.
After receiving the client's access request and verifying the user's permissions, the system performs clustering processing on the high-precision map data in the database and sends the clustered data to the client. The system uses a preset clustering algorithm, such as k-means, to cluster the data and determine the data type associated with the access request.
It improves the display efficiency of high-precision map data, solves the access difficulties caused by database field association, and enables the rapid display of relevant data.
Smart Images
Figure CN115733614B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of high-precision map data processing technology, and in particular to a data processing method, apparatus, electronic device, and storage medium. Background Technology
[0002] High-definition map web pages need to efficiently display different data based on query requests. When the data volume is very large, the data storage method will affect the display efficiency. The storage method of high-definition map data is basically the same as that of other map data, which is to store it in a server database.
[0003] Currently, accessing high-precision map data involves linking fields in a database. However, this method can lead to access problems when dealing with more flexible access needs, and querying data takes a lot of time, resulting in low efficiency in displaying high-precision map data. Summary of the Invention
[0004] This invention provides a data processing method, apparatus, electronic device, and storage medium to solve the problem of low display efficiency of high-precision map data. By clustering high-precision map data in the database through access requests, the data associated with the access requests is determined, enabling rapid display of data related to the access requests and improving the display efficiency of high-precision map data.
[0005] This invention provides a data processing method, comprising:
[0006] Receive access requests for high-precision map data sent by clients;
[0007] Verify user permissions based on the authentication token carried in the access request;
[0008] If the user permission verification is successful, the high-precision map data in the database is clustered based on the access request, and the clustered data is sent to the client; wherein, the client displays the received clustered data.
[0009] In one embodiment, clustering the high-precision map data in the database based on the access request includes:
[0010] Based on the access request, the access requirement is determined, and the access requirement is input into the first script;
[0011] The first script calls the second script, and the high-precision map data in the database is clustered based on the second script.
[0012] In one embodiment, clustering the high-precision map data in the database based on the second script includes:
[0013] Based on the second script, a preset clustering algorithm is used to cluster the high-precision map data in the database.
[0014] In one embodiment, sending the clustered data obtained from clustering to the client includes:
[0015] Determine the data encoding method;
[0016] The clustering data obtained from clustering is encoded using the aforementioned encoding method, and the encoded clustering data is sent to the client.
[0017] In one embodiment, verifying user permissions based on the authentication token carried in the access request includes:
[0018] Determine the comparison result between the authentication token carried in the access request and the target authentication token;
[0019] The user's permissions are verified based on the comparison results.
[0020] In one embodiment, prior to receiving the access request for high-precision map data sent by the client, the following steps are included:
[0021] Receive the authentication token request sent by the client;
[0022] The authentication token is generated based on the authentication token request, and the authentication token is sent to the client.
[0023] In one embodiment, prior to receiving the authentication token request sent by the client, the process includes:
[0024] Receive the authentication request sent by the client;
[0025] The encryption method for sending the authentication request to the client;
[0026] The system receives a username and password digest sent by the client, performs authentication on the user's permissions based on the username and password digest to obtain an authentication result, and sends the authentication result back to the client.
[0027] The present invention also provides a data processing apparatus, comprising:
[0028] The receiving module is used to receive access requests for high-precision map data sent by the client;
[0029] The verification module is used to verify user permissions based on the authentication token carried in the access request;
[0030] The clustering module is used to cluster high-precision map data in the database based on the access request if the user permission verification is successful, and send the clustered data to the client; wherein the client displays the received clustered data.
[0031] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the data processing methods described above.
[0032] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method as described above.
[0033] The data processing method, apparatus, electronic device, and storage medium provided by this invention receive access requests for high-precision map data sent by a client; verify user permissions based on an authentication token carried in the access request; if the user permission verification is successful, cluster the high-precision map data in the database based on the access request, and send the clustered data to the client; wherein the client displays the received clustered data. This invention clusters high-precision map data in the database using access requests to determine data associated with the access request, thereby enabling rapid display of data related to the access request and improving the display efficiency of high-precision map data. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0035] Figure 1 This is one of the flowcharts illustrating the data processing method provided by the present invention;
[0036] Figure 2 This is the second flowchart of the data processing method provided by the present invention;
[0037] Figure 3 This is a schematic diagram of the data processing device provided by the present invention;
[0038] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0040] The following is combined Figures 1-4 The present invention describes a data processing method, apparatus, electronic device, and storage medium.
[0041] Specifically, the present invention provides a data processing method, referring to... Figure 1 , Figure 1 This is one of the flowcharts of the data processing method provided by the present invention.
[0042] The data processing method provided in this embodiment of the invention includes:
[0043] Step 100: Receive the access request for high-precision map data sent by the client;
[0044] It should be noted that the execution subject of the data processing method provided in the embodiments of the present invention can be a server, a computer device, such as a mobile phone, tablet computer, laptop computer, handheld computer, vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.
[0045] It receives access requests for high-precision map data sent by clients. For example, when a user needs to query high-precision map data, they can send an access request to the server through a query webpage.
[0046] Step 200: Verify user permissions based on the authentication token carried in the access request;
[0047] It should be noted that, in order to improve the security of accessing high-precision map data, permissions have been set for each user. Therefore, upon receiving a request to access high-precision map data, it is necessary to verify whether the user has the necessary permissions.
[0048] The system retrieves the authentication token carried in the access request and compares it with the target authentication token stored on the server. If the comparison results are the same, the authentication token is valid, and the current user has the necessary access permissions. The authentication token is a temporary token generated by the server and expires automatically after one authentication or after a period of time.
[0049] Step 300: If the user permission verification is successful, the high-precision map data in the database is clustered based on the access request, and the clustered data is sent to the client; wherein, the client displays the received clustered data.
[0050] If the user's permissions are verified, the high-precision map data in the database is clustered based on the access request, and the clustered data is sent to the client, which then displays the received clustered data.
[0051] Specifically, the access requirements are determined based on the access request and input into a first script. The first script then calls a second script, and based on the second script, a pre-defined clustering algorithm is used to cluster the high-precision map data in the database. For example, assuming the first script is a Lua script and the second script is a Python script, after determining the access requirements, these requirements are input into the Lua script. The Lua script then calls the Python script based on the received access requirements. The Python script, after obtaining the access requirements, calls PySpark to perform parallel clustering operations on the high-precision map data in the database. Here, data clustering refers to classifying samples into different categories based on the similarity between samples. Different similarity calculation methods will yield different clustering results.
[0052] Clustering algorithms can use k-means, which is an iterative clustering analysis algorithm. Its main idea is: given a value of K and K initial cluster centroids, each point (i.e., data record) is assigned to the cluster represented by the nearest cluster centroid. After all points are assigned, the centroid of a cluster is recalculated based on all points in the cluster (taking the average value). Then, the steps of assigning points and updating cluster centroids are iteratively performed until the change in cluster centroids is very small or the specified number of iterations is reached.
[0053] The steps of k-means clustering include:
[0054] Step 1: Randomly select K points in the feature space as the initial cluster centers;
[0055] Step 2: For each other point, calculate the distance to the K centers. For unknown points, select the nearest cluster center as the label category.
[0056] Step 3: After identifying the marked cluster centers, recalculate the new center point (average value) for each cluster;
[0057] Step 4: If the calculated new center point is the same as the original center point (the centroid no longer moves), then the process ends; otherwise, repeat Step 2.
[0058] Alternatively, other clustering algorithms can be used, such as DBSCAN (density-based spatial clustering), spectral clustering, GMM (Gaussian mixture model), MeanShift (mean shift), and hierarchical clustering.
[0059] refer to Figure 2 , Figure 2 This is the second flowchart illustrating the data processing method provided by the present invention.
[0060] In this embodiment of the invention, the steps for clustering and displaying high-precision map data are as follows:
[0061] (1) The client sends an access request for high-precision map data carrying a token to the server.
[0062] (2) The server verifies the token carried in the access request. If the verification is successful, the access requirements in the access request are input into the Lua script.
[0063] (3) After receiving the access request, the Lua script calls the Python script according to the access request;
[0064] (4) The Python script obtains the access requirements and calls PySpark to perform parallel clustering operations on the high-precision map data in the database based on the access requirements. The clustering algorithm used is k-means.
[0065] (5) The generated clustering data is returned to the client and displayed in JSON format.
[0066] Based on the above steps, high-precision map data can be clustered to obtain different types of high-precision map data. Therefore, when accessing high-precision map data, the corresponding data can be accessed directly based on the data type obtained from the clustering results. This avoids the problem of being unable to access data because accessing high-precision map data involves associating fields in the database, thus improving data access efficiency.
[0067] The data processing method provided in this invention involves receiving an access request for high-precision map data sent by a client; verifying user permissions based on an authentication token carried in the access request; if the user permission verification is successful, clustering the high-precision map data in the database based on the access request, and sending the clustered data to the client; wherein the client displays the received clustered data. This invention clusters the high-precision map data in the database using access requests to determine the data associated with the access request, thereby enabling rapid display of data related to the access request and improving the display efficiency of high-precision map data.
[0068] Based on the above embodiments, sending the clustered data obtained from clustering to the client includes: determining the encoding method of the data; encoding the clustered data obtained from clustering using the encoding method; and sending the encoded clustered data to the client.
[0069] After determining the clustering data, it is also necessary to determine the encoding method for the clustering data, then encode the clustering data using this method, and finally send the encoded clustering data to the client. For example, if the clustering data, i.e., the high-precision map data to be returned, is: pedestrian flow data within 1 kilometer of XX Building in Chaoyang District, Beijing, from 5 pm to 7 pm, then its field encoding would be as follows:
[0070] Beijing: 0001, Chaoyang District: 002, xx Building: 00003, 1 km: 001, Afternoon: 1, 5 pm - 7 pm: 0101, 0110.
[0071] The overall combination is: 0001, 002, 00003, 001, [010, 0110].
[0072] This invention provides an embodiment of the invention that determines the encoding method of the data, uses the encoding method to encode the clustered data obtained by clustering, and sends the encoded clustered data to the client. Based on this, data storage space can be saved and data interaction efficiency can be improved.
[0073] Based on the above embodiments, before receiving the access request for high-precision map data sent by the client, the process includes: receiving an authentication request sent by the client; sending the encryption method of the authentication request to the client; receiving a username and password digest sent by the client, authenticating user permissions based on the username and password digest to obtain an authentication result, and sending the authentication result to the client. The process also includes receiving an authentication token request sent by the client; generating an authentication token based on the authentication token request, and sending the authentication token to the client.
[0074] The browser obtains the username and password, and then sends an authentication request to the server based on the username and password. The username and password can be entered by the user or pre-stored by the browser.
[0075] It should be noted that the server's database stores data used for authenticating user permissions. This data may include usernames and passwords, or it may include usernames and password digests. The password digest is the data obtained by performing a digest calculation on the user's password.
[0076] After receiving the authentication request from the client, the server compares the username and password carried in the authentication request with the username and password stored in the server's database. If the same username and password exist in the database, the authentication is successful; otherwise, the authentication fails.
[0077] Optionally, if the server stores username and password digests, the server can perform a digest calculation on the password carried in the authentication request, compare the calculation result with the password digest stored in the database, and compare the received username with the username stored in the database. If they are the same, the authentication is considered successful; otherwise, the authentication is considered unsuccessful.
[0078] It's important to note that the server can first receive the authentication request from the browser, then send the browser an encrypted version of the request, followed by the username and password digest from the browser. The server then authenticates the user's permissions based on the username and password digest, obtaining the authentication result. For example, before sending the username and password to the server, the browser can communicate with the server to obtain an encrypted version. The browser then calculates a password digest based on this encrypted version, and the server authenticates the user's permissions based on the username and password digest. Because the password digest is no longer plaintext, it offers higher security, thus improving the security of user authentication.
[0079] After receiving the authentication result from the server, the client sends an authentication token request to the server. The server generates an authentication token based on the request and sends it to the client. For example, if the authentication result indicates successful authentication, the client requests an authentication token from the server, which then generates one. The authentication token can expire after a single authentication or automatically after a period of time. For instance, the validity period of the authentication token can be set to 5 minutes. This way, 5 minutes after the server generates the authentication token, regardless of whether the client has used it for user authentication, the authentication token will be set as invalid.
[0080] It should be noted that the token-based authentication mechanism of this invention returns and stores the authentication information to the client. Subsequent visits to other pages require the authentication information to be transmitted from the client back to the server. The specific steps are as follows:
[0081] (1) The client requests login using a username and password;
[0082] (2) The server receives the login request and verifies the username and password;
[0083] (3) After successful verification, the server will issue a token and then send the token to the client;
[0084] (4) After receiving the token, the client can store it, for example, in a cookie or in LocalStorage. Cookie and LocalStorage are containers for temporary data storage.
[0085] (5) The client needs to include the token issued by the server each time it requests resources from the server;
[0086] (6) The server receives the request and then verifies the token carried in the client's request. If the verification is successful, the server returns the requested data to the client.
[0087] Token-based authentication mechanisms offer the following advantages: They support cross-domain access by including the token in the request header, unlike cookies which do not; they are stateless, as the server does not need to store the token, only verifying its correctness, whereas sessions require server-side storage, typically retrieved via the session ID in a cookie; they do not require binding to a specific authentication scheme (traditional username and password login), only needing the generated token to meet expectations; they are suitable for mobile devices (such as Android, iOS, and mini-programs), while native platforms do not support cookies, such as WeChat mini-programs, where each request constitutes a session and requires manual cookie addition; and because they do not rely on cookies, they avoid CSRF (Cross-site request forgery) attacks.
[0088] This invention, in its embodiments, receives an authentication request from a client, then sends an encrypted version of the authentication request to the client. It receives the username and password digest from the client, authenticates the user's permissions based on the username and password digest, obtains the authentication result, and sends the authentication result to the client. It also receives an authentication token request from the client, generates an authentication token based on the request, and sends the authentication token to the client. This invention, based on a token authentication mechanism, performs authentication and uses the authenticated token for communication, thus solving the user permission problem and improving data security.
[0089] Figure 3 This is a schematic diagram of the data processing device provided by the present invention, with reference to... Figure 3The present invention provides a data processing device, including a receiving module 301, a verification module 302 and a clustering module 303.
[0090] The receiving module 301 is used to receive access requests for high-precision map data sent by the client.
[0091] The verification module 302 is used to verify user permissions based on the authentication token carried in the access request;
[0092] The clustering module 303 is used to cluster the high-precision map data in the database based on the access request if the user permission verification is successful, and send the clustered data to the client; wherein the client displays the received clustered data.
[0093] The data processing apparatus provided in this embodiment of the invention receives access requests for high-precision map data sent by a client; verifies user permissions based on the authentication token carried in the access request; if the user permission verification is successful, clusters the high-precision map data in the database based on the access request, and sends the clustered data to the client; wherein the client displays the received clustered data. This invention performs clustering processing on high-precision map data in the database through access requests, determines the data associated with the access request, and achieves rapid display of data related to the access request, thus improving the display efficiency of high-precision map data.
[0094] In one embodiment, the clustering module 303 is specifically used for:
[0095] Based on the access request, the access requirement is determined, and the access requirement is input into the first script;
[0096] The first script calls the second script, and the high-precision map data in the database is clustered based on the second script.
[0097] In one embodiment, the clustering module 303 is specifically used for:
[0098] Based on the second script, a preset clustering algorithm is used to cluster the high-precision map data in the database.
[0099] In one embodiment, the clustering module 303 is specifically used for:
[0100] Determine the data encoding method;
[0101] The clustering data obtained from clustering is encoded using the aforementioned encoding method, and the encoded clustering data is sent to the client.
[0102] In one embodiment, the verification module 302 is specifically used for:
[0103] Determine the comparison result between the authentication token carried in the access request and the target authentication token;
[0104] The user's permissions are verified based on the comparison results.
[0105] In one embodiment, the receiving module 301 is further configured to:
[0106] Receive the authentication token request sent by the client;
[0107] The authentication token is generated based on the authentication token request, and the authentication token is sent to the client.
[0108] In one embodiment, the receiving module 301 is further configured to:
[0109] Receive the authentication request sent by the client;
[0110] The encryption method for sending the authentication request to the client;
[0111] The system receives a username and password digest sent by the client, performs authentication on the user's permissions based on the username and password digest to obtain an authentication result, and sends the authentication result back to the client.
[0112] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a data processing method, which includes:
[0113] Receive access requests for high-precision map data sent by clients;
[0114] Verify user permissions based on the authentication token carried in the access request;
[0115] If the user permission verification is successful, the high-precision map data in the database is clustered based on the access request, and the clustered data is sent to the client; wherein, the client displays the received clustered data.
[0116] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0117] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the data processing methods provided by the methods described above, the method comprising:
[0118] Receive access requests for high-precision map data sent by clients;
[0119] Verify user permissions based on the authentication token carried in the access request;
[0120] If the user permission verification is successful, the high-precision map data in the database is clustered based on the access request, and the clustered data is sent to the client; wherein, the client displays the received clustered data.
[0121] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A data processing method, characterized in that, include: Receive access requests for high-precision map data sent by clients; Verify user permissions based on the authentication token carried in the access request; If the user permission verification is successful, the high-precision map data in the database is clustered based on the access request, and the clustered data is sent to the client; wherein, the client displays the received clustered data; The clustering of high-precision map data in the database based on the access request includes: Based on the access request, the access requirement is determined, and the access requirement is input into the first script; The first script calls the second script, and based on the second script, the high-precision map data in the database is clustered in parallel using the PySpark parallel computing framework and the k-means clustering algorithm to classify the data associated with the access request into different types, so that the client can directly access the corresponding clustered data according to the data type during subsequent accesses; the first script is a Lua script, and the second script is a Python script.
2. The data processing method according to claim 1, characterized in that, Sending the clustered data obtained from the clustering to the client includes: Determine the data encoding method; The clustering data obtained from clustering is encoded using the aforementioned encoding method, and the encoded clustering data is sent to the client.
3. The data processing method according to claim 1, characterized in that, The process of verifying user permissions based on the authentication token carried in the access request includes: Determine the comparison result between the authentication token carried in the access request and the target authentication token; The user's permissions are verified based on the comparison results.
4. The data processing method according to claim 1, characterized in that, Before receiving the high-precision map data access request sent by the client, the following steps are included: Receive the authentication token request sent by the client; The authentication token is generated based on the authentication token request, and the authentication token is sent to the client.
5. The data processing method according to claim 4, characterized in that, Before receiving the authentication token request sent by the client, the process includes: Receive the authentication request sent by the client; The encryption method for sending the authentication request to the client; The system receives a username and password digest sent by the client, performs authentication on the user's permissions based on the username and password digest to obtain an authentication result, and sends the authentication result back to the client.
6. A data processing apparatus, characterized in that, include: The receiving module is used to receive access requests for high-precision map data sent by the client; The verification module is used to verify user permissions based on the authentication token carried in the access request; The clustering module is used to cluster the high-precision map data in the database based on the access request if the user permission verification is successful, and send the clustered data to the client; wherein the client displays the received clustered data. The clustering module is further configured to determine access requirements based on the access request and input the access requirements into a first script; use the first script to call a second script, and based on the second script, use the PySpark parallel computing framework and k-means clustering algorithm to perform parallel clustering on the high-precision map data in the database, so as to classify the data associated with the access request into different types, so that the client can directly access the corresponding clustered data according to the data type during subsequent accesses; the first script is a Lua script and the second script is a Python script.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the data processing method as described in any one of claims 1 to 5.
8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data processing method as described in any one of claims 1 to 5.