Data processing method, device, electronic device and storage medium

By using KD tree and DBSCAN clustering algorithms in artificial intelligence technology to screen and cluster data, the problem of high computational complexity of existing data processing methods is solved, and efficient data processing and hot-spot road segment judgment is achieved.

CN113836358BActive Publication Date: 2024-11-12TANGSHAN XINDU TECHNOLOGY DEVELOPMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111084260.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-14
Publication Date
2024-11-12
Estimated Expiration
2041-09-14

AI Technical Summary

Technical Problem

In the field of artificial intelligence technology, existing data processing methods usually use full statistics and summary, resulting in high computational complexity and low efficiency, especially in big data processing, it is difficult to effectively reduce computational complexity and improve processing efficiency.

Method used

By determining the pending data from the data set and filtering out the target data from the remaining data according to the constraints of the preset search tree, forming a data set, using the KD tree and DBSCAN clustering algorithm for clustering, reducing invalid calculations and improving data Processing efficiency.

Benefits of technology

It effectively reduces the computational complexity of data processing and improves the efficiency and accuracy of data processing, especially in hot-spot road judgment application scenarios, which significantly improves the accuracy and efficiency of GPS position judgment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113836358B_ABST
    Figure CN113836358B_ABST
Patent Text Reader

Abstract

The present disclosure provides a data processing method, device, electronic device and storage medium, which relate to the field of computer technology, and in particular to the field of artificial intelligence technology such as cloud computing and big data processing. The specific implementation scheme is: determine the data to be processed from the data set, and determine the target data from the remaining data based on the data to be processed, the data to be processed and the target data meet the constraints of the preset search tree, the data to be processed and the remaining data together constitute the data set, and cluster the data to be processed and the target data to obtain the clustering result, thereby effectively reducing the computational complexity of data processing, effectively improving the effect of data processing, and effectively improving the efficiency of data processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technology such as cloud computing and big data processing, and specifically to a data processing method, device, electronic device and storage medium. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning, deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] In the related art, the data to be processed is usually fully counted, summarized, and clustered to obtain the data processing results. Summary of the invention

[0004] The present disclosure provides a data processing method, apparatus, electronic device, storage medium, and computer program product.

[0005] According to a first aspect of the present disclosure, a data processing method is provided, comprising: determining data to be processed from a data set; determining target data from remaining data based on the data to be processed, the data to be processed and the target data satisfying constraints of a preset search tree, the data to be processed and the remaining data together constituting the data set; and clustering the data to be processed and the target data to obtain a clustering result.

[0006] According to a second aspect of the present disclosure, a data processing device is provided, comprising: a first determination module, used to determine data to be processed from a data set; a second determination module, used to determine target data from remaining data based on the data to be processed, the data to be processed and the target data satisfy the constraints of a preset search tree, and the data to be processed and the remaining data together constitute the data set; and a clustering module, used to cluster the data to be processed and the target data to obtain a clustering result.

[0007] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data processing method of an embodiment of the present disclosure.

[0008] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, where the computer instructions are used to enable a computer to execute the data processing method disclosed in the embodiments of the present disclosure.

[0009] According to a fifth aspect of the present disclosure, a computer program product is proposed, including a computer program, which implements the data processing method disclosed in the embodiment of the present disclosure when the computer program is executed by a processor.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0017] Figure 6 The schematic block diagram shows an example electronic device that can be used to implement the data processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure.

[0020] It should be noted that the executor of the data processing method of this embodiment is a data processing device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, which may include but is not limited to a terminal, a server, etc.

[0021] The embodiments of the present disclosure relate to the fields of artificial intelligence technologies such as cloud computing and big data processing.

[0022] Among them, Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0023] Cloud computing is a type of distributed computing. It refers to breaking down huge data computing programs into countless small programs through the network "cloud", and then processing and analyzing these small programs through a system composed of multiple servers to obtain results and return them to users.

[0024] Big data processing refers to the process of analyzing and processing huge amounts of data using artificial intelligence. Big data can be summarized as five Vs: volume, velocity, variety, value, and veracity.

[0025] It should be noted that the processes of acquiring, storing, processing and using the data involved in the embodiments of the present disclosure comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0026] like Figure 1 As shown, the data processing method comprises:

[0027] S101: Determine data to be processed from a data set.

[0028] Among them, a set consisting of multiple data can be called a data set, and the data set can be, for example, a global positioning system (GPS) data set of a vehicle. Accordingly, the GPS data of the vehicle currently to be processed can be called data to be processed.

[0029] The data to be processed may also be presented in the form of a GPS data sequence, which includes multiple GPS data, each of which may be referred to as data to be processed. The GPS data sequence may be sampled from the above-mentioned GPS data set, and there is no limitation to this.

[0030] That is to say, the application scenario of the embodiment of the present disclosure may be, for example: processing the vehicle GPS data in the vehicle GPS data set to obtain a corresponding processing result, which may be used to describe the information of a certain hot road section (such as location information, congestion time information, etc., without limitation).

[0031] The following description of the embodiments of the present disclosure can be illustrated with the above-mentioned application scenarios. In addition, the embodiments of the present disclosure can also be applied to any other possible data processing application scenarios (for example, data clustering, data aggregation, etc.) without limitation.

[0032] The data to be processed is determined from the data set, which may be a random selection of part of the data from the data set and use it as the data to be processed, or a designated data in the data set is used as the data to be processed, and there is no limitation on this.

[0033] In the above application scenario, the data in the vehicle's GPS data set may have corresponding time information, that is, the vehicle's GPS data set may be a series of position sequences with time information generated by the vehicle over time.

[0034] Accordingly, in this embodiment, the data to be processed is determined from the data set, and for example, GPS data of the vehicle in a certain time period can be selected from the GPS data set of the vehicle and used as the data to be processed.

[0035] In the embodiments of the present disclosure, the process of acquiring the data to be processed complies with the provisions of relevant laws and regulations and does not violate public order and good morals.

[0036] It should be noted that the above-mentioned data to be processed is not data obtained for a specific user and cannot reflect the personal information of a specific user.

[0037] S102: Determine target data from the remaining data according to the data to be processed, the data to be processed and the target data satisfy the constraints of a preset search tree, and the data to be processed and the remaining data together constitute a data set.

[0038] After the data to be processed is determined from the data set, the target data can be determined from the remaining data according to the data to be processed.

[0039] The data in the data set other than the data to be processed may be referred to as the remaining data, that is, the remaining data and the data to be processed together constitute the data set.

[0040] Among them, the pre-constructed search tree can be called a preset search tree (the preset search tree can be specifically, for example, a K-dimensional search tree (KD tree), without limitation), the preset search tree can be pre-constructed based on multiple data in a data set, and the preset search tree can have corresponding constraints, which can be specifically, for example, constraints on feature dimensions between multiple data in the preset search tree, space constraints, subject constraints, etc., without limitation.

[0041] Among them, the KD tree is essentially a binary search tree, which divides the dimensions of the data through a hyperplane, obtains several subspaces after the division, and searches for data in the subspaces, thereby reducing the time required to calculate adjacent points.

[0042] Among them, the data to be processed and the target data meet the constraints of the preset search tree, and the corresponding constraints can be adaptively configured according to the business needs of the actual business scenario, and there is no restriction on this.

[0043] In this embodiment, a KD tree can be constructed in advance based on the vehicle's GPS data set. Accordingly, the constraints between the data to be processed and the target data described by the KD tree can be, for example, constraints on the location dimension of multiple GPS data related to the vehicle, or constraints on the time dimension of multiple GPS data related to the vehicle, etc., or may also be constraints on any other possible dimension of multiple GPS data related to the vehicle (for example, semantic feature dimension, subject similarity dimension, etc.), without limitation.

[0044] That is to say, in this embodiment, the target data is determined from the remaining data based on the data to be processed. The GPS data set of the vehicle can be used to determine the GPS data of multiple vehicles that satisfy the constraints of the location dimension of multiple GPS data related to the vehicle and the constraints of the time dimension of multiple GPS data related to the vehicle, respectively, based on the data to be processed, and the GPS data of the multiple vehicles determined above are used together as the target data, without limitation.

[0045] In the disclosed embodiment, when in the above-mentioned application scenario, by determining the target data that satisfies the multiple dimensional constraints of multiple GPS data related to the vehicle from the remaining data of the vehicle's GPS data set, a preliminary hotspot section judgment result can be obtained, so that in the subsequent data processing process, the amount of data calculation can be effectively reduced, thereby effectively improving the efficiency of data processing.

[0046] S103: Clustering the data to be processed and the target data to obtain a clustering result.

[0047] After the target data is determined from the remaining data according to the data to be processed, clustering processing can be performed on the data to be processed and the target data to obtain a corresponding processing result, which can be called a clustering result.

[0048] In some embodiments, the processed data and the target data are clustered by adopting corresponding clustering algorithms, such as an unsupervised clustering algorithm (K-means clustering algorithm), a density-based clustering algorithm with noise (Density Based Spatial Clustering Of Applications With Noise, DBSCAN) algorithm, etc., to cluster the processed data and the target data to obtain clustering results.

[0049] Alternatively, any other possible method may be used to cluster the data to be processed and the target data to obtain a clustering result, such as a partitioning method, a hierarchical method, etc., which is not limited.

[0050] In the disclosed embodiment, the DBSCAN algorithm can be used to cluster the processing data and the target data to obtain a clustering result. The DBSCAN algorithm is a density-based clustering algorithm. The density clustering algorithm is generally based on the degree of density of the distribution of the categories through the samples. The connections between samples of the same category are relatively tight. By dividing the closely connected samples into one category, a cluster category can be obtained, and by dividing all groups of closely connected samples into different categories, the final clustering result is obtained.

[0051] In the above application scenario, the GPS data of multiple vehicles are clustered to obtain a clustering result, which can be used to characterize the GPS position of the hotspot section.

[0052] It can be seen that in the embodiment of the present disclosure, by using the data processing method of combining the KD tree with the DBSCAN clustering algorithm, the target data is first screened out through the KD tree, and the target data can be used as the preliminary judgment result of the hot section. Then, the target points after the rough screening of the KD tree are clustered according to the DBSCAN algorithm to obtain the GPS position of the hot section. Since the clustering rough screening is added before the density clustering is performed, the performance of the clustering algorithm can be effectively improved. While reducing a large amount of invalid calculations, the accuracy and judgment efficiency of the GPS position judgment of the hot section are effectively improved. At the same time, the DBSCAN algorithm can better match the application scenarios of hot section judgment through density clustering.

[0053] In this embodiment, by determining the data to be processed from the data set, and determining the target data from the remaining data based on the data to be processed, the data to be processed and the target data satisfy the constraints of a preset search tree, the data to be processed and the remaining data together constitute the data set, and the data to be processed and the target data are clustered to obtain a clustering result, thereby effectively reducing the computational complexity of data processing, effectively improving the effect of data processing, and effectively improving the efficiency of data processing.

[0054] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure.

[0055] like Figure 2 As shown, the data processing method comprises:

[0056] S201: Determine data to be processed from a data set.

[0057] The description of S201 can be specifically referred to the above embodiment, which will not be repeated here.

[0058] S202: Determine target data from the remaining data according to the description dimension, wherein the data to be processed and the target data satisfy the constraint conditions corresponding to the description dimension in the preset search tree.

[0059] The data to be processed may have multiple description dimensions, and different description dimensions are different. The multiple description dimensions may specifically include, for example, a time description dimension, a location description dimension, etc., and there is no limitation on this.

[0060] Among them, since the data to be processed can have multiple description dimensions, correspondingly, there can be multiple constraints corresponding to the multiple description dimensions in the preset search tree.

[0061] That is to say, in this embodiment, target data is determined from the remaining data according to different description dimensions. Data that meets the constraint conditions corresponding to the description dimensions in a preset search tree may be determined from the remaining data according to multiple description dimensions corresponding to the data to be processed, and the data is used as the target data. Since data that meets the constraint conditions corresponding to the description dimensions is determined from the remaining data according to the description dimensions, and the data is used as the target data, the comprehensiveness of the identification of the target data to be clustered can be effectively guaranteed, and the computational amount of data clustering can be effectively reduced, thereby effectively assisting in improving data processing efficiency.

[0062] For example, if the description dimension corresponding to the data to be processed is the time description dimension, and the constraint corresponding to the constraint is the same time, then the target data can be determined from the remaining data based on the time description dimension, and the target data and the data to be processed satisfy the constraint of the same time in the preset search tree.

[0063] S203: Clustering the data to be processed and the target data to obtain a clustering result.

[0064] The description of S203 can be specifically referred to the above embodiment, which will not be repeated here.

[0065] In this embodiment, by determining the data to be processed from the data set, and determining the target data from the remaining data according to the description dimension, wherein the data to be processed and the target data satisfy the constraints corresponding to the description dimension in the preset search tree, the comprehensiveness of the identification of the target data to be clustered can be effectively guaranteed, and the amount of calculation of data clustering can be effectively reduced, and the data to be processed and the target data can be clustered to obtain clustering results, which can effectively improve the effect of data processing and the efficiency of data processing.

[0066] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure.

[0067] like Figure 3 As shown, the data processing method comprises:

[0068] S301: Determine data to be processed from a data set.

[0069] The description of S301 can be specifically referred to the above embodiment, which will not be repeated here.

[0070] S302: According to the first description dimension, multiple candidate data are determined from the remaining data, and the data to be processed and the multiple candidate data respectively satisfy the first constraint condition of the preset search tree, and the first constraint condition is the constraint condition corresponding to the first description dimension.

[0071] The data to be processed has multiple description dimensions, which may specifically include, for example, a time description dimension, a location description dimension, etc. Accordingly, the location description dimension may be referred to as a first description dimension, and the time description dimension may be referred to as a second description dimension.

[0072] For example, the first description dimension can be specifically, for example, longitude and latitude, and the second description dimension can be specifically, for example, time information. Thus, the data to be processed can be processed based on the two description dimensions of time description dimension and location description dimension, thereby effectively saving data processing time during the subsequent execution of the data processing method.

[0073] After determining the data to be processed from the data set, multiple data can be determined from the remaining data according to the first description dimension. The data can be called candidate data. The multiple candidate data and the data to be processed meet the first constraint condition of the preset search tree.

[0074] Among them, the first constraint condition is a constraint condition corresponding to the first description dimension (location description dimension), and the first constraint condition can be specifically, for example, that the locations are A kilometers apart.

[0075] Accordingly, according to the first description dimension, multiple candidate data are determined from the remaining data. According to the time description dimension, data with time information corresponding to the data to be processed in the time period t1-t2 is determined from the remaining data and used as candidate data.

[0076] S303: According to the second description dimension, target data is determined from multiple candidate data, and the data to be processed and the target data satisfy a second constraint condition of a preset search tree, where the second constraint condition is a constraint condition corresponding to the second description dimension.

[0077] After multiple candidate data are determined from the remaining data according to the first description dimension, multiple data can be determined from the multiple candidate data according to the second description dimension. The data can be called target data. The multiple target data and the data to be processed meet the second constraint condition of the preset search tree. Since the target data is determined from the remaining data according to the first description dimension and the second description dimension, a large amount of invalid data can be eliminated from the remaining data, thereby reducing a large amount of invalid calculations, and can effectively reduce the calculation complexity while effectively assisting in improving data processing efficiency.

[0078] The second constraint condition is a constraint condition corresponding to the second description dimension (time description dimension), and the second constraint condition may specifically be, for example, the time period t1-t2.

[0079] Accordingly, according to the second description dimension, the target data is determined from multiple candidate data, and the data with time information of the time period t1-t2 can be further screened out from the multiple candidate data with a distance of A kilometers from the above-mentioned location and used as the target data.

[0080] S304: Determine first clustering information corresponding to the first description dimension between the data to be processed and the target data.

[0081] After determining the target data, the first clustering information corresponding to the first description dimension (position description dimension) between the data to be processed and the target data can be determined. The first clustering information can be used to describe whether the data to be processed and the target data are in the same position.

[0082] That is, in the embodiment of the present disclosure, after the target data is determined, it can be determined whether the data to be processed and the target data correspond to the same position based on the first description dimension (position description dimension) of the data to be processed and the target data.

[0083] For example, in the above application scenario, after the target data is determined from the remaining data in the vehicle's GPS data set, it can be determined whether the vehicle corresponding to the data to be processed and the vehicle corresponding to the target data are located on the same road section based on the location description dimensions of the data to be processed and the target data.

[0084] S305: Determine second clustering information corresponding to the second description dimension between the data to be processed and the target data.

[0085] The second clustering information may be used to describe whether the data to be processed and the target data are located at the same location at the same time.

[0086] That is to say, in the embodiment of the present disclosure, after determining the target data, it can be determined whether the data to be processed and the target data are located at the same location at the same time based on the second description dimension (time description dimension) of the data to be processed and the target data.

[0087] For example, in the above application scenario, after the target data is determined from the remaining data in the vehicle's GPS data set, it can be determined whether the vehicle corresponding to the data to be processed and the vehicle corresponding to the target data are located on the same road section at the same time based on the time description dimensions corresponding to the data to be processed and the target data.

[0088] S306: Clustering the data to be processed and the target data according to the first clustering information and the second clustering information to obtain a clustering result.

[0089] After determining the first clustering information corresponding to the first description dimension between the data to be processed and the target data, and determining the second clustering information corresponding to the second description dimension between the data to be processed and the target data, the data to be processed and the target data can be clustered according to the first clustering information and the second clustering information to obtain a clustering result. Since the data to be processed and the target data are clustered according to the first clustering information and the second clustering information, the computational complexity of the clustering algorithm can be effectively reduced, the execution efficiency of the clustering algorithm can be effectively improved, and the accuracy of the clustering results can be effectively improved.

[0090] Optionally, in some embodiments, the processed data and the target data are clustered according to the first clustering information and the second clustering information to obtain a clustering result. When the first clustering information and the second clustering information meet the clustering conditions, the target road section information is determined according to the first clustering information and the second clustering information. The first clustering information, the second clustering information, and the target road section information are collectively used as the clustering result, thereby effectively improving the accuracy and reference value of the clustering results, and enabling the clustering results to more accurately characterize the situation of the target road section in actual business scenarios, thereby effectively meeting the business needs of actual business scenarios.

[0091] Among them, the target section can specifically be the section corresponding to the first clustering information and the second clustering information determined above, that is, after determining the first clustering information and the second information, the section corresponding to the first clustering information and the second clustering information can be determined according to the first clustering information and the second clustering information, and used as the target section.

[0092] Among them, when in the above application scenario, the target section can be called a hot section.

[0093] Accordingly, the information used to describe the target road section may be referred to as target road section information. The target road section information may specifically include, for example, the identification of the target road section, the location of the target road section, the time information of the target road section congestion, etc., without limitation.

[0094] The clustering condition may specifically be that the data to be processed and the target data are located at the same location, or may be that the data to be processed and the target data are located at the same location at the same time, and there is no limitation to this.

[0095] That is to say, the target road section can be determined according to the first clustering information and the second clustering information, and then the target road section information such as the location of the target road section, the identification of the target road section, and the congestion condition of the target road section can be analyzed and obtained.

[0096] For example, when the clustering condition that the data to be processed and the target data are located at the same position at the same time is met according to the first clustering information and the second clustering information, it can be considered that the hotspot sections corresponding to the first clustering information and the second clustering information are congested.

[0097] Among them, when the data processing method described in the embodiment of the present disclosure is applied to the application scenario of hot section judgment, multiple GPS data of the vehicle can be obtained, and then the multiple GPS data of the vehicle can be roughly screened based on the constraints of the KD tree, so as to obtain preliminary hot section judgment results, and then the DBSCAN clustering algorithm is used to cluster the preliminary hot section judgment results obtained above, so as to obtain the hot section judgment results.

[0098] To sum up, the data processing method described in the embodiment of the present invention avoids the problem of high cost of video data because the data source it relies on is the multiple GPS data of the vehicle instead of video data. It can reduce a large amount of invalid calculations by using the KD tree to perform initial screening on the multiple GPS data of the vehicle, and can reduce the algorithm calculation time in the subsequent DBSCAN clustering algorithm calculation process. At the same time, DBSCAN can consider the problem of hotspot section judgment from the perspective of density through density clustering, which is more in line with the application scenarios of hotspot section judgment, thereby effectively improving the accuracy of hotspot intersection judgment results.

[0099] In this embodiment, by determining the data to be processed from the data set, and determining multiple candidate data from the remaining data according to the first description dimension, and then determining multiple data from the multiple candidate data according to the second description dimension, the data can be called target data, and the multiple target data and the data to be processed meet the second constraint condition of the preset search tree. Since the target data is determined from the remaining data according to the first description dimension and the second description dimension, a large amount of invalid data can be eliminated from the remaining data, thereby reducing a large amount of invalid calculations, and can effectively assist in improving data processing efficiency while effectively reducing the calculation complexity. After determining the first clustering information corresponding to the first description dimension between the data to be processed and the target data, and determining the second clustering information corresponding to the second description dimension between the data to be processed and the target data, the data to be processed and the target data can be clustered according to the first clustering information and the second clustering information to obtain a clustering result. Since the data to be processed and the target data are clustered according to the first clustering information and the second clustering information, the calculation complexity of the clustering algorithm can be effectively reduced, the execution efficiency of the clustering algorithm can be effectively improved, and the accuracy of the clustering result can be effectively improved.

[0100] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure.

[0101] like Figure 4 As shown, the data processing device 40 includes:

[0102] A first determining module 401 is used to determine the data to be processed from the data set;

[0103] A second determination module 402 is used to determine target data from the remaining data according to the data to be processed, the data to be processed and the target data satisfy the constraints of a preset search tree, and the data to be processed and the remaining data together constitute a data set; and

[0104] The clustering module 403 is used to cluster the data to be processed and the target data to obtain a clustering result.

[0105] In some embodiments of the present disclosure, Figure 5 As shown, Figure 5 is a schematic diagram according to the fifth embodiment of the present disclosure, the data processing device 50 comprises: a first determination module 501, a second determination module 502, and a clustering module 503, wherein the second determination module 502 is specifically used to:

[0106] According to the description dimension, target data is determined from the remaining data, wherein the data to be processed and the target data satisfy the constraint conditions corresponding to the description dimension in the preset search tree.

[0107] In some embodiments of the present disclosure, the multiple description dimensions include: a first description dimension and a second description dimension, the first description dimension and the second description dimension are different;

[0108] The second determining module 502 is specifically configured to:

[0109] According to the first description dimension, multiple candidate data are determined from the remaining data, and the data to be processed and the multiple candidate data respectively satisfy the first constraint condition of the preset search tree, and the first constraint condition is a constraint condition corresponding to the first description dimension;

[0110] According to the second description dimension, target data is determined from multiple candidate data, and the data to be processed and the target data satisfy a second constraint condition of a preset search tree, and the second constraint condition is a constraint condition corresponding to the second description dimension.

[0111] In some disclosed embodiments, the clustering module 503 includes:

[0112] A first determination submodule 5031 is used to determine first clustering information corresponding to a first description dimension between the data to be processed and the target data;

[0113] The second determining submodule 5032 is used to determine second clustering information corresponding to the second description dimension between the data to be processed and the target data;

[0114] The clustering submodule 5033 is used to cluster the data to be processed and the target data according to the first clustering information and the second clustering information to obtain a clustering result.

[0115] In some embodiments of the present disclosure, the clustering submodule 5033 is specifically used to:

[0116] If the first clustering information and the second clustering information meet the clustering condition, the target road section information is determined according to the first clustering information and the second clustering information, and the first clustering information, the second clustering information, and the target road section information are collectively used as the clustering result.

[0117] In some embodiments of the present disclosure, the first description dimension is a location description dimension, and the second description dimension is a time description dimension.

[0118] It is understandable that the present embodiment Figure 5 The data processing device 50 in the embodiment may have the same function and structure as the data processing device 40 in the above embodiment, the first determination module 501 may have the same function and structure as the first determination module 401 in the above embodiment, the second determination module 502 may have the same function and structure as the second determination module 402 in the above embodiment, and the clustering module 503 may have the same function and structure as the clustering module 403 in the above embodiment.

[0119] It should be noted that the aforementioned explanation of the data processing method is also applicable to the data processing device of this embodiment and will not be repeated here.

[0120] In this embodiment, by determining the data to be processed from the data set, and determining the target data from the remaining data based on the data to be processed, the data to be processed and the target data satisfy the constraints of a preset search tree, the data to be processed and the remaining data together constitute the data set, and the data to be processed and the target data are clustered to obtain a clustering result, thereby effectively reducing the computational complexity of data processing, effectively improving the effect of data processing, and effectively improving the efficiency of data processing.

[0121] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0122] Figure 6 A schematic block diagram of an example electronic device that can be used to implement the data processing method of an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0123] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0124] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0125] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0128] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or ball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0131] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0133] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data processing method, comprising: Determining data to be processed from the data set, wherein the data set includes a GPS data set of the vehicle; Determine target data from the remaining data of the GPS data set of the vehicle according to the data to be processed, the data to be processed and the target data satisfy the constraint conditions of a preset search tree, and the data to be processed and the remaining data together constitute the data set; and Clustering the data to be processed and the target data to obtain a clustering result, wherein the clustering result is used to characterize the GPS position of the hotspot section; The data to be processed has multiple description dimensions, and different description dimensions are different; Wherein, determining the target data from the remaining data of the GPS data set of the vehicle according to the data to be processed includes: According to the description dimension, determining the target data from the remaining data, wherein the data to be processed and the target data satisfy the constraint condition corresponding to the description dimension in the preset search tree; The multiple description dimensions include: a first description dimension and a second description dimension, wherein the first description dimension and the second description dimension are different; Wherein, determining the target data from the remaining data according to the description dimension includes: According to the first description dimension, a plurality of candidate data are determined from the remaining data, the data to be processed and the plurality of candidate data respectively satisfy a first constraint condition of the preset search tree, the first constraint condition being a constraint condition corresponding to the first description dimension; According to the second description dimension, the target data is determined from the multiple candidate data, the data to be processed and the target data satisfy the second constraint condition of the preset search tree, and the second constraint condition is a constraint condition corresponding to the second description dimension.

2. The method according to claim 1, wherein: The clustering of the to-be-processed data and the target data to obtain a clustering result includes: Determine first clustering information between the data to be processed and the target data corresponding to the first description dimension; Determining second clustering information between the data to be processed and the target data corresponding to the second description dimension; The data to be processed and the target data are clustered according to the first clustering information and the second clustering information to obtain a clustering result.

3. The method according to claim 2, wherein: The clustering the to-be-processed data and the target data according to the first clustering information and the second clustering information to obtain a clustering result includes: If the first clustering information and the second clustering information meet the clustering conditions, the target road section information is determined according to the first clustering information and the second clustering information, and the first clustering information, the second clustering information, and the target road section information are collectively used as the clustering result, wherein the first clustering information is used to describe whether the data to be processed and the target data are in the same location, and the second clustering information is used to describe whether the data to be processed and the target data are in the same location at the same time, and the target road section is the road section corresponding to the first clustering information and the second clustering information.

4. The method according to claim 1, wherein: The first description dimension is a location description dimension, and the second description dimension is a time description dimension.

5. A data processing device, comprising: A first determination module, used to determine the data to be processed from the data set, wherein the data set includes a GPS data set of the vehicle; A second determination module is used to determine target data from the remaining data of the GPS data set of the vehicle according to the data to be processed, the data to be processed and the target data satisfy the constraint conditions of a preset search tree, and the data to be processed and the remaining data together constitute the data set; and A clustering module, used for clustering the data to be processed and the target data to obtain a clustering result, wherein the clustering result is used to characterize the GPS position of the hotspot section; The data to be processed has multiple description dimensions, and different description dimensions are different; Wherein, the second determining module is specifically used to: According to the description dimension, determining the target data from the remaining data of the GPS data set of the vehicle, wherein the data to be processed and the target data satisfy the constraint condition corresponding to the description dimension in the preset search tree; The multiple description dimensions include: a first description dimension and a second description dimension, wherein the first description dimension and the second description dimension are different; Wherein, the second determining module is specifically used to: According to the first description dimension, a plurality of candidate data are determined from the remaining data, the data to be processed and the plurality of candidate data respectively satisfy a first constraint condition of the preset search tree, the first constraint condition being a constraint condition corresponding to the first description dimension; According to the second description dimension, the target data is determined from the multiple candidate data, the data to be processed and the target data satisfy the second constraint condition of the preset search tree, and the second constraint condition is a constraint condition corresponding to the second description dimension.

6. The device according to claim 5, wherein the clustering module comprises: A first determination submodule, used to determine first clustering information corresponding to the first description dimension between the data to be processed and the target data; A second determination submodule, used to determine second clustering information corresponding to the second description dimension between the data to be processed and the target data; A clustering submodule is used to cluster the data to be processed and the target data according to the first clustering information and the second clustering information to obtain a clustering result.

7. The device according to claim 6, wherein: The clustering submodule is specifically used for: If the first clustering information and the second clustering information meet the clustering conditions, the target road section information is determined according to the first clustering information and the second clustering information, and the first clustering information, the second clustering information, and the target road section information are collectively used as the clustering result, wherein the first clustering information is used to describe whether the data to be processed and the target data are in the same location, and the second clustering information is used to describe whether the data to be processed and the target data are in the same location at the same time, and the target road section is the road section corresponding to the first clustering information and the second clustering information.

8. The device according to claim 5, wherein: The first description dimension is a location description dimension, and the second description dimension is a time description dimension.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

11. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.