Data processing method, device, apparatus, and computer storage medium

Abstract

This invention relates to the field of computer data processing technology and discloses a data processing method. The method includes: filtering multiple original data points included in the data to be processed according to a data processing model to obtain data points to be rendered; wherein the constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point; and rendering the data points to be rendered to obtain processed data. Through the above method, this invention improves the efficiency of rendering massive amounts of data and enhances the user experience.

Description

Technical Field

[0001] The present invention relates to the field of computer data processing technology, specifically to a data processing method, apparatus, device, and computer storage medium. Background Technology

[0002] Currently, when presenting massive amounts of data, such as rendering and displaying map resource data, the speed at which the system loads such data is mainly limited by two factors: the data volume level and the performance of the rendering engine. Existing methods typically involve performing calculations of varying complexity on data within a certain grid area, aggregating neighboring point data, and ultimately displaying a collection of points on the map.

[0003] The inventors of this application discovered during the implementation of the embodiments of the present invention that existing massive data rendering suffers from problems such as low efficiency, computational complexity, and poor user experience. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a data processing method to solve the problems of low efficiency, computational complexity and poor user experience in rendering massive amounts of data in the prior art.

[0005] According to one aspect of the present invention, a data processing method is provided, the method comprising:

[0006] The data processing model filters multiple raw data points in the data to be processed to obtain data points to be rendered; wherein the constraints of the data processing model are determined based on the distribution density information of the multiple raw data points and the data point type information of each raw data point.

[0007] The data points to be rendered are then rendered to obtain the processed data.

[0008] In one alternative approach, the data processing model includes a genetic optimization model; the method further includes:

[0009] The multiple original data points are encoded according to the screening results of each original data point to obtain multiple chromosomes; wherein, one chromosome corresponds to one combination of screening results of the multiple original data points; and the screening result of one original data point corresponds to the encoding value of one gene locus in one chromosome.

[0010] The fitness of each chromosome is determined based on the distribution density information of the multiple original data points and the data point type information of each original data point.

[0011] Based on the fitness, iterative genetic optimization is performed on the multiple chromosomes to obtain the optimal solution;

[0012] Based on the optimal solution, the multiple original data points are filtered accordingly to obtain the data points to be rendered.

[0013] In one alternative approach, the encoding includes binary encoding; the encoded value includes a binary encoded value; the method further includes:

[0014] For each chromosome, the first subfitness is determined based on the distribution density information and the binary coding values ​​at all gene positions on the chromosome;

[0015] The second subfitness is determined based on the data point type information and the binary encoding values ​​at all gene positions in the chromosome;

[0016] The fitness is obtained by weighted summation of the first sub-fitness and the second sub-fitness.

[0017] In one alternative approach, the distribution density information includes the area of ​​the Venn diagram corresponding to each of the original data points; the method further includes:

[0018] For each chromosome, the first subfitness is obtained by summing the binary code values ​​corresponding to all gene positions in the chromosome and the area of ​​the Venn diagram.

[0019] In an alternative approach, the method further includes:

[0020] The binary encoded values ​​corresponding to each of the original data points are corrected based on the preset area threshold and the area of ​​the Venn diagram.

[0021] In an alternative approach, the method further includes:

[0022] The selection weight corresponding to each of the original data points is determined based on the data point type information.

[0023] For each chromosome, the second subfitness is obtained by summing the binary codes corresponding to each original data point in the chromosome and the selection weights.

[0024] In one alternative approach, the plurality of chromosomes constitute a population; the method further includes:

[0025] In each iteration, the chromosomes are screened based on the fitness to obtain a screened population;

[0026] The screened population was subjected to multiple chromosome mutations and crossover processes to obtain an updated population;

[0027] When the preset termination condition is met, genetic evolution is stopped, and the chromosome with the highest fitness in the updated population is determined as the optimal solution.

[0028] According to another aspect of the present invention, a data processing apparatus is provided, comprising:

[0029] The filtering module is used to filter multiple original data points included in the data to be processed according to the data processing model to obtain data points to be rendered; wherein, the constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point.

[0030] The rendering module is used to render the data points to be rendered to obtain the processed data.

[0031] According to another aspect of the present invention, a data processing apparatus is provided, comprising:

[0032] The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus.

[0033] The memory is used to store at least one executable instruction that causes the processor to perform the operation of the data processing method as described in any of the preceding descriptions.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing at least one executable instruction that causes a data processing device to perform the operation of any of the preceding data processing methods.

[0035] This invention, in its embodiments, filters multiple original data points included in the data to be processed according to a data processing model to obtain data points to be rendered. The constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point. The data points to be rendered are then rendered to obtain processed data. This invention, by filtering key information from massive amounts of data based on the distribution density and data point type, reduces the amount of data to be processed, improves the data processing efficiency of massive data rendering, and retains key characteristics of the data, thereby enhancing the data rendering effect and user experience.

[0036] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0037] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0038] Figure 1 A schematic diagram of the point aggregation tool is shown;

[0039] Figure 2 This illustrates the presentation of massive amounts of map data;

[0040] Figure 3 A flowchart illustrating the data processing method provided in an embodiment of the present invention is shown;

[0041] Figure 4 This diagram illustrates the encoding of raw data points in the data processing method provided by an embodiment of the present invention.

[0042] Figure 5 A schematic diagram of the Vino polygon in the data processing method provided by an embodiment of the present invention is shown;

[0043] Figure 6 This diagram illustrates the correction of gene loci in the data processing method provided in an embodiment of the present invention.

[0044] Figure 7 A schematic diagram of the chromosome crossover process in the data processing method provided in an embodiment of the present invention is shown;

[0045] Figure 8 This diagram illustrates the raw data point aggregation process in the data processing method provided by an embodiment of the present invention;

[0046] Figure 9 A schematic diagram of the structure of the data processing apparatus provided in an embodiment of the present invention is shown;

[0047] Figure 10 A schematic diagram of the structure of the data processing device provided in an embodiment of the present invention is shown. Detailed Implementation

[0048] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0049] Before describing the embodiments of the present invention, the prior art and its problems will be further explained:

[0050] The main technologies for processing massive amounts of data include the following:

[0051] Point aggregation technology: When presenting massive amounts of data, clustering algorithms are used to cluster data. Commonly used clustering algorithms include k-means, density-based clustering (DBSCAN), and agglomerative hierarchical clustering. The core idea of ​​these algorithms is to aggregate data within a certain range after acquiring the data in the view window, and to aggregate hundreds or thousands of data points into a point-surface data, thereby effectively compressing the data and reducing client rendering latency. Specifically, in existing technology 1: Yan Haowen, Wang Jiayao. Universal comprehensive algorithm for point group targets based on Voronoi diagram [J]. Journal of Image and Graphics, 2005(05):633-636. It proposes a new point set that adds the idea of ​​virtual boundary to the Voronoi algorithm to construct the fuzzy distribution range of points. The prior art [2]: Wang Jiayao, Deng Hongyan. Research on Cartographic Generalization Model Based on Genetic Algorithm [J]. Journal of Wuhan University (Information Science Edition), 2005(07):565-569. proposed a scheme to select the midpoint based on the maximum distance and minimum distance, thereby ensuring that the selected points cover the entire convex hull range as much as possible, so as to maintain the distribution characteristics of the point group. The problem of the above prior art is that: the symbolization of point features is not considered in prior art 1, which may lead to spatial conflicts such as land cover and congestion caused by scale changes on the generalized map. In prior art 2, since the point target is selected from the perspective of global optimization, the calculation speed is relatively slow compared with other methods. In addition, in order to adapt to the ever-increasing data scale, many map vendors have provided countermeasures, including Baidu Maps, Gaode Maps, etc., which provide point aggregation APIs for WEBGIS developers to use, such as Figure 1 The point aggregation tool provided by Baidu Maps is shown. The core idea of ​​the point aggregation algorithm is to perform calculations of varying complexity on data within a certain grid range, aggregate neighboring point data, and finally display a collection area of ​​multiple points on the map. This significantly reduces the amount of data presented and lowers rendering latency. However, the following unavoidable problems still exist: (1) Most aggregation algorithms require calculating the distance between pairs of points, which has high computational complexity and too high aggregation cost. (2) Using aggregation algorithms may cause two adjacent points to be divided into two different point aggregations, which cannot accurately reflect the data information of the original points.

[0052] Server-side pre-rendered layer technology: Common server-side pre-rendered layer technologies include ArcGIS. ArcGIS, as a scalable platform, provides GIS functionality to individual and group users, whether on desktop, server, in the field, or via the web. Its advantage lies in its rich API offerings, allowing users to combine and call APIs appropriately according to project needs. ArcGIS provides a server-side pre-rendered layer solution where users pre-render data into feature services on the server and publish the layer to the corresponding server interface. The client (browser) retrieves the feature service layer from the server by accessing the corresponding API interface and loads it onto the map (e.g., ...). Figure 2 As shown in the image, this technology enables data presentation. Pre-rendered layer technology relies on creating feature service layers from the data beforehand, currently requiring specialized software such as ArcGIS Desktop. For application systems with large data volumes, the initial investment is high, and there are performance issues in processing massive amounts of resource data. Furthermore, in systems with frequently updated resource data, there are data update delays. In current resource management systems, due to the complexity of business operations and the diversity of data, there are numerous business needs for resource data presentation across different specialties and business areas. Especially in provincial resource management systems, when viewing data resources for a specific city, the dense presentation of large amounts of data in a short period not only poses a risk of system crashes but also puts considerable pressure on the backend server.

[0053] Therefore, a more efficient data processing method is needed to solve the problems of low efficiency and poor user experience in the existing rendering of massive resource data.

[0054] Figure 3 A flowchart of a data processing method provided in an embodiment of the present invention is shown. This method is executed by a computer processing device. The computer processing device may include a mobile phone, a laptop computer, etc. Figure 3 As shown, the method includes the following steps:

[0055] Step 10: Filter the multiple original data points included in the data to be processed according to the data processing model to obtain the data points to be rendered; wherein, the constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point.

[0056] In one embodiment of the present invention, an original data point can correspond to a resource on a map. By rendering the original data point, the corresponding resource can be displayed on the map, such as an icon, text, or a geographical location. Considering that the number of original data points may be massive, such as when the data to be processed is map data, it is necessary to filter the original data points to retain key resource information while reducing the amount of data processing and improving data processing efficiency. Specifically, during the filtering process, on the one hand, it is considered that there are correlations between resource points, that is, the distribution of resources follows certain patterns. Generally, they are mainly distributed in multiple clusters of varying sizes. The larger the cluster, the richer the resources in the cluster, and the greater the weight of its impact on rendering efficiency. Therefore, points with low density distribution, that is, more scattered distribution, can be deleted from the original data points. On the other hand, considering that users pay different amounts of attention to different types of data points, original data points with low user attention can be filtered out based on the data point type.

[0057] It should be noted that, in one embodiment of the present invention, the density information may include the area of ​​the Venn diagram corresponding to each original data point. The density information considered is the density situation corresponding to its aggregation form, which avoids spatial problems such as land cover and congestion caused by the symbolization of point elements, as well as the problem of increased computational complexity caused by selecting the global optimal solution. Therefore, it has certain advantages in the aggregation and processing of massive resource data points.

[0058] Therefore, in one embodiment of the present invention, the data processing model includes a genetic optimization model; the genetic optimization model performs multi-attribute optimization on the population and finally outputs the individual with the best fitness, which corresponds to the screening scheme of the original data points.

[0059] Step 10 also includes:

[0060] Step 101: Encode the multiple original data points according to the screening results of each original data point to obtain multiple chromosomes; wherein, one chromosome corresponds to one combination of screening results of the multiple original data points; and the screening result of one original data point corresponds to the encoding value of one gene locus in one chromosome.

[0061] In one embodiment of the present invention, in order to more intuitively determine the selection or retention of each original data point based on the encoding value, and to facilitate subsequent weighted calculation of the fitness of each chromosome based directly on the encoding values ​​of all gene loci, the encoding can be binary encoding, that is, the encoding value corresponding to each original data point has two possibilities: 0 or 1, and the distribution corresponds to whether or not it is selected. Specifically, as shown in the figure... Figure 4As shown, when the encoding value is 1, it means that the original data point corresponding to the gene locus will be retained as the data point to be rendered, and when the encoding value is 0, it means that the original data point corresponding to the gene locus will be filtered.

[0062] Step 102: Determine the fitness of each chromosome based on the distribution density information of the multiple original data points and the data point type information of each original data point.

[0063] In one embodiment of the present invention, the distribution density information may be the area of ​​the Voronoi polygon (i.e., the Veno diagram) corresponding to the original data points, such as... Figure 5 As shown, the area of ​​the Vinno map is smaller for regions with dense original data points, and conversely, the area of ​​the Vinno map is larger for regions with more dispersed original data points. Therefore, the original data points can be directly filtered based on the area of ​​the Vinno map.

[0064] In one embodiment of the present invention, the data point type information includes the resource type represented by the original data point, as well as the user attention level and resource importance corresponding to that resource type. It is readily understood that original data points for resource types with higher user attention and resource importance are more likely to be retained, thereby ensuring a better user experience in data rendering.

[0065] In another embodiment of the present invention, the encoded value includes a binary encoded value; step 102 further includes:

[0066] Step 1021: For each chromosome, determine the first subfitness based on the distribution density information and the binary coding values ​​at all gene loci in the chromosome.

[0067] In one embodiment of the present invention, the binary encoding-based method can directly determine the first computational weight based on the distribution density information, for... Figure 4 For each chromosome, the binary encoding values ​​of all gene positions are weighted and summed according to the first calculation weight to obtain the first sub-fitness.

[0068] In another embodiment of the present invention, the distribution density information includes the area of ​​the Venn diagram corresponding to each of the original data points; step 1021 further includes:

[0069] Step 211: For each chromosome, the first subfitness is obtained by summing the binary code values ​​corresponding to all gene positions in the chromosome and the area of ​​the Venn diagram.

[0070] In one embodiment of the present invention, the area of ​​the Venn diagram can be directly used as the first calculation weight, and the binary code values ​​corresponding to all gene positions can be weighted and summed according to the first calculation weight to obtain the first subfitness.

[0071] Specifically, the first sub-fitness The following can be calculated:

[0072] ;

[0073] in, Let be the area of ​​the Venn diagram corresponding to the original data point i, and N be the total number of original data points.

[0074] Furthermore, considering that some original data points may correspond to Veno map areas that are too large or too small, i.e. extremely concentrated or extremely dispersed, and that directly using the Veno map area as the first calculation weight for weighted summation may lead to inaccurate first sub-fitness due to the extreme distribution density of the original data points, the gene segment can be corrected on the chromosome gene based on the Veno map area to eliminate the influence of the gene position corresponding to the extreme Veno map area on the first sub-fitness.

[0075] Therefore, in another embodiment of the present invention, the method further includes the following step before step 211:

[0076] Step 212: Correct the binary code value corresponding to each of the original data points according to the preset area threshold and the area of ​​the Venn diagram.

[0077] In one embodiment of the present invention, the binary code values ​​corresponding to the original data points whose area of ​​the Vinio diagram is greater than a preset area threshold are modified from being retained to being filtered, thereby filtering the genes corresponding to the original data points with relatively extreme areas of the Vinio diagram from the influence of the calculation results of the first fitness, reducing the degree of gene abnormality in the population, and making the calculation of the first subfitness more accurate.

[0078] Specifically, an area threshold can be set. For areas smaller than the threshold Points greater than the threshold are retained. Correcting for extreme points, such as... Figure 6 As shown, the binary code of the gene position corresponding to the extreme data point on the gene sequence included in the chromosome can be corrected from 1 to 0, so that it no longer affects the sum of the first subfitness of the population as a whole.

[0079] Step 1022: Determine the second subfitness based on the data point type information and the binary encoding values ​​of all gene loci in the chromosome.

[0080] In one embodiment of the present invention, a second computational weight can be determined first based on the data point type information. For each chromosome, the binary encoding values ​​of all gene positions included therein are weighted and summed according to the second computational weight to obtain the second subfitness.

[0081] Specifically, in one embodiment of the present invention, step 1022 further includes:

[0082] Step 221: Determine the selection weight corresponding to each of the original data points based on the data point type information.

[0083] In one embodiment of the present invention, the resource importance of the original data points is determined based on data point type information, and the selection weight is determined based on the resource importance. The greater the resource importance, the greater its selection weight, and thus the greater the fitness of the second child. This embodiment of the present invention references the concept of "survival of the fittest" in nature, scoring each resource point selection scheme (i.e., each chromosome) based on fitness, and selecting the locally optimal selection scheme as the specimen for the next generation of inheritance.

[0084] Step 222: For each chromosome, the second subfitness is obtained by summing the binary codes corresponding to each original data point in the chromosome and the selection weights.

[0085] Specifically, for each chromosome, its corresponding second sub-fitness The calculation is as follows:

[0086]

[0087] in, Let be the attention coefficient corresponding to the data point type of the original data point i, and N be the total number of the original data points.

[0088] Step 1023: Perform a weighted summation based on the first sub-fitness and the second sub-fitness to obtain the fitness.

[0089]

[0090] in, For the fitness of chromosome i, These are the weighting coefficients corresponding to the distribution density information and the data point type information, respectively.

[0091] Step 103: Perform iterative genetic optimization on the multiple chromosomes based on the fitness to obtain the optimal solution.

[0092] In one embodiment of the present invention, in order to enable the population to acquire new chromosomes, while maintaining the total population size M unchanged, in order to improve the chromosome diversity of the population, so that the superior chromosome genes can be inherited and the genetic algorithm can be driven to converge toward the optimal result, the chromosomes in the population can be screened according to fitness, and the screened chromosomes can be further subjected to gene mutation to obtain the mutated population. Then, a new round of screening is carried out according to fitness, and gene mutation is carried out again based on the screened population. Through multiple rounds of mutation-screening iterative process, the optimal solution is obtained.

[0093] Therefore, in another embodiment of the present invention, the plurality of chromosomes constitute a population; step 103 further includes:

[0094] Step 1031: In each iteration, the chromosomes are screened according to the fitness to obtain the screened population.

[0095] In one embodiment of the present invention, in each iteration, chromosomes with fitness values ​​lower than a preset fitness threshold are removed from the population to obtain a screened population. Optionally, chromosomes arranged in descending fitness order at a preset position can also be removed. By removing chromosomes with poor fitness from the population, the genetic quality of the population is improved.

[0096] Step 1032: Perform multiple chromosome mutations and crossovers on the screened population to obtain an updated population.

[0097] In one embodiment of the present invention, chromosome mutation and crossover processing can be random. Specifically, the chromosome crossover process is as follows: Figure 7 As shown, suppose the initial population size is M, and the length of each chromosome is N. During the second generation, after selection using a fitness function, the population size becomes... , will be carried out The crossover process is repeated to bring the second-generation population to M. This process continues, with at least one chromosome mutation performed in each iteration to obtain a mutated population. Then, the mutated population is screened based on fitness to obtain an optimized population for the next iteration.

[0098] Step 1033: When the preset termination condition is met, stop the genetic evolution and determine the chromosome with the highest fitness in the updated population as the optimal solution.

[0099] In one embodiment of the present invention, the termination condition may be at least one of the following: 1. When the number of generations (i.e., the number of iterations) reaches a certain value; 2. After several generations of inheritance, the individuals in the genetic population no longer change. By determining the chromosome with the highest fitness as the optimal solution, it is possible to retain key data points with high user attention and greater information entropy from a massive amount of original data points.

[0100] Step 104: Based on the optimal solution, filter the multiple original data points accordingly to obtain the data points to be rendered.

[0101] In one embodiment of the present invention, as described above, the original data point corresponding to the gene position is retained or filtered according to the binary encoding value of the gene position in the chromosome of the optimal solution. When the binary encoding value of the gene position is 1, the original data point is retained and selected as the data point to be rendered. When the binary encoding value of the gene position is 0, the original data point is filtered out.

[0102] Step 20: Render the data points to be rendered to obtain the processed data.

[0103] In one embodiment of the present invention, the filtered data points to be rendered are rendered and then displayed. On the one hand, the amount of data is relatively reduced and the rendering efficiency is higher. On the other hand, the rendering effect is guaranteed by using distribution density information.

[0104] For example, data from the resource management system can be selected as experimental data. Preprocessing for data filtering follows the steps described above. Specific resource point types can include: manholes, utility poles, core computer rooms, marker stones, network elements, etc. Taking representative manholes and core computer rooms as examples... Figure 8 As shown: After preprocessing with a genetic algorithm, data points in densely populated resource areas are retained, while those in sparsely populated areas are deleted. Points with special significance or high attention coefficients (such as those representing core data centers) are also retained, resulting in the final data... Figure 8 The results are displayed in the image.

[0105] This invention, in its embodiments, filters multiple original data points within the data to be processed according to a data processing model to obtain data points to be rendered. The constraints of the data processing model are determined based on the distribution density information and data point type information of the multiple original data points. The data points to be rendered are then rendered to obtain processed data. By filtering key information from massive amounts of data based on both data distribution density and data point type, the invention reduces the amount of data to be processed, improves the data processing efficiency of massive data rendering, and preserves key data characteristics, thereby enhancing the data rendering effect and user experience.

[0106] Figure 9 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present invention is shown. Figure 9 As shown, the device 30 includes a filtering module 301 and a rendering module 302.

[0107] The filtering module 301 is used to filter multiple original data points included in the data to be processed according to the data processing model to obtain data points to be rendered; wherein the constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point.

[0108] The rendering module 302 is used to render the data points to be rendered to obtain the processed data.

[0109] The operation process of the data processing device provided in this embodiment of the invention is largely the same as that of the aforementioned method embodiment, and will not be described again.

[0110] The data processing apparatus provided in this embodiment of the invention filters multiple original data points included in the data to be processed according to a data processing model to obtain data points to be rendered. The constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point. The data points to be rendered are then rendered to obtain processed data. By filtering key information from massive amounts of data based on both data distribution density and data point type, the apparatus reduces the amount of data to be processed, improves the data processing efficiency of massive data rendering, and retains key data characteristics, thereby enhancing the data rendering effect and user experience.

[0111] Figure 10 The diagram shows a structural schematic of a data processing device provided in an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the data processing device.

[0112] like Figure 10 As shown, the data processing device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0113] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other network elements such as clients or other servers. The processor 402 executes program 410, specifically performing the relevant steps described above in the data processing method embodiment.

[0114] Specifically, program 410 may include program code, which includes computer-executable instructions.

[0115] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The data processing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0116] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0117] Specifically, program 410 can be called by processor 402 to cause the data processing device to perform the following operations:

[0118] The data processing model filters multiple raw data points in the data to be processed to obtain data points to be rendered; wherein the constraints of the data processing model are determined based on the distribution density information of the multiple raw data points and the data point type information of each raw data point.

[0119] The data points to be rendered are then rendered to obtain the processed data.

[0120] The operation process of the data processing device provided in this embodiment of the invention is largely the same as that of the aforementioned method embodiment, and will not be described again.

[0121] The data processing device provided in this embodiment of the invention filters multiple original data points included in the data to be processed according to a data processing model to obtain data points to be rendered. The constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point. The data points to be rendered are then rendered to obtain processed data. By filtering key information from massive amounts of data based on both data distribution density and data point type, the device reduces the amount of data to be processed, improves the data processing efficiency of massive data rendering, and retains key data characteristics, thereby enhancing the data rendering effect and user experience.

[0122] This invention provides a computer-readable storage medium storing at least one executable instruction that, when executed on a data processing device, causes the data processing device to perform the data processing method described in any of the above method embodiments.

[0123] Executable instructions can be used to cause the data processing device to perform the following operations:

[0124] The data processing model filters multiple raw data points in the data to be processed to obtain data points to be rendered; wherein the constraints of the data processing model are determined based on the distribution density information of the multiple raw data points and the data point type information of each raw data point.

[0125] The data points to be rendered are then rendered to obtain the processed data.

[0126] The execution process of the executable instructions stored in the computer-readable storage medium provided in this embodiment of the invention is largely the same as that in the aforementioned method embodiments, and will not be described again.

[0127] The executable instructions stored in the computer-readable storage medium provided in this embodiment of the invention filter multiple original data points included in the data to be processed according to a data processing model to obtain data points to be rendered. The constraints of the data processing model are determined based on the distribution density information of the multiple original data points and the data point type information of each original data point. The data points to be rendered are then rendered to obtain processed data. By filtering key information from massive amounts of data based on both data distribution density and data point type, the invention reduces the amount of data to be processed, improves the data processing efficiency of massive data rendering, and retains key data characteristics, thereby enhancing the data rendering effect and user experience.

[0128] This invention provides a data processing apparatus for executing the above-described data processing method.

[0129] This invention provides a computer program that can be invoked by a processor to cause a data processing device to execute the data processing method described in any of the above method embodiments.

[0130] This invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed on a computer, cause the computer to perform the data processing method described in any of the above method embodiments.

[0131] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0132] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0133] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0134] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0135] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A data processing method, characterized in that, The method includes: The data processing model filters multiple raw data points in the data to be processed to obtain data points to be rendered; wherein the constraints of the data processing model are determined based on the distribution density information of the multiple raw data points and the data point type information of each raw data point. The data points to be rendered are rendered to obtain the processed data; The data processing model includes a genetic optimization model; the step of filtering multiple original data points in the data to be processed according to the data processing model to obtain data points to be rendered further includes: The multiple original data points are encoded according to the screening results of each original data point to obtain multiple chromosomes; wherein, one chromosome corresponds to one combination of screening results of the multiple original data points; the screening result of one original data point corresponds to the encoding value of one gene locus in one chromosome; the encoding value includes binary encoding value; For each chromosome, the first subfitness is determined based on the distribution density information and the binary coding values ​​at all gene positions on the chromosome; The second subfitness is determined based on the data point type information and the binary encoding values ​​at all gene positions in the chromosome; The fitness is obtained by weighted summation of the first sub-fitness and the second sub-fitness; Based on the fitness, iterative genetic optimization is performed on the multiple chromosomes to obtain the optimal solution; Based on the optimal solution, the multiple original data points are filtered accordingly to obtain the data points to be rendered.

2. The method according to claim 1, characterized in that, The distribution density information includes the area of ​​the Venn diagram corresponding to each of the original data points; the step of determining the first sub-fitness based on the distribution density information and the binary encoded value further includes: For each chromosome, the first subfitness is obtained by summing the binary code values ​​corresponding to all gene positions in the chromosome and the area of ​​the Venn diagram.

3. The method according to claim 2, characterized in that, Before calculating the first sub-fitness by summing the binary code values ​​corresponding to all gene loci on each chromosome and the area of ​​the Venn diagram for each chromosome, the process further includes: The binary encoded values ​​corresponding to each of the original data points are corrected based on the preset area threshold and the area of ​​the Venn diagram.

4. The method according to claim 1, characterized in that, The step of determining the second sub-fitness based on the data point type information and the binary encoding values ​​at all gene loci in the chromosome further includes: The selection weight corresponding to each of the original data points is determined based on the data point type information. For each chromosome, the second subfitness is obtained by summing the binary codes corresponding to each original data point in the chromosome and the selection weights.

5. The method according to claim 1, characterized in that, The multiple chromosomes constitute a population; the iterative genetic optimization of the multiple chromosomes based on the fitness to obtain the optimal solution further includes: In each iteration, the chromosomes are screened based on the fitness to obtain a screened population; The screened population was subjected to multiple chromosome mutations and crossover processes to obtain an updated population; When the preset termination condition is met, genetic evolution is stopped, and the chromosome with the highest fitness in the updated population is determined as the optimal solution.

6. A data processing apparatus, characterized in that, The device includes: A filtering module is used to filter multiple raw data points included in the data to be processed according to a data processing model to obtain data points to be rendered. The constraints of the data processing model are determined based on the distribution density information and data point type information of the multiple raw data points, and the data processing model includes a genetic optimization model. The multiple raw data points are encoded according to their respective filtering conditions to obtain multiple chromosomes. Each chromosome corresponds to a combination of filtering conditions for the multiple raw data points; the filtering condition of each raw data point corresponds to the encoded value of a gene locus in a chromosome; the encoded value includes a binary encoded value. For each chromosome, the first subfitness is determined based on the distribution density information and the binary coding values ​​at all gene positions on the chromosome; The second subfitness is determined based on the data point type information and the binary encoding values ​​at all gene positions in the chromosome; The fitness is obtained by weighted summation of the first sub-fitness and the second sub-fitness; Based on the fitness, iterative genetic optimization is performed on the multiple chromosomes to obtain the optimal solution; Based on the optimal solution, the multiple original data points are filtered accordingly to obtain the data points to be rendered. The rendering module is used to render the data points to be rendered to obtain the processed data.

7. A data processing device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation of the data processing method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on the data processing device, causes the data processing device to perform the operation of the data processing method as described in any one of claims 1-5.

Images