Data processing method, device and electronic equipment

Abstract

The present disclosure provides a data processing method, device and electronic device, which relate to the field of data processing technology, and specifically to the field of map technology. The specific implementation scheme is: obtaining a first data set related to the characteristics of a first data processing model, wherein the first data processing model is a data processing model obtained after upgrading the second data processing model; performing data processing on the first data set based on the first data processing model and the second data processing model, respectively, to obtain a second data set output by the first data processing model and a third data set output by the second data processing model; comparing the second data set and the third data set to obtain first difference data; classifying the first difference data based on the expression form of the first difference data to obtain a target clustering cluster of the first difference data in the expression form.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, in particular to the field of map technology, and specifically to a data processing method, device and electronic device. Background Art

[0002] The information point, or Point of Interest (POI) system, collects POI information from multiple channels through a data processing model, selects and obtains the optimal value of a certain field. For example, after the name of a tourist attraction is calculated by the data processing model in the POI system, the real name of the tourist attraction can be obtained and displayed to the user.

[0003] Due to the continuous upgrading of the data processing model, the POI data produced by the data processing model will change. Compared with the data processing model before the upgrade, whether the changed data values ​​are consistent with the actual situation in the real world requires verification and analysis of the data.

[0004] Currently, services are usually upgraded by manually modifying the data processing model, adding logs to the service, storing the difference data that has changed after the data processing model is calculated into a file, and randomly extracting data from the file for manual analysis. Summary of the invention

[0005] The present disclosure provides a data processing method, device and electronic device.

[0006] According to a first aspect of the present disclosure, there is provided a data processing method, comprising:

[0007] Acquire a first data set related to the characteristics of a first data processing model, wherein the first data processing model is a data processing model obtained by upgrading the second data processing model, and the first data processing model is used to process basic data of information points (POIs) to obtain POI data displayed by a map service;

[0008] Based on the first data processing model and the second data processing model, respectively, the first data set is processed to obtain a second data set output by the first data processing model and a third data set output by the second data processing model;

[0009] Comparing the second data set with the third data set to obtain first difference data;

[0010] Based on the representation form of the first difference data, the first difference data is classified to obtain a target cluster of the first difference data in the representation form, and the target cluster is used to perform data effect analysis of the first data processing model relative to the second data processing model.

[0011] According to a second aspect of the present disclosure, there is provided a data processing device, comprising:

[0012] A first acquisition module, used to acquire a first data set related to the characteristics of a first data processing model, wherein the first data processing model is a data processing model obtained by upgrading the second data processing model, and the first data processing model is used to process basic data of information points POIs to obtain POI data displayed by a map service;

[0013] A first data processing module, configured to perform data processing on the first data set based on the first data processing model and the second data processing model, respectively, to obtain a second data set output by the first data processing model and a third data set output by the second data processing model;

[0014] A first comparison module, used for comparing the second data set with the third data set to obtain first difference data;

[0015] A classification module is used to classify the first difference data based on the representation form of the first difference data to obtain a target cluster of the first difference data in the representation form, and the target cluster is used to perform data effect analysis of the first data processing model relative to the second data processing model.

[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to at least one processor; wherein,

[0019] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the first aspect.

[0020] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute any one of the methods in the first aspect.

[0021] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements any one of the methods in the first aspect when executed by a processor.

[0022] The technology disclosed in the present invention solves the problem of low accuracy of data effect analysis and improves the accuracy of data effect analysis.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0025] Figure 1 is a flowchart of a data processing method according to the first embodiment of the present disclosure;

[0026] Figure 2 is a structural schematic diagram of a data processing device according to a second embodiment of the present disclosure;

[0027] Figure 3 is a schematic block diagram of an example electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] First embodiment

[0030] like Figure 1 As shown, the present disclosure provides a data processing method, comprising the following steps:

[0031] Step S101: Acquire a first data set related to the characteristics of a first data processing model, where the first data processing model is a data processing model obtained by upgrading a second data processing model, and the first data processing model is used to process basic data of information points POIs to obtain POI data displayed by a map service.

[0032] In this embodiment, the data processing method relates to the field of data processing technology, and in particular to the field of map technology, which can be widely used in the POI data measurement scenario of location-based services (LBS). The data processing method of the disclosed embodiment can be executed by the data processing device of the disclosed embodiment. The data processing device of the disclosed embodiment can be configured in any electronic device to execute the data processing method of the disclosed embodiment. The electronic device can be a server or a terminal device, which is not specifically limited here.

[0033] The first data processing model may be an algorithm model, the first data processing model may be a POI basic data model of an LBS map, and the first data processing model may be a data processing model obtained after the second data processing model is upgraded. The second data processing model upgrade refers to a change in the application strategy involved in the second data processing model, that is, the first data processing model may be obtained after the application strategy involved in the second data processing model is changed, and the first data processing model is used to process the basic data of the POI using the changed application strategy to obtain the POI data displayed by the map service, i.e., the LBS.

[0034] For the POI system, the POI system may include a data processing model for processing POI basic data. The data processing model may use application strategies to process the POI basic data, that is, the data processing model uses application strategies to calculate POI information from multiple different channels (the POI information is the POI basic data), and selects the optimal value of a certain field, that is, selects the optimal POI data for display.

[0035] Among them, due to different application strategies, the POI data calculated by the data processing model for the same POI basic data may be different, and because the POI data in the real world is constantly changing, in order to make the POI data calculated by the data processing model conform to the real world, the application strategy used by the data processing model needs to be continuously updated to match the changes in POI data. The first data processing model is the data processing model after the application strategy involved in the data processing model is changed, and the second data processing model is the data processing model before the application strategy is changed.

[0036] In view of the change of application strategy, before the first data processing model goes online, it is necessary to analyze the data effect after the application strategy change, so that the POI data output by the data processing model can be more consistent with the changes in the real world. That is, the purpose of this embodiment is to process the POI data output by the data processing model to better analyze the data effect output by the first data processing model and improve the accuracy of the data effect analysis.

[0037] The first data set may be a data set related to the characteristics of the first data processing model, and the first data set may include multiple POI basic data. Among them, related to the characteristics of the first data processing model may refer to the characteristics that conform to the first data processing model, such as regional characteristics, industry characteristics, etc. Correspondingly, the first data set related to the characteristics of the first data processing model may include POI basic data that conform to the characteristics of the first data processing model. In an optional embodiment, due to changes in application strategies involved in the data processing model, model characteristics may change, and the first data set may be POI data affected by the change in model characteristics.

[0038] There may be multiple ways to obtain the first data set. For example, a pre-stored first data set related to the characteristics of the first data processing model may be obtained, or a first data set that conforms to the characteristics of the first data processing model and is submitted by professionals such as developers responsible for upgrading the data processing model may be obtained.

[0039] It should be noted that the POI system relies on original POI information from various channels to process data. Due to the different functions of the data processing model upgrade, each application strategy change affects different data. In order to simulate the actual situation in the real world, the first data set and other data can be stored online to ensure the authenticity of the basic data.

[0040] Step S102: Perform data processing on the first data set based on the first data processing model and the second data processing model to obtain a second data set output by the first data processing model and a third data set output by the second data processing model.

[0041] In this step, the first data set may be processed based on the first data processing model to obtain the second data set, and the first data set may be processed based on the second data processing model to obtain the third data set. The processing of the first data set may refer to selecting the optimal POI data from multiple different channels through calculations by the data processing model. The POI data may include data from multiple data fields, such as data in fields such as name, location coordinates, brand, and telephone number.

[0042] In an optional implementation, a first data processing model and a second data processing model may be obtained, and the first data set may be input into the first data processing model and the second data processing model for data processing, respectively, to output a second data set and a third data set.

[0043] In another optional implementation, two sets of environments can be constructed, namely, a first data service system for a test environment and a second data service system for a reference environment. The first data service system is constructed based on the first data processing model, that is, the test environment is a service environment after the data processing model is upgraded, and the second data service system is constructed based on the second data processing model, that is, the reference environment is a service environment before the data processing model is upgraded. Accordingly, a data request can be constructed based on the first data set, and the data request can be sent to the first data service system and the second data service system respectively to obtain a second data set and a third data set.

[0044] Step S103: Compare the second data set and the third data set to obtain first difference data.

[0045] In this step, the second data set and the third data set are both POI data calculated based on the same basic data. The application strategies used by the first data processing model and the second data processing model are different, and the calculated POI data may be different. Therefore, there may be differences between the second data set and the third data set. The data in the second data set and the data in the third data set can be compared one by one to obtain first difference data.

[0046] When comparing, each data field in the POI data may be compared one by one. For example, for the name, the name of the POI in the second data set may be compared with the name of the corresponding POI in the third data set to obtain difference data.

[0047] Step S104: Based on the representation form of the first difference data, classify the first difference data to obtain target clusters of the first difference data in the representation form, and the target clusters are used to perform data effect analysis of the first data processing model relative to the second data processing model.

[0048] In this step, the first difference data may be caused by many reasons. The proportion of difference data caused by one or several reasons in the first difference data may be relatively low. By randomly extracting data, the difference data with a relatively low proportion cannot be extracted, resulting in low accuracy of data effect analysis, that is, it is impossible to accurately judge the impact of the first data processing model after upgrading on the data.

[0049] The expression form of the first difference data may refer to a phenomenon of data difference. For different POIs, the expression forms of the difference data are different. For the same POI, different data fields have different expression forms.

[0050] For example, for different POIs, for the data field "telephone", the expression form of the difference data of a certain POI may change from one phone number to two phone numbers, while the expression form of the difference data of another POI may change from two phone numbers to one phone number, etc. For another example, for the same POI, for the data fields "telephone" and "name", the expression forms of the difference data are different.

[0051] Since different reasons may lead to different manifestations of the first difference data, the first difference data may be classified based on the manifestation of the first difference data. The different reasons leading to the first difference data may be classified to obtain target clustering clusters of the first difference data in different manifestations, that is, the difference data caused by the same reason may be aggregated together.

[0052] The target cluster is used to perform data effect analysis of the first data processing model relative to the second data processing model. The data effect analysis may refer to analyzing the impact of the data after the data processing model is upgraded relative to before the upgrade. When the data effect analysis shows that the POI data output by the first data processing model conforms to the changes in the real world, or when the data effect is evaluated multiple times to be good, the first data processing model can be arranged to go online.

[0053] By classifying the first difference data based on the expression form of the first difference data, the difference data generated by different reasons for the first difference data can be obtained. In this way, when analyzing the data effect, the difference data under each cluster can be analyzed, and when the same amount of difference data is analyzed, the difference causes obtained based on the expression form classification are more than the random sampling method, which can improve the coverage of the causes of the difference data, and further improve the accuracy of the data effect analysis. Among them, evaluation samples can be extracted from each target cluster respectively, and the data accuracy can be verified by making phone calls, checking street view pictures in the real world, and other real-world acquisition tools, so as to analyze the data effect impact brought about by the upgrade of the data processing model based on the evaluation samples.

[0054] In the related art, when analyzing the data effect, each step of data is produced, and it is necessary to check whether the produced data is accurate. For example, when the data set output by the data processing model is produced, it is necessary to manually check whether the data is accurate. When the difference data is produced, it is also necessary to manually check whether the data is accurate. In this embodiment, in the specific process of data processing, an automated process can be used to fully automatically produce target clusters to automatically produce evaluation samples for data effect analysis.

[0055] Specifically, the data processing process can apply the following modules: pipeline unified scheduling module, evaluation environment construction module, difference data output module, clustering module, etc. Among them, the pipeline unified scheduling module can be regarded as a master control service. This service carries the scheduling and task management between the various sub-processes of the evaluation sample output. It is the general entrance exposed to the user. The user starts and manages tasks through this scheduling service. The evaluation environment construction module can automatically construct the evaluation environment, such as constructing a first data service system and a second data service system to process the first data set based on the first data processing model and the second data processing model respectively. The difference data output module can automatically produce difference data based on the evaluation environment constructed by the evaluation environment construction module, so that the clustering module can classify the first difference data based on the expression form, obtain the target clustering cluster, and finally produce the evaluation sample from the target clustering cluster. In this way, the data effect analysis can be realized automatically, thereby improving the evaluation execution efficiency and the analysis efficiency of the data effect.

[0056] Optionally, the first difference data corresponds to at least two data fields, and the step S104 specifically includes:

[0057] Determine a rule set, wherein the rule set includes a representation form of the first difference data in each of the data fields;

[0058] Based on the rule set, determining a mapping relationship between the representation form and the at least two data fields;

[0059] Based on the representation form, the first difference data is classified according to the data fields corresponding to the mapping relationship to obtain a target cluster of each data field in the representation form.

[0060] In this embodiment, the rule set may refer to a clustering planning set, which may include clustering rules for each data field. The clustering rule may be a representation of the data field. For example, for the data field "phone", the representation is a change from one phone to two phones. The clustering rule may be to aggregate the difference data from one phone to two phones.

[0061] Since the difference data for a data field can be expressed in multiple forms, each data field can also include multiple clustering rules. For example, the clustering rule of a data field can be represented by a one-dimensional array [A1, A2, A3, A4]. The relationship between all data fields and their clustering rules is introduced into a dictionary description. The dictionary is a set of rules, such as dictionary C = {A: [A1, A2, A3], B: [A1, B1, A2, B3], ..., F: [A1, B1, F1, F2, A2]}. Among them, dictionary C represents the set of all data fields and clustering rules, A, B, F represent fields, and the characters A1, B1, etc. in brackets represent clustering rules.

[0062] The rule set can be traversed, and the clustering rules of each data field can be traversed to obtain the mapping between the clustering rules and the data fields. Taking clustering rule A1 as an example, the mapping relationship A1 = [A, B, C, ..., F] can be obtained. This mapping relationship can be a full mapping between clustering rule A1 and all data fields, which ensures the rule coverage while avoiding redundancy.

[0063] Accordingly, we can obtain the full mapping set E of all clustering rules and data fields, namely, E = {A1: [A, B, C, ..., F], B1: [B, ..., F], A2: [A, F], A3: [A], B3: [B], F1: [F]}. At this time, the full mapping set E can avoid redundancy while ensuring coverage. Taking dictionary C as an example, clustering rule A1 is valid for all data fields and can be called a general clustering rule. Clustering rule B1 is valid for data fields B and F, and can be called a characteristic clustering rule.

[0064] Afterwards, for each representation form, i.e., each clustering rule, the first difference data can be classified according to the data fields corresponding to the mapping relationship to obtain the target clustering clusters of each data field in the representation form. For example, for clustering rule A1, the first difference data can be classified according to data fields A, B, C, ..., F, and the target clustering clusters of data fields A, B, C, ..., F under clustering rule A1 can be obtained. Each target clustering cluster can include difference data of the same data field in the same representation form.

[0065] In this implementation, by determining a rule set, the rule set includes the expression form of the first difference data in each of the data fields; based on the rule set, determining the mapping relationship between the expression form and the at least two data fields; based on the expression form, classifying the first difference data according to the data fields corresponding to the mapping relationship, and obtaining the target clustering cluster of each of the data fields in the expression form. In this way, the difference data generated by each data field under different reasons can be obtained, and the data effect analysis can be performed in units of data fields, which can further improve the accuracy of the data effect analysis.

[0066] Furthermore, by determining the full mapping between clustering rules and data fields, rule redundancy can be avoided while ensuring rule coverage, and the computational complexity of data processing can be reduced, further improving the efficiency of evaluation execution.

[0067] Optionally, the step S104 specifically includes:

[0068] Decomposing the first difference data to obtain N second difference data, where N is an integer greater than 1;

[0069] For each second difference data, the second difference data is classified based on the representation form of the first difference data to obtain a first cluster of the second difference data in the representation form, and the target cluster includes the first cluster.

[0070] In this implementation, in order to achieve full data analysis as much as possible, the data volume of the first difference data is usually relatively large, and the execution time of the clustering task is long. During the task execution, the difference data cannot be viewed, resulting in a waste of time.

[0071] Therefore, the first difference data can be disassembled to obtain N second difference data, each second difference data can correspond to a subtask, so that the clustering task can be split into multiple subtasks, and after each subtask is completed, the second difference data can be clustered based on the expression form of the first difference data. The clustering method of each subtask can be similar to the clustering method described in the above implementation method, which can ensure that the clustering results of the subtask are quickly obtained and the stability of the clustering task can be guaranteed.

[0072] In the specific implementation process, the pipeline unified scheduling module can be used to schedule the clustering model to realize the task splitting and cluster execution of each subtask.

[0073] Optionally, the step S102 specifically includes:

[0074] constructing a data request based on the first data set, wherein the data request is used to request service data of the first data set;

[0075] Sending the data request to a pre-built first data service system and a pre-built second data service system for data processing, respectively, to obtain the second data set returned by the first data service system and the third data set returned by the second data service system;

[0076] The first data service system is constructed based on the first data processing model, and the second data service system is constructed based on the second data processing model.

[0077] In this implementation, two sets of environments can be constructed by evaluating the environment construction module, namely a first data service system for a test environment and a second data service system for a benchmark environment. The first data service system is constructed based on a first data processing model, that is, the test environment is a service environment after the data processing model is upgraded, and the second data service system is constructed based on a second data processing model, that is, the benchmark environment is a service environment before the data processing model is upgraded.

[0078] You can use the environment deployment tool to add parameter controls to modify business configuration according to business characteristics, so that the data processed by the POI system is not stored in the database, thereby satisfying the requirement that the original data in the database can be reused. And the test environment built can be a subsystem environment to ensure that the evaluation data obtained is the same as the environment of the map service after it goes online, that is, the map online service, so as to ensure that the POI data produced by the first data processing model in the test environment is the same as the POI data after the first data processing model goes online.

[0079] Accordingly, a data request may be constructed based on the first data set, and the data request may be sent to the first data service system and the second data service system respectively to obtain a second data set returned by the first data service system and a third data set returned by the second data service system.

[0080] A difference data output module can be used to automatically construct a data request based on the first data set, and send the data request to the first data service system and the second data service system respectively to obtain the second data set and the third data set, and then compare the second data set and the third data set to obtain the first difference data.

[0081] In this implementation, two sets of environments are constructed based on the first data processing model and the second data processing model, and the second data set and the third data set are generated based on the data request corresponding to the first data set based on the two sets of environments, and the first difference data is obtained by comparing the second data set and the third data set. In this way, the automatic generation of difference data can be achieved, and the generated difference data can be more consistent with the actual environment after the first data processing model is launched, thereby further improving the accuracy of data effect analysis.

[0082] Optionally, the method further includes:

[0083] Acquire a fourth data set, wherein the fourth data set includes data labels of the basic data set;

[0084] Based on the first data processing model, data processing is performed on the basic data set to obtain a fifth data set output by the first data processing model;

[0085] The fifth data set is compared with the fourth data set to obtain second difference data, and the second difference data is used to perform data effect analysis of the first data processing model relative to the second data processing model in combination with the target cluster.

[0086] In this implementation manner, the fourth data set may be real POI data that conforms to the real world, which may be data labels of POIs that conform to the real world and are marked by professionals such as project managers.

[0087] A white-box evaluation method may be introduced. White-box evaluation refers to using white-box samples to analyze the data effect of the first data processing model. Specifically, the white-box samples are some real POI data sets in the real world marked by project managers, namely the fourth data set.

[0088] Based on the first data processing model, the method of processing the basic data set can be: constructing a data request through the basic data set corresponding to the fourth data set and sending it to the first data service system of the test environment, obtaining the POI data calculated by the first data processing model, and obtaining the fifth data set.

[0089] Afterwards, the fifth data set is compared with the fourth data set in a manner similar to that of comparing the second data set with the third data set, to obtain second difference data, which can be the gap between the model processing result and the real world.

[0090] Since white-box samples are real data that conforms to the real world, the white-box evaluation method can avoid manual verification of whether the difference data is reasonable, and can directly evaluate the data effect after the data processing model is upgraded. However, since the sample set of white-box evaluation is a fixed set, the data affected by each change in the data processing model application strategy may be different, so the white-box sample may not meet the coverage of the data affected by the change of the data processing model. Accordingly, the second difference data is combined with the target cluster cluster to perform a data effect analysis of the first data processing model relative to the second data processing model, which can meet the coverage of the data affected by the change of the data processing model, and the accuracy of the data effect analysis can be further improved through double evaluation. In addition, since the white-box evaluation uses true value data, it can reduce manpower investment and improve the efficiency of evaluation execution.

[0091] Second embodiment

[0092] like Figure 2 As shown, the present disclosure provides a data processing device 200, including:

[0093] A first acquisition module 201 is used to acquire a first data set related to the characteristics of a first data processing model, wherein the first data processing model is a data processing model obtained by upgrading the second data processing model, and the first data processing model is used to process basic data of information points POIs to obtain POI data displayed by a map service;

[0094] A first data processing module 202, configured to perform data processing on the first data set based on the first data processing model and the second data processing model, respectively, to obtain a second data set output by the first data processing model and a third data set output by the second data processing model;

[0095] A first comparison module 203, configured to compare the second data set with the third data set to obtain first difference data;

[0096] The classification module 204 is used to classify the first difference data based on the representation form of the first difference data to obtain a target cluster of the first difference data in the representation form, and the target cluster is used to perform data effect analysis of the first data processing model relative to the second data processing model.

[0097] Optionally, the first difference data corresponds to at least two data fields, and the classification module 204 includes:

[0098] A first determining unit, configured to determine a rule set, wherein the rule set includes a representation form of the first difference data in each of the data fields;

[0099] A second determining unit, configured to determine a mapping relationship between the representation form and the at least two data fields based on the rule set;

[0100] The first classification unit is used to classify the first difference data according to the data fields corresponding to the mapping relationship based on the representation form, so as to obtain a target cluster of each data field in the representation form.

[0101] Optionally, the classification module 204 includes:

[0102] a disassembling unit, configured to disassemble the first difference data to obtain N second difference data, where N is an integer greater than 1;

[0103] The second classification unit is used to classify each second difference data based on the expression form of the first difference data to obtain a first cluster of the second difference data in the expression form, and the target cluster includes the first cluster.

[0104] Optionally, the first data processing module 202 is specifically configured to:

[0105] constructing a data request based on the first data set, wherein the data request is used to request service data of the first data set;

[0106] Sending the data request to a pre-built first data service system and a pre-built second data service system for data processing, respectively, to obtain the second data set returned by the first data service system and the third data set returned by the second data service system;

[0107] The first data service system is constructed based on the first data processing model, and the second data service system is constructed based on the second data processing model.

[0108] Optionally, also include:

[0109] A second acquisition module, used to acquire a fourth data set, wherein the fourth data set includes data labels of the basic data set;

[0110] A second data processing module, configured to perform data processing on the basic data set based on the first data processing model to obtain a fifth data set output by the first data processing model;

[0111] The second comparison module is used to compare the fifth data set with the fourth data set to obtain second difference data, and the second difference data is used to analyze the data effect of the first data processing model relative to the second data processing model in combination with the target cluster.

[0112] The data processing device 200 provided in the present disclosure can implement each process implemented by the data processing method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0113] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0114] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0115] Figure 3 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0116] like Figure 3 As shown, the device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0117] A number of components in the device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0118] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0125] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0126] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data processing method, comprising: Acquire a first data set related to the characteristics of a first data processing model, wherein the first data processing model is a data processing model obtained by upgrading the second data processing model, and the first data processing model is used to process basic data of information points (POIs) to obtain POI data displayed by a map service; Based on the first data processing model and the second data processing model, respectively, the first data set is processed to obtain a second data set output by the first data processing model and a third data set output by the second data processing model; Comparing the second data set with the third data set to obtain first difference data; Based on the representation form of the first difference data, the first difference data is classified to obtain a target cluster of the first difference data in the representation form, and the target cluster is used to perform data effect analysis of the first data processing model relative to the second data processing model.

2. The method according to claim 1, wherein: The first difference data corresponds to at least two data fields, and the first difference data is classified based on the representation form of the first difference data to obtain a target cluster of the first difference data in the representation form, including: Determine a rule set, wherein the rule set includes a representation form of the first difference data in each of the data fields; Based on the rule set, determining a mapping relationship between the representation form and the at least two data fields; Based on the representation form, the first difference data is classified according to the data fields corresponding to the mapping relationship to obtain a target cluster of each data field in the representation form.

3. The method according to claim 1, wherein: The classifying the first difference data based on the representation form of the first difference data to obtain a target cluster of the first difference data in the representation form includes: Decomposing the first difference data to obtain N second difference data, where N is an integer greater than 1; For each second difference data, the second difference data is classified based on the representation form of the first difference data to obtain a first cluster of the second difference data in the representation form, and the target cluster includes the first cluster.

4. The method according to claim 1, wherein: The performing data processing on the first data set based on the first data processing model and the second data processing model respectively to obtain a second data set output by the first data processing model and a third data set output by the second data processing model comprises: constructing a data request based on the first data set, wherein the data request is used to request service data of the first data set; Sending the data request to a pre-built first data service system and a pre-built second data service system for data processing, respectively, to obtain the second data set returned by the first data service system and the third data set returned by the second data service system; The first data service system is constructed based on the first data processing model, and the second data service system is constructed based on the second data processing model.

5. The method according to claim 1, further comprising: Acquire a fourth data set, wherein the fourth data set includes data labels of the basic data set; Based on the first data processing model, data processing is performed on the basic data set to obtain a fifth data set output by the first data processing model; The fifth data set is compared with the fourth data set to obtain second difference data, and the second difference data is used to perform data effect analysis of the first data processing model relative to the second data processing model in combination with the target cluster.

6. A data processing device, comprising: A first acquisition module, used to acquire a first data set related to the characteristics of a first data processing model, wherein the first data processing model is a data processing model obtained by upgrading the second data processing model, and the first data processing model is used to process basic data of information points POIs to obtain POI data displayed by a map service; A first data processing module, configured to perform data processing on the first data set based on the first data processing model and the second data processing model, respectively, to obtain a second data set output by the first data processing model and a third data set output by the second data processing model; A first comparison module, used for comparing the second data set with the third data set to obtain first difference data; A classification module is used to classify the first difference data based on the representation form of the first difference data to obtain a target cluster of the first difference data in the representation form, and the target cluster is used to perform data effect analysis of the first data processing model relative to the second data processing model.

7. The device according to claim 6, wherein: The first difference data corresponds to at least two data fields, and the classification module includes: A first determining unit, configured to determine a rule set, wherein the rule set includes a representation form of the first difference data in each of the data fields; A second determining unit, configured to determine a mapping relationship between the representation form and the at least two data fields based on the rule set; The first classification unit is used to classify the first difference data according to the data fields corresponding to the mapping relationship based on the representation form, so as to obtain a target cluster of each data field in the representation form.

8. The device according to claim 6, wherein: The classification module comprises: a disassembling unit, configured to disassemble the first difference data to obtain N second difference data, where N is an integer greater than 1; The second classification unit is used to classify each second difference data based on the expression form of the first difference data to obtain a first cluster of the second difference data in the expression form, and the target cluster includes the first cluster.

9. The device according to claim 6, wherein: The first data processing module is specifically used to: constructing a data request based on the first data set, wherein the data request is used to request service data of the first data set; Sending the data request to a pre-built first data service system and a pre-built second data service system for data processing, respectively, to obtain the second data set returned by the first data service system and the third data set returned by the second data service system; The first data service system is constructed based on the first data processing model, and the second data service system is constructed based on the second data processing model.

10. The apparatus according to claim 6, further comprising: A second acquisition module, used to acquire a fourth data set, wherein the fourth data set includes data labels of the basic data set; A second data processing module, configured to perform data processing on the basic data set based on the first data processing model to obtain a fifth data set output by the first data processing model; The second comparison module is used to compare the fifth data set with the fourth data set to obtain second difference data, and the second difference data is used to analyze the data effect of the first data processing model relative to the second data processing model in combination with the target cluster.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images