A data processing method, device and electronic equipment

Abstract

The present disclosure provides a data processing method and device and electronic equipment, relates to the technical field of data processing, and particularly relates to the technical field of big data. The specific implementation scheme is as follows: in response to an instruction for calling a target processing set to process target data, a plurality of data processing tools included in the target processing set are determined; each data processing tool is called in turn according to a specified order to process corresponding to-be-processed data, and a processing result of the data processing tool is obtained until the plurality of data processing tools are called, and the processing result of the last-called data processing tool is taken as a processing result of the target data, wherein the to-be-processed data corresponding to the first-called data processing tool is the target data, and the to-be-processed data corresponding to other called data processing tools is a processing result of the last-called data processing tool. The present disclosure can reduce the complexity of user operation.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to the field of big data technology. Background Technology

[0002] The data platform provides a variety of data processing tools. User devices use these tools to process data and enable business applications deployed on the user's device. Summary of the Invention

[0003] This disclosure provides a data processing method, apparatus, and electronic device.

[0004] According to a first aspect of this disclosure, a data processing method is provided, comprising:

[0005] In response to an instruction to invoke a target processing set to process target data, it is determined that the target processing set includes multiple data processing tools;

[0006] Each data processing tool is called sequentially in a specified order to process the corresponding data to be processed, and the processing result of the data processing tool is obtained. This process continues until all data processing tools have been called. The processing result of the last called data processing tool is taken as the processing result of the target data. The data to be processed corresponding to the first called data processing tool is the target data, and the data to be processed corresponding to the other called data processing tools is the processing result of the previous called data processing tool.

[0007] According to a second aspect of this disclosure, a method for generating processing sets is provided, comprising:

[0008] Obtain multiple target tool identifiers, wherein the target tool identifiers are used to identify the corresponding data processing tools;

[0009] A processing set is generated based on the target tool identifier. The processing set includes data processing tools corresponding to the multiple target tool identifiers. When the processing set processes the target data, the data processing tools corresponding to the multiple target tool identifiers are called in a specified order until all the multiple data processing tools have been called.

[0010] According to a third aspect of this disclosure, a data processing apparatus is provided, comprising:

[0011] The processing set parsing module is used to determine the multiple data processing tools included in the target processing set in response to an instruction to call the target processing set to process the target data;

[0012] The serial execution module is configured to sequentially invoke each data processing tool in a specified order to process corresponding to-be-processed data, to obtain a processing result of the data processing tool, until all the data processing tools are invoked, and the processing result of the last invoked data processing tool is taken as the processing result of the target data, wherein the to-be-processed data corresponding to the first invoked data processing tool is the target data, and the to-be-processed data corresponding to other invoked data processing tools is the processing result of the last invoked data processing tool.

[0013] According to a fourth aspect of the present disclosure, a processing set generation apparatus is provided, comprising:

[0014] The information acquisition module is configured to acquire a plurality of target tool identifiers, wherein the target tool identifiers are used to identify corresponding data processing tools.

[0015] The processing set generation module is configured to generate a processing set according to the target tool identifiers, wherein the processing set comprises the data processing tools corresponding to the target tool identifiers, and the data processing tools corresponding to the target tool identifiers are invoked in a specified order when processing target data until all the data processing tools are invoked.

[0016] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising:

[0017] at least one processor; and

[0018] a memory connected with the at least one processor; wherein

[0019] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of the first aspect or the second aspect.

[0020] According to a sixth aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method of any one of the first aspect or the second aspect.

[0021] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, and the computer program, when executed by a processor, implements the method of any one of the first aspect or the second aspect.

[0022] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0024] Figure 1 is a flow diagram of a data processing method according to the present disclosure;

[0025] Figure 2 is a flow diagram of a data set generation method according to the present disclosure;

[0026] Figure 3 is a flow diagram of a target tool identification and order acquisition method according to the present disclosure;

[0027] Figure 4 is a layout diagram of a processing set editing page according to the present disclosure;

[0028] Figure 5 is a structural diagram of a data processing apparatus according to the present disclosure;

[0029] Figure 6 is a structural diagram of a data set generation apparatus according to the present disclosure;

[0030] Figure 7 is a block diagram of an electronic device for implementing the data processing method or processing set generation method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0031] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, descriptions of well-known functions and structures are omitted in the following description.

[0032] In order to more clearly illustrate the data processing method provided by the present disclosure, the possible application scenarios of the data processing method provided by the present disclosure will be exemplarily described below. It should be understood that the following examples are only one possible application scenario of the data processing method provided by the present disclosure, and in other embodiments, the data processing method provided by the present disclosure can also be applied to other possible application scenarios, and the following examples do not limit the data processing method provided by the present disclosure in any way.

[0033] The implementation of a business often relies on data processing. For example, in order to reasonably plan the flow of vehicles in a city to reduce the probability of traffic jams, the collected map data, vehicle data and other data need to be processed. For example, in order to reasonably allocate power resources to avoid power shortages in some areas, the collected power consumption data of each region, the working conditions of each power distribution station and other data need to be processed.

[0034] In the related art, data processing is often completed by a data platform provided by a third party. Users can process data through data processing tools provided by the data platform. The data platform is often provided for different users to use, and the data processing processes required by different users are often different.

[0035] For example, assume that there are five different users, denoted as users 1-5. The data processing process required by user 1 is: step A→step B, the data processing process required by user 2 is: step A→step B→step C, the data processing process required by user 3 is: step A→step C, the data processing process required by user 4 is: step B→step C, and the data processing process required by user 5 is: step C→step A.

[0036] If the data platform provides tools that can be used to implement the data processing process required by each user, such as providing user 1 with a data processing tool that can implement step A→step B, the data platform needs to provide five different data processing tools to meet the actual needs of users 1-5. If the data platform only provides data processing tools that implement a single step, such as only providing data processing tools that implement step A, data processing tools that implement step B, and data processing tools that implement step C, the data platform only needs to provide three different data processing tools.

[0037] It can be seen that, for the data platform, if a special data processing tool is provided for each user, the number of data processing tools that the data platform needs to provide will increase, which is not conducive to the operation and maintenance of the data platform. Therefore, the data platform often only provides data processing tools that implement a single step

[0038] Therefore, in this example, user 2 needs to first call the data processing tool that implements step A to process the data and obtain a first processing result, then call the data processing tool that implements step B to process the data and obtain a second processing result, and then call the data processing tool that implements step C to process the data and obtain a third processing result. The third result is the data processing result required by user 2.

[0039] It can be seen that, in this processing process, user 2 needs to call the data processing tool in the data platform three times to complete the data processing, which has a high operation complexity.

[0040] Based on this, the present disclosure provides a data processing method, as shown in the following, comprising: Figure 1

[0041] S101, in response to an instruction of calling a target processing set to process target data, determining a plurality of data processing tools included in the target processing set.

[0042] S102, sequentially calling each data processing tool to process corresponding to-be-processed data in a specified order, obtaining a processing result of the processing tool, until the plurality of data processing tools are called, and taking the processing result of the last called data processing tool as a processing result of the target data.

[0043] Among them, the to-be-processed data corresponding to the first called data processing tool is the target data, and the to-be-processed data corresponding to other called data processing tools is the processing result of the last called data processing tool.

[0044] By selecting this embodiment, the data processing tools in the processing set can be automatically called in sequence in response to the instruction of calling the processing set, so as to realize data processing. Therefore, the user can reasonably set the data processing tools included in the processing set according to the actual demand, so as to realize data processing through one calling instruction for the processing set. It is not necessary to manually call each data processing tool in sequence, which effectively reduces the operation complexity of data processing.

[0045] The foregoing S101-S102 will be described respectively as follows:

[0046] In S101, the target processing set includes a plurality of data processing tools. The target processing set can be generated by the data processing method provided by the present disclosure, or can be generated by other electronic devices other than the execution subject, and the present disclosure does not make any limitation on this. How to generate the target processing set will be exemplarily described in the following, and will not be described here.

[0047] The plurality of data processing tools determined from the target processing set can be all data processing tools in the target processing set, or can be part of the processing tools in the target processing set. For example, it is assumed that the target processing set includes five data processing tools, which are respectively denoted as data processing tools 1-5, then the plurality of data processing tools determined from the target processing set can be data processing tools 1-5, can be data processing tools 1-3, can be data processing tool 2 and data processing tool 4, etc.

[0048] ​The manner of determining the data processing tools in the target processing set can be different according to different application scenarios, for example, all the data processing tools included in the target processing set can be determined, or the preset number of data processing tools with the highest priority in the target processing set can be determined, or the data processing tools supporting the data format of the target data in the target processing set can be determined, and the present disclosure does not make any limitation in this regard.

[0049] In S102, the specified order can be different according to different application scenarios. For example, in one possible embodiment, the specified order refers to the order of the data processing tools in the target processing set, in another possible embodiment, a priority is set for each data processing tool in advance, and the specified order is in the order from high to low according to the priority of the data processing tools. In yet another possible embodiment, a data processing tool sequence is set in advance, and the data processing tool sequence includes all the data processing tools in the target processing set, and in this embodiment, the specified order refers to the order of the data processing tools in the data processing tool sequence.

[0050] For each data processing tool called, the corresponding data to be processed of the data processing tool is the processing result of the last called data processing tool, and the first called data processing tool is an exception, and the corresponding data to be processed of the first called data processing tool is the target data.

[0051] For example, it is assumed that three data processing tools are determined from the target processing set, which are respectively denoted as data processing tools 1-3, and the calling order is: data processing tool 1→data processing tool 2→data processing tool 3.

[0052] In this example, the data processing tools 1, 2 and 3 are called in turn. The corresponding data to be processed of the data processing tool 1 is the target data, the corresponding data to be processed of the data processing tool 2 is the processing result of the data processing tool 1, and the corresponding data to be processed of the data processing tool 3 is the processing result of the data processing tool 2. The processing result of the last called data processing tool is the processing result of the data processing tool 3.

[0053] The data processing tools included in the target processing set can be different according to different application scenarios, and five possible data processing tools will be described respectively as follows:

[0054] First: coordinate extraction tool.

[0055] The coordinate extraction tool refers to a tool capable of realizing coordinate extraction and recording the extracted coordinates. For example, in one possible embodiment, the manner of calling the coordinate extraction tool to process the data to be processed is as follows:

[0056] S11, invoke the coordinate extraction tool to determine the coordinate information of the features in the data to be processed corresponding to the coordinate extraction tool.

[0057] Determining the coordinate information of features in the data to be processed can refer to determining all the coordinate information of the features in the data to be processed, or it can refer to determining some of the coordinate information of the features in the data to be processed. For example, taking the coordinate information as representing two-dimensional map coordinates, and the two-dimensional map coordinates include horizontal coordinates (x) and vertical coordinates (y), it can mean determining the x-coordinate of the features in the data to be processed, or it can mean determining the y-coordinate of the features in the data to be processed, or it can mean determining both the x-coordinate and y-coordinate of the features in the data to be processed.

[0058] The elements in this article can refer to different types of objects depending on the application scenario, including but not limited to: buildings, vehicles, roads, lakes, mountains, etc.

[0059] S12, invoke the coordinate extraction tool, write the coordinate information into the first preset field corresponding to the feature in the data to be processed, and obtain the processing result of the coordinate extraction tool.

[0060] The first preset field can be a field that already exists in the data to be processed, or it can be a newly added field. For example, suppose there are four fields in the data to be processed, denoted as fields 1-4. In one possible embodiment, field 4 can be used as the first preset field, that is, the determined coordinate information can be recorded in field 4. In another possible embodiment, a new field 5 can be added to the data to be processed, and field 5 can be used as the first preset field, that is, the determined coordinate information can be recorded in field 5.

[0061] For situations where coordinate information in multiple dimensions is determined, such as the aforementioned determination of x-coordinates and y-coordinates, in one possible embodiment, the coordinate information in different dimensions is recorded in the same first preset field; in another possible embodiment, the coordinate information in different dimensions is recorded in different first preset fields. Using this embodiment can improve the efficiency of identifying coordinate information in different dimensions.

[0062] It is understandable that in some application scenarios, it is necessary to convert image data into numerical coordinate information to facilitate subsequent numerical calculations. By selecting this embodiment, which provides a coordinate extraction tool, users can include the coordinate extraction tool in the target processing set to convert image data into coordinate information, thereby meeting the needs of these application scenarios. In other words, selecting this embodiment improves the applicability of the data processing method provided in this disclosure.

[0063] The second type: spatial location verification tool.

[0064] The spatial position verification tool refers to a tool capable of verifying whether the positions of various elements recorded in the data are accurate. Exemplarily, in a possible embodiment, the spatial position verification tool is called to process the to-be-processed data in the following manner:

[0065] S21. The spatial position verification tool is called to determine, according to the coordinate information of the elements in the to-be-processed data corresponding to the spatial position verification tool, the region where the elements are located, as a predicted region.

[0066] The region herein refers to a continuous space in the real space, and there is no intersection between different regions. The division manner of the region can be different according to different application scenarios. Exemplarily, in a possible embodiment, the real space can be divided into different regions according to administrative division.

[0067] The range of each region can be determined according to the division manner of the region. In combination with the coordinate information of the elements in the to-be-processed data and the range of each region, the region where the elements are located can be determined. Exemplarily, it is assumed that there are two regions in total, which are denoted as region 1 and region 2 respectively, wherein the boundary line of region 1 and region 2 is a straight line x = 0, and region 1 is located on the left side (negative direction of x-axis) of the straight line x = 0, and region 2 is located on the right side (positive direction of x-axis) of the straight line x = 0. If the coordinate information of an element indicates that the element is located at coordinates (1, 2), it can be determined that the element is located in region 2.

[0068] S22. The spatial position verification tool is called to determine, as a recorded region, the region indicated by the region information of the elements in the to-be-processed data.

[0069] The region information is information used to indicate the region where the elements are located in the to-be-processed data. It can be understood that the position of an element at a specific time is certain, and therefore, if the coordinate information and the region information are both correct, theoretically, the predicted region determined based on the coordinate information and the recorded region determined based on the region information should be the same. However, due to various reasons, such as abnormal process of recording the to-be-processed data, errors generated in the process of determining the coordinate information, etc., the predicted region and the recorded region can be different.

[0070] Exemplarily, it is assumed that when the target data is recorded, a building belongs to administrative region A in administrative planning, and the region information of the element recorded in the target data indicates that the region is administrative region A. However, due to actual needs, administrative region A is split into administrative region B and administrative region C, and according to the latest administrative division and the coordinates of the building, it is assumed that it is determined that the building is located in administrative region B. In this example, the predicted region is administrative region B, and the recorded region is administrative region A, which are different.

[0071] S23, marking elements different from the predicted area and the recorded area in the data to be processed, obtaining a processing result of the spatial position verification tool.

[0072] As the foregoing analysis, the predicted area should be the same as the recorded area, thus if the predicted area should be different from the recorded area, it is considered that the information recorded in the data to be processed about the element is incorrect, and by marking the element, the user can be reminded to correct, discard, or the like, the incorrect information. Thus the accuracy of subsequent data processing is avoided, that is, the accuracy of data processing is effectively improved.

[0073] The third is an element classification assignment tool.

[0074] The element classification assignment tool refers to a tool capable of importing a user-defined field into data. Exemplarily, in a possible embodiment, the element classification assignment tool is called to process the data to be processed in the following manner:

[0075] S31, obtaining a mapping table.

[0076] The mapping table is used to record the correspondence between the first element attribute and the second element attribute, and according to different application scenarios, the mapping table can be represented in different forms, including but not limited to tables, texts, linked lists, and the like. Exemplarily, in a possible embodiment, the mapping table is represented in the form of Table 1:

[0077] Table 1

[0078]

[0079]

[0080] The second row in the mapping table indicates that the element attribute corresponding to the element Typecode is Typecode1, and the third row indicates that the element attribute corresponding to the element 110110 is 11111.

[0081] The first element attribute and the second element attribute can be any attribute of an element, and the first element attribute and the second element attribute can be the same or different. For example, taking an element as a building as an example, the first element attribute is the element name, and the second element attribute is the coordinate information of the element. For another example, the first element attribute is the coordinate information of the element, and the second element attribute is the construction time of the element. For another example, the first element attribute is the element name, and the second element attribute is the element name.

[0082] S32, for each first element attribute in the data to be processed corresponding to the element classification assignment tool, determining the second element attribute corresponding to the first element attribute in the mapping table as a target second element attribute.

[0083] Still taking the aforementioned Table 1 as an example, assuming that there is a first element attribute "typecode" in the data to be processed, the second element attribute "typecode1" is determined as the target second element attribute corresponding to the first element attribute "typecode".

[0084] S33, calling the element classification assignment tool to sequentially write the target second element attribute into the second preset field corresponding to the first element attribute in the data to be processed for each first element attribute in the data to be processed, to obtain the processing result of the element classification assignment tool.

[0085] It can be understood that in some application scenarios, the information recorded in the target data is incomplete or has errors, for example, the target data includes an element "building", but the target data only records the element name of the "building" and does not record the coordinate information of the "building", and for another example, the target data includes an element "building", and the target data records that the element name of the "building" is "A building", but the "building" is renamed as "B building" for various reasons, so the element name of the "building" recorded in the target data is incorrect.

[0086] By selecting this embodiment, the second element attribute can be imported into the data to be processed by reasonably setting the mapping table, so as to correct the incomplete and / or incorrect information recorded in the data to be processed, thereby improving the accuracy of the information recorded in the processing result, to avoid the influence of these incomplete or incorrect information on the accuracy of subsequent data processing, that is, to effectively improve the accuracy of data processing.

[0087] For example, taking the aforementioned example that only the element name of the "building" is recorded and the coordinate information of the "building" is not recorded as an example, assuming that the element name of the "building" is "A building" and the coordinate information is 111222, adding a table item in the mapping table for indicating that "A building" corresponds to 111222, so that the coordinate information 111222 can be recorded into the second preset field corresponding to the element name "A building" by calling the element classification assignment tool, thereby completing the coordinate information of the "building".

[0088] Taking the aforementioned example that the "building" is renamed as an example, adding a table item in the mapping table for indicating that "A building" corresponds to "B building", so that "B building" can be recorded into the second preset field corresponding to the element name "A building" by calling the element classification assignment tool, thereby recording the correct element name of the "building" in the data to be processed.

[0089] Fourth: attribute field standardization tool.

[0090] The attribute field standardization tool refers to a tool capable of modifying a specific field to conform to a specific specification. For example, assuming that there are multiple pieces of data about the same building in the target data, but the names of the building in different data are different, such as the element name of the building in some data is "A Building", the element name of the building in some other data is "A Mansion", and the element name of the building in some other data is "Building A", it can be seen that the naming rules of the building in different data are different, and the attribute field standardization tool can standardize the naming rules.

[0091] For example, in a possible embodiment, the attribute field standardization tool is called to process the to-be-processed data in the following manner:

[0092] S41, calling the attribute field standardization tool, determining a to-be-modified field that meets a preset matching rule in the to-be-processed data corresponding to the attribute field standardization tool.

[0093] The preset matching rule can be different according to different application scenarios. For example, the preset matching rule can refer to that the field includes a specific keyword, or refer to that the field includes a specific keyword and the order of the keywords meets a specific order.

[0094] Taking the example of the building element name, the preset matching rule can be that the field includes the keyword "A" and the keyword "Building".

[0095] S42, calling the attribute field standardization tool, modifying the to-be-modified field to a preset standard field in the to-be-processed data to obtain a processing result of the attribute field standardization tool.

[0096] The preset standard field should be a field conforming to a specific specification. Taking the example of the building element name, the preset standard field is "A Building".

[0097] By selecting this embodiment, the specific field in the target data can be standardized, and the processing efficiency of subsequent data processing can be effectively improved.

[0098] Fifth: coordinate conversion tool.

[0099] The coordinate conversion tool refers to a tool capable of converting coordinate information into a specific format. For example, in a possible embodiment, the coordinate conversion tool is called to process the to-be-processed data in the following manner:

[0100] The coordinate conversion tool is called to convert the format of the coordinate information in the to-be-processed data corresponding to the coordinate conversion tool into a target format, to obtain a processing result of the coordinate conversion tool.

[0101] The format of the coordinate information herein refers to a representation form of the coordinate information, that is, coordinate information in different formats is represented in different coordinate systems. For example, coordinate information in one format is represented in the WGS84 Mercator projection coordinate system, and coordinate information in another format is represented in the BD09LL geographic coordinate system. Changing the representation form of the coordinate information does not change the spatial position represented by the coordinate.

[0102] By selecting this embodiment, the formats of the coordinate information in the to-be-processed data can be unified, and the processing efficiency of subsequent data processing can be effectively improved.

[0103] In the data processing method provided by the present disclosure, the target processing set can include one or more of the five data processing tools, and can also include other data processing tools other than the five data processing tools, such as a slicing tool for slicing data into multiple slices according to a preset rule.

[0104] Corresponding to the data processing method, the present disclosure further provides a processing set generation method, as shown in Figure 2 The processing set generation method comprises the following steps:

[0105] S201, obtaining a plurality of target tool identifiers. The target tool identifier is used to identify the corresponding data processing tool.

[0106] S202, generating a processing set according to the target tool identifier.

[0107] The processing set comprises a plurality of data processing tools corresponding to the target tool identifiers, respectively. When the target data is processed, the data processing tools corresponding to the plurality of target tool identifiers are called in a specified order until the plurality of data processing tools are called.

[0108] By selecting this embodiment, the processing set can be generated according to the obtained target tool identifier and order, so that the generated processing set can be called to process the target data subsequently, thereby realizing data processing through one call instruction for the processing set. It is not necessary to manually call each data processing tool in sequence, thereby effectively reducing the operation complexity of data processing.

[0109] In S201, the target tool identifier is used to represent the data processing tool, and different target tool identifiers are used to represent different data processing tools. The way of obtaining the target tool identifier and the order can be different according to different application scenarios. For example, in one possible embodiment, a command line instruction input by a user can be received, and the plurality of target tool identifiers and the order of each target tool identifier can be obtained according to the command line instruction input by the user.

[0110] In another possible embodiment, the way of obtaining the target tool identifier is as follows:Figure 3 As shown in the figure, the method comprises the following steps:

[0111] S2011, displaying a toolbar and an editing window.

[0112] The toolbar comprises a plurality of alternative tool identifiers.

[0113] S2012, moving a target alternative tool identifier in the plurality of alternative tool identifiers to the editing window in response to an identifier moving instruction input for the toolbar and the editing window.

[0114] The moving instruction can be input by the user through a drag operation. For example, in response to an operation of the user clicking an alternative tool identifier in the toolbar and dragging it to the editing window, the user-clicked alternative tool identifier is moved to the editing window as the target alternative tool identifier. The moving instruction can also be input by the user through other operations such as a sliding gesture operation or a long-press operation, and the present disclosure does not make any limitation in this regard.

[0115] S2014, obtaining all target alternative tool identifiers in the editing window as target tool identifiers.

[0116] With this embodiment, the user can more intuitively edit the data processing tools included in the processing set on the graphical interface, effectively reducing the operation difficulty of the user.

[0117] The order in which the target tool identifiers are obtained can be as shown in the figure, and the target tool identifiers are obtained in the following manner: Figure 3

[0118] S2013, connecting the plurality of target alternative tool identifiers in the editing window to obtain an identifier pipeline in response to a connection instruction input for the editing window.

[0119] S2015, obtaining an order of each target tool identifier in the identifier pipeline as a specified order corresponding to the target tool identifier.

[0120] With this embodiment, the user can more intuitively edit the order of the data processing tools included in the processing set on the graphical interface, effectively reducing the operation difficulty of the user.

[0121] How to generate a processing set will be described below by way of example in combination with the operation of the user:

[0122] For the convenience of description, the execution subject of the data processing method provided by the present disclosure is taken as a data platform as an example. The user can remotely locate the data platform through a foreground device. The data platform sends page information to the foreground device. The foreground device displays a processing set editing page according to the page information.

[0123] The processing set editing page can be as shown in the figure: Figure 4 ​The shown includes a toolbar and an editing window, the toolbar includes a plurality of alternative tool icons, each alternative tool icon is used to represent a data processing tool. The user can move the icons in the toolbar to the editing window through a drag operation, and can also change the position of each tool icon in the editing window through a drag operation.

[0124] And the user can specify the order of different icons through a specifying order operation. For example, the user can input an order for each icon as the specified order of the icon. For another example, the user can use a workflow connector to connect each icon into an ordered identification pipeline through the connection function provided by the set editing page. The specified order of the icon located upstream of the identification pipeline is higher than the specified order of the icon located downstream of the identification pipeline.

[0125] The flow of data processing required by the user is: spatial position verification → coordinate extraction → coordinate conversion. The user moves icon 1 representing the spatial position verification tool, icon 2 representing the coordinate extraction tool, and icon 3 representing the coordinate conversion tool to the editing window through a drag operation. And use a workflow connector to sequentially connect icon 1, icon 2, and icon 3 through the connection function provided by the set editing page. At this time, the set editing page is as shown in Figure 3 .

[0126] The foreground device generates a processing set according to the icons in the editing window and the specified order. The generated processing set satisfies the following conditions:

[0127] The processing set includes the data processing tools represented by each icon in the editing window. And the order of each data processing tool in the processing set corresponds to the specified order of the data processing tool.

[0128] Taking the editing window as shown in Figure 4 , for example, since the editing window includes icon 1, icon 2, and icon 3, the generated processing set includes the spatial position verification tool, the coordinate extraction tool, and the coordinate conversion tool. And since the order of icon 1 in the editing window shown in Figure 3 is higher than the order of icon 2, and the order of icon 2 is higher than the order of icon 3, that is, the specified order is: spatial position verification tool → coordinate extraction tool → coordinate conversion tool. Therefore, in the processing set: the order of the spatial position verification tool is higher than the order of the coordinate extraction tool, and the order of the coordinate extraction tool is higher than the order of the coordinate conversion tool.

[0129] Participate Figure 5 , Figure 5 The shown is a structural schematic diagram of a data processing apparatus provided by the present disclosure, which includes:

[0130] The processing set analysis module 501 is configured to determine a plurality of data processing tools included in a target processing set in response to an instruction of calling the target processing set to process target data.

[0131] The serial execution module 502 is configured to sequentially call each data processing tool in a specified order to process corresponding to-be-processed data, and obtain a processing result of the data processing tool, until all the plurality of data processing tools are called, and the processing result of the last called data processing tool is taken as a processing result of the target data, wherein the to-be-processed data corresponding to the first called data processing tool is the target data, and the to-be-processed data corresponding to other called data processing tools is a processing result of a last called data processing tool.

[0132] In a possible embodiment, the target processing set includes an extraction coordinate tool.

[0133] The serial execution module 502 sequentially calls each data processing tool in a specified order to process corresponding to-be-processed data, and obtain a processing result of the data processing tool, including:

[0134] The extraction coordinate tool is called to determine coordinate information of an element in the to-be-processed data corresponding to the extraction coordinate tool.

[0135] The extraction coordinate tool is called to write the coordinate information into a first preset field corresponding to the element in the to-be-processed data, and obtain a processing result of the extraction coordinate tool.

[0136] In a possible embodiment, the target processing set includes a spatial position verification tool.

[0137] The serial execution module 502 sequentially calls each data processing tool in a specified order to process corresponding to-be-processed data, and obtain a processing result of the data processing tool, including:

[0138] The spatial position verification tool is called to determine a region where an element in the to-be-processed data corresponding to the spatial position verification tool is located as a predicted region according to coordinate information of the element.

[0139] The spatial position verification tool is called to determine a region represented by region information of the element in the to-be-processed data as a recorded region.

[0140] An element whose predicted region is different from the recorded region in the to-be-processed data is marked, and a processing result of the spatial position verification tool is obtained.

[0141] In a possible embodiment, the target processing set includes an element classification and assignment tool.

[0142] The serial execution module 502 sequentially calls each data processing tool in the specified order to process the corresponding to-be-processed data, and obtains the processing result of the data processing tool, including:

[0143] Obtain a mapping table, and the mapping table is used to record the corresponding relationship between the first element attribute and the second element attribute.

[0144] Determine, for each first element attribute in the to-be-processed data corresponding to the element classification and value assignment tool, the second element attribute corresponding to the first element attribute in the mapping table as a target second element attribute.

[0145] Call the element classification and value assignment tool to sequentially write the target second element attribute into a second preset field corresponding to the first element attribute in the to-be-processed data for each first element attribute in the to-be-processed data, and obtain the processing result of the element classification and value assignment tool.

[0146] In a possible embodiment, the target processing set includes an attribute field standardization tool.

[0147] The serial execution module 502 sequentially calls each data processing tool in the specified order to process the corresponding to-be-processed data, and obtains the processing result of the data processing tool, including:

[0148] Call the attribute field standardization tool to determine, in the to-be-processed data corresponding to the attribute field standardization tool, a to-be-modified field that meets a preset matching rule.

[0149] Call the attribute field standardization tool to modify the to-be-modified field to a preset standard field in the to-be-processed data, and obtain the processing result of the attribute field standardization tool.

[0150] In a possible embodiment, the target processing set includes a coordinate conversion tool.

[0151] The serial execution module 502 sequentially calls each data processing tool in the specified order to process the corresponding to-be-processed data, and obtains the processing result of the data processing tool, including:

[0152] Call the coordinate conversion tool to convert the format of coordinate information in the to-be-processed data corresponding to the coordinate conversion tool into a target format, and obtain the processing result of the coordinate conversion tool.

[0153] Participate Figure 6 , Figure 6 FIG. 1 shows a structural schematic diagram of a data set generation apparatus provided by the present disclosure, which includes:

[0154] The information obtaining module 601 is configured to obtain a plurality of target tool identifiers, wherein the target tool identifiers are used to identify corresponding data processing tools.

[0155] The processing set generating module 602 is configured to generate a processing set according to the target tool identifiers, wherein the processing set comprises data processing tools corresponding to the plurality of target tool identifiers respectively, and the data processing tools corresponding to the plurality of target tool identifiers are invoked in a specified order until the plurality of data processing tools are invoked completely when target data is processed.

[0156] In a possible embodiment, the information obtaining module 601 obtains a plurality of tool identifiers, comprising:

[0157] The tool bar and the editing window are displayed, and the tool bar comprises a plurality of alternative tool identifiers;

[0158] In response to an identifier moving instruction input for the tool bar and the editing window, the plurality of alternative tool identifiers are moved to the editing window;

[0159] All the alternative tool identifiers in the editing window are obtained as target tool identifiers.

[0160] In a possible embodiment, the information obtaining module 601 is further configured to:

[0161] In response to a connection instruction input for the editing window, all the alternative tool identifiers in the editing window are connected to obtain an identifier pipeline;

[0162] An order of each target tool identifier in the identifier pipeline is obtained as the specified order of the target tool identifier.

[0163] In a possible embodiment, the processing set generating module 602 is further configured to:

[0164] A plurality of target tool identifiers are obtained respectively corresponding to specified orders;

[0165] The processing set is generated according to the plurality of target tool identifiers and the specified orders respectively corresponding to the plurality of target tool identifiers.

[0166] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution comply with relevant laws and regulations and do not violate public order and good customs.

[0167] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0168] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0169] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0170] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0171] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as a data processing method or a data set generation method. For example, in some embodiments, the data processing method or the data set generation method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the data processing method or the data set generation method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the data processing method or the data set generation method by other any appropriate means, such as by means of firmware.

[0172] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0173] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0174] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0175] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0176] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0177] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0178] It should be understood that the various forms of flow shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure can be achieved, which is not limited herein.

[0179] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data processing method, comprising: In response to an instruction to invoke a target processing set to process target data, the multiple data processing tools included in the target processing set are determined. Each data processing tool is called sequentially in a specified order to process the corresponding data to be processed, and the processing result of the data processing tool is obtained. This process continues until all data processing tools have been called. The processing result of the last called data processing tool is taken as the processing result of the target data. The data to be processed corresponding to the first called data processing tool is the target data, and the data to be processed corresponding to the other called data processing tools is the processing result of the previous called data processing tool. The target processing set includes a coordinate extraction tool; The step of sequentially calling each data processing tool in a specified order to process the corresponding data to be processed, and obtaining the processing result of the data processing tool, includes: Invoke the coordinate extraction tool to determine the coordinate information of the features in the data to be processed corresponding to the coordinate extraction tool; The coordinate information is written into the first preset field corresponding to the element in the data to be processed, and the processing result of the coordinate extraction tool is obtained. The target processing set includes spatial location verification tools; The spatial location verification tool is invoked, and the region where the element is located is determined based on the coordinate information of the element in the data to be processed corresponding to the spatial location verification tool, which is then used as the prediction region. The region represented by the region information of the element in the data to be processed is determined as the recording region; In the data to be processed, the features that are different between the predicted region and the recorded region are marked to obtain the processing result of the spatial location verification tool; The target processing set includes feature classification and assignment tools; The step of sequentially calling each data processing tool in a specified order to process the corresponding data to be processed, and obtaining the processing result of the data processing tool, includes: Obtain a mapping table, which is used to record the correspondence between the first element attribute and the second element attribute; wherein, the element is a building, the first element attribute is the element name, and the second element attribute is the coordinate information of the element; For each first element attribute in the data to be processed corresponding to the element classification and assignment tool, determine the second element attribute corresponding to each first element attribute in the mapping table, and use it as the target second element attribute. The feature classification and assignment tool is invoked to sequentially assign the target second feature attribute to each first feature attribute in the data to be processed, and write the target second feature attribute into the second preset field corresponding to the first feature attribute in the data to be processed, thereby obtaining the processing result of the feature classification and assignment tool.

2. The method according to claim 1, wherein, The target processing set includes attribute field standardization tools; The step of sequentially calling each data processing tool in a specified order to process the corresponding data to be processed, and obtaining the processing result of the data processing tool, includes: Invoke the attribute field standardization tool to determine the field to be modified that matches the preset matching rule in the data to be processed corresponding to the attribute field standardization tool; The attribute field standardization tool is invoked to modify the field to be modified into a preset standard field in the data to be processed, thereby obtaining the processing result of the attribute field standardization tool.

3. The method according to claim 1, wherein, The target processing set includes coordinate transformation tools; The step of sequentially calling each data processing tool in a specified order to process the corresponding data to be processed, and obtaining the processing result of the data processing tool, includes: The coordinate transformation tool is invoked to convert the coordinate information in the data to be processed corresponding to the coordinate transformation tool into the target format, thereby obtaining the processing result of the coordinate transformation tool.

4. A method for generating processing sets, comprising: Obtain multiple target tool identifiers, wherein the target tool identifiers are used to identify the corresponding data processing tools; A processing set is generated based on the target tool identifiers. The processing set includes data processing tools corresponding to the multiple target tool identifiers. When the processing set processes the target data, the data processing tools corresponding to the multiple target tool identifiers are called in a specified order until all the multiple data processing tools have been called. The processing set is used as a target processing set to implement the data processing method according to any one of claims 1-3.

5. The method according to claim 4, wherein, The acquisition of multiple target tool identifiers includes: Display a toolbar and an editing window, the toolbar including multiple alternative tool icons; In response to an identifier movement command input to the toolbar and the editing window, the target candidate tool identifier from a plurality of candidate tool identifiers is moved to the editing window; Obtain the target candidate tool identifier within the editing window and use it as the target tool identifier.

6. The method according to claim 5, wherein the specified order is obtained in the following manner: In response to a connection command input to the editing window, multiple target candidate tool identifiers within the editing window are connected to obtain an identifier pipeline; Obtain the order of each target tool identifier in the identifier pipeline, as the specified order corresponding to the target tool identifier.

7. The method according to claim 4, wherein generating the processing set based on the target tool identifier comprises: Obtain the specified order corresponding to multiple target tool identifiers; The processing set is generated based on the multiple target tool identifiers and their respective specified order.

8. A data processing apparatus, comprising: The processing set parsing module is used to determine the multiple data processing tools included in the target processing set in response to an instruction to call the target processing set to process the target data. The serial execution module is used to sequentially call each of the data processing tools in a specified order to process the corresponding data to be processed, and obtain the processing result of the data processing tool, until all the data processing tools are called. The processing result of the last called data processing tool is taken as the processing result of the target data. The data to be processed corresponding to the first called data processing tool is the target data, and the data to be processed corresponding to the other called data processing tools is the processing result of the previous called data processing tool. The target processing set includes a coordinate extraction tool; The serial execution module sequentially calls each of the data processing tools in a specified order to process the corresponding data to be processed, and obtains the processing result of the data processing tool, including: Invoke the coordinate extraction tool to determine the coordinate information of the features in the data to be processed corresponding to the coordinate extraction tool; The coordinate information is written into the first preset field corresponding to the element in the data to be processed, and the processing result of the coordinate extraction tool is obtained. The target processing set includes spatial location verification tools; The spatial location verification tool is invoked, and the region where the element is located is determined based on the coordinate information of the element in the data to be processed corresponding to the spatial location verification tool, which is then used as the prediction region. The region represented by the region information of the element in the data to be processed is determined as the recording region; In the data to be processed, the features that are different between the predicted region and the recorded region are marked to obtain the processing result of the spatial location verification tool; The target data processing set includes feature classification and assignment tools; The serial execution module sequentially calls each of the data processing tools in a specified order to process the corresponding data to be processed, and obtains the processing result of the data processing tool, including: Obtain a mapping table, which is used to record the correspondence between the first element attribute and the second element attribute; wherein, the element is a building, the first element attribute is the element name, and the second element attribute is the coordinate information of the element; For each first element attribute in the data to be processed corresponding to the element classification and assignment tool, determine the second element attribute corresponding to each first element attribute in the mapping table, and use it as the target second element attribute. The feature classification and assignment tool is invoked to sequentially assign the target second feature attribute to each first feature attribute in the data to be processed, and write the target second feature attribute into the second preset field corresponding to the first feature attribute in the data to be processed, thereby obtaining the processing result of the feature classification and assignment tool.

9. The apparatus according to claim 8, wherein, The target processing set includes attribute field standardization tools; The serial execution module sequentially calls each of the data processing tools in a specified order to process the corresponding data to be processed, and obtains the processing result of the data processing tool, including: Invoke the attribute field standardization tool to determine the field to be modified that matches the preset matching rule in the data to be processed corresponding to the attribute field standardization tool; The attribute field standardization tool is invoked to modify the field to be modified into a preset standard field in the data to be processed, thereby obtaining the processing result of the attribute field standardization tool.

10. The apparatus according to claim 8, wherein, The target processing set includes coordinate transformation tools; The serial execution module sequentially calls each of the data processing tools in a specified order to process the corresponding data to be processed, and obtains the processing result of the data processing tool, including: The coordinate transformation tool is invoked to convert the coordinate information in the data to be processed corresponding to the coordinate transformation tool into the target format, thereby obtaining the processing result of the coordinate transformation tool.

11. A processing set generation apparatus, comprising: An information acquisition module is used to acquire multiple target tool identifiers, wherein the target tool identifier is used to identify the corresponding data processing tool; A processing set generation module is used to generate a processing set based on the target tool identifiers. The processing set includes data processing tools corresponding to the plurality of target tool identifiers. When the processing set processes target data, the data processing tools corresponding to the plurality of target tool identifiers are called in a specified order until all the plurality of data processing tools have been called. The processing set is used as a target processing set to implement the data processing method according to any one of claims 1-3.

12. The apparatus according to claim 11, wherein, The information acquisition module acquires multiple tool identifiers, including: Display a toolbar and an editing window, the toolbar including multiple alternative tool icons; In response to an identifier movement command input to the toolbar and the editing window, the target candidate tool identifier from a plurality of candidate tool identifiers is moved to the editing window; Obtain the target candidate tool identifier within the editing window and use it as the target tool identifier.

13. The apparatus according to claim 12, wherein the information acquisition module is further configured to: In response to a connection command input to the editing window, multiple alternative tool identifiers within the editing window are connected to obtain an identifier pipeline; Obtain the order of each target tool identifier in the identifier pipeline, as the specified order corresponding to the target tool identifier.

14. The apparatus according to claim 11, wherein the processing set generation module is further configured to: Obtain the specified order corresponding to multiple target tool identifiers; The processing set is generated based on the multiple target tool identifiers and their respective specified order.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3 or 4-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-3 or 4-7.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-3 or 4-7.

Images