A data processing method, system, apparatus, device, and storage medium

Abstract

This disclosure provides a data processing method, system, apparatus, device, and storage medium, relating to the field of data processing technology, and particularly to the field of autonomous driving technology. The specific implementation scheme is as follows: receiving a data acquisition request for a target service sent by a client; in response to receiving the data acquisition request, obtaining a target data matching model set for the target service; performing data filtering processing on tagged takeover data based on the filtering items of the target data matching model; wherein the content represented by the tags belongs to the candidate content of the filtering items; and feeding back the processing result of the data filtering processing to the client. It is evident that this scheme can improve data acquisition efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to the field of autonomous driving technology, specifically to a data processing method, system, apparatus, device, and storage medium. Background Technology

[0002] In the field of autonomous driving, takeover data is a more valuable type of scenario data in road testing, and it is extremely important for the iteration of simulation testing algorithms. Takeover data can include vehicle state data, road structure data, and obstacle data collected during vehicle road testing, etc., and it is usually generated due to certain data takeover reasons.

[0003] In related technologies, simulation testing for different algorithm directions requires users to manually retrieve the necessary control data from massive amounts of control data for simulation testing of that algorithm direction. Summary of the Invention

[0004] This disclosure provides a data processing method, system, apparatus, device, and storage medium.

[0005] According to one aspect of this disclosure, a data processing method is provided, applied to a server, the method comprising:

[0006] Receive a data acquisition request sent by a client for a target service; wherein the target service is a service for data processing using managed data;

[0007] In response to receiving the data acquisition request, a target data matching model set for the target service is obtained;

[0008] Based on the filtering options of the target data matching model, data filtering processing is performed on the takeover category data with tags; wherein, the content represented by the tags belongs to the candidate content of the filtering options;

[0009] The processing results of the data filtering are then fed back to the client.

[0010] According to another aspect of this disclosure, a data processing system is provided, including a client and a server;

[0011] The server is configured to receive a data acquisition request for a target business sent by a client; in response to receiving the data acquisition request, obtain a target data matching model set for the target business; perform data filtering processing on the takeover-type data with tags based on the filtering items of the target data matching model; and feed back the processing result of the data filtering processing to the client; wherein, the target business is a business that uses takeover-type data for data processing; and the content represented by the tags belongs to the candidate content of the filtering items;

[0012] The client is used to receive the processing results fed back by the server.

[0013] According to another aspect of this disclosure, a data processing apparatus is provided for use on a server, the apparatus comprising:

[0014] A receiving module is used to receive a data acquisition request sent by a client for a target service; wherein the target service is a service for data processing using takeover-type data;

[0015] The acquisition module is used to obtain the target data matching model set for the target business in response to receiving the data acquisition request;

[0016] The filtering module is used to perform data filtering processing on the takeover-type data with tags based on the filtering items of the target data matching model; wherein the content represented by the tags belongs to the candidate content of the filtering items;

[0017] The feedback module is used to send the processing results of the data filtering to the client.

[0018] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the data processing methods described above.

[0019] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the data processing method according to any of the preceding claims.

[0020] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the data processing method according to any of the preceding claims.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0023] Figure 1 This is a flowchart of a data processing method according to the present disclosure;

[0024] Figure 2 This is a schematic diagram of a label type according to this disclosure;

[0025] Figure 3 This is a schematic diagram of a data processing flow according to this disclosure;

[0026] Figure 4 This is a schematic diagram of the structure of a data processing system according to the present disclosure;

[0027] Figure 5 This is a schematic diagram of the structure of a data processing device according to the present disclosure;

[0028] Figure 6 This is a block diagram of an electronic device used to implement the data processing method of the embodiments of this disclosure. Detailed Implementation

[0029] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] In the field of autonomous driving, when certain problems occur during vehicle operation, it is often necessary for staff to take over the vehicle manually. The scene data collected before and after the need for manual intervention, such as road structure data and obstacle data of the geographical area where the vehicle is located, constitutes the initial takeover data.

[0031] In related technologies, initial takeover data is sliced ​​and structured to generate takeover data with a certain index. When simulation tests are needed for different algorithm directions, users need to manually retrieve the takeover data required for the simulation test of that algorithm direction from massive amounts of takeover data as input data for the simulation test algorithm, resulting in low data acquisition efficiency.

[0032] Based on the above, in order to improve the efficiency of data acquisition, this disclosure provides a data processing method, system, apparatus, device, and storage medium.

[0033] Below, we will first introduce a data processing method provided by an embodiment of this disclosure.

[0034] The data processing method provided in this disclosure can be applied to a server that can communicate with a client. In practical applications, the server can receive data acquisition requests sent by the client and return the request results to the client.

[0035] Specifically, the entity executing this data processing method can be a data processing device. For example, when this data processing method is applied to a server, the data processing device can be a computer program running on the server, which can be used to acquire takeover-type data.

[0036] The data processing method provided in this embodiment may include the following steps:

[0037] Receive a data acquisition request sent by a client for a target service; wherein the target service is a service for data processing using managed data;

[0038] In response to receiving the data acquisition request, a target data matching model set for the target service is obtained;

[0039] Based on the filtering options of the target data matching model, data filtering processing is performed on the takeover category data with tags; wherein, the content represented by the tags belongs to the candidate content of the filtering options;

[0040] The processing results of the data filtering are then fed back to the client.

[0041] In the solution provided in this disclosure, since the values ​​of the filter items in the target data matching model are selected from the candidate content of the filter items, and the candidate content of the filter items is the content represented by the tags of the takeover data, the filter items can be used to filter out takeover data that matches the filter items from the tagged takeover data. Therefore, users only need to set up a target data matching model for the target business in the client, that is, to configure the content of the filter items in the initial data matching model, and then send a data acquisition request to the server. After receiving the data acquisition request, the server can filter data from the tagged takeover data according to the values ​​of the filter items in the target data matching model, thereby automatically retrieving the required takeover data from massive amounts of takeover data. It is evident that this solution improves data acquisition efficiency compared to the existing technology of manually retrieving the desired takeover data from massive amounts of takeover data.

[0042] The data processing method provided in the embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0043] like Figure 1 As shown, the data processing method provided in this embodiment, when applied to a server, may include steps S101-S104:

[0044] S101, Receive a data acquisition request sent by the client for a target service; wherein, the target service is a service for data processing using the managed data;

[0045] In this embodiment, a user can send a data retrieval request for a specific service to the server via a client. The server receives the data retrieval request and responds accordingly. In practical applications, the client can display an interactive interface for data retrieval to the user. Through this interface, the user can issue a data retrieval command for a specific service, and upon detecting the command, the client can send a data retrieval request for that specific service to the server.

[0046] Understandably, since the target business is one that uses takeover data for data processing, and different businesses require different types of takeover data, a data acquisition request for the target business is essentially a request to acquire the takeover data required by that business. For example, if the target business is a simulation test business for collision direction, then the data acquisition request for the target business is a request to acquire takeover data for vehicle collisions.

[0047] S102, in response to receiving the data acquisition request, obtain the target data matching model set for the target business;

[0048] The target data matching model is a model obtained by configuring the filtering items of the initial data matching model based on the data requirements of the target business.

[0049] In this embodiment, the data acquisition request may carry a target data matching model set for the target business, so that the server can obtain the target data matching model from the data acquisition request after receiving the data acquisition request; or, the data acquisition request may not carry a target data matching model, but when the server receives the data acquisition request, it can request the target data matching model set for the target business from the client, which is reasonable.

[0050] Understandably, different business operations have different data requirements. For example, algorithm simulation testing for collision direction requires takeover data related to vehicle collisions, while algorithm simulation testing for congestion direction requires takeover data related to vehicle congestion. Therefore, when a user needs to obtain the takeover data required for a specific business operation, they can configure a target data matching model on the client side to filter the data required for that business. It should be noted that this target data matching model is the model obtained after configuring the filter items of the initial data matching model. The initial data matching model can be a model with default content configured for the filter items, or a model with empty configuration content. Furthermore, this content configuration process can include configuring the values ​​of the filter items, setting priorities and time information for the filter items, etc. It should be noted that configuring the values ​​of the filter items means configuring the content of the filter items. In practical applications, a candidate item can be selected from the various candidate items of the filter item as the value of that filter item.

[0051] Optionally, in one implementation, obtaining the target data matching model set for the target business may include:

[0052] Obtain the target data matching model set in the client for the target business;

[0053] The method of setting up the target data matching model in the client can include steps A1-A2:

[0054] A1, the target configuration interface is output in the client; the target configuration interface is used to configure the content of the filter items for the initial data matching model;

[0055] A2, based on the target configuration interface, obtain the content obtained by configuring the content for the filter item, and obtain the target data matching model.

[0056] In this implementation, the client can output a target configuration interface. This interface is used to configure the filter items of the initial data matching model. Specifically, the target configuration interface displays filter items for the initial data matching model that users can configure. The content of these filter items can be empty or have default values. In practical applications, the client can provide corresponding candidate content for each filter item, allowing the user to select candidate content as the value for that filter item. Essentially, by outputting the target configuration interface on the client, the user only needs to configure the target data matching model for the target business within this interface to trigger a data retrieval command for that target business. Upon detecting the data retrieval command, the client can then send a data retrieval request for the target business to the server.

[0057] Additionally, it's important to note that the initial data matching model's filtering options can include multiple options categorized by different granularities. This is understandable, as takeover data can be divided into scenario data, road test problem data, and scenario data can be further subdivided into main vehicle behavior data, road structure data, and main vehicle behavior data can be further subdivided into left-turn data, right-turn data, and so on. Therefore, the initial data matching model's filtering options can also be categorized according to different granularities. For example, the initial data matching model can be set with first-level, second-level, and third-level filtering options. This hierarchical filtering allows for multi-level classification when selecting from a large dataset, facilitating the selection process.

[0058] S103, Based on the filtering options of the target data matching model, perform data filtering processing on the takeover category data with labels; wherein, the content represented by the label belongs to the candidate content of the filtering option;

[0059] Understandably, when users set up a target data matching model for a specific business on the client side, they can select candidate content from the candidate content of the filter options as the value of that filter option. Since the content represented by the tags of takeover-type data belongs to the candidate content of that filter option, the candidate content configured for the filter option is the content represented by the tags of the takeover-type data. Therefore, the filter options of this target data matching model can be used to perform data filtering on takeover-type data with tags, that is, to filter out takeover-type data related to the value of that filter option, as the result of the data filtering process.

[0060] It should be noted that the tags for takeover data can include the reason for the takeover, the time of takeover, the location of takeover, and so on.

[0061] Optionally, in one implementation, the label of the takeover data represents the reason for the takeover of the data.

[0062] In this implementation, the reason for data takeover can be a situation where certain problems occur during vehicle operation, requiring manual intervention by personnel. For example, the reason for data takeover could be a traffic violation, a vehicle collision, etc. It is understood that different business needs have different requirements for takeover data, and takeover data has labels representing the reasons for takeover. The content represented by these labels serves as candidate content for filtering options. Therefore, when a user needs to obtain takeover data required by a specific business, they can configure the content representing the takeover reason of the required data as the value of the filtering option, thereby filtering out takeover data that matches the value of the filtering option from the tagged takeover data.

[0063] For example, if the target service is a simulation test service for collision direction, the value of the filter can be selected from the content represented by each tag as candidate content, which represents the data takeover reason of vehicle collision in the data to be taken over. In this way, the filter can be used to obtain the takeover data with the data takeover reason of vehicle collision from the massive takeover data, thereby obtaining the takeover data required for the simulation test service of the algorithm direction.

[0064] It should be noted that the labels for this type of takeover data can also represent the takeover time, takeover location, and so on. Additionally, for clarity, the method of filtering labeled takeover data using the target data matching model's filtering options is described below and will not be repeated here.

[0065] Optionally, in one implementation, the method of obtaining the tagged takeover class data may include steps B1-B2:

[0066] B1, retrieves the initial takeover class data from the specified data source;

[0067] In this implementation, the initial takeover data can include scene data collected by various functional modules in the vehicle when manual takeover is required. For example, the specified data source can be various functional modules of the vehicle, such as the map module and the perception module. In practical applications, map data and perception data of the geographical area where the vehicle is located before and after manual takeover can be obtained from the vehicle's map module, perception module, and other functional modules to obtain the initial takeover data.

[0068] B2. Based on the semantic features of the initial takeover data, the initial takeover data is labeled to obtain labeled takeover data.

[0069] In this implementation, the semantic features of the initial takeover data can be the behavior features of the main vehicle, the operational features of the obstacle vehicle, and so on. It is understood that after obtaining the initial takeover data, it can be labeled based on its semantic features, i.e., the takeover reason can be tagged, thus obtaining labeled takeover data. It should be noted that the initial takeover data can be labeled manually or automatically using existing scene recognition algorithms; both are reasonable. Furthermore, it is understood that since there can be multiple reasons for data takeover, each takeover data can have multiple tags.

[0070] S104, the processing result of the data filtering is fed back to the client.

[0071] After filtering the labeled takeover data in step S103, the processing result is obtained, which is the takeover data filtered from a large set of takeover data. In practical applications, this processing result can be used to construct a database of data required for the target business; or, the processing result can be fed back to the client so that the client can display the processing result, thereby allowing users to obtain the takeover data required for the target business through the client.

[0072] In the solution provided in this disclosure, since the values ​​of the filter items in the target data matching model are selected from the candidate content of the filter items, and the candidate content of the filter items is the content represented by the tags of the takeover data, the filter items can be used to filter out takeover data that matches the filter items from the tagged takeover data. Therefore, users only need to set up a target data matching model for the target business in the client, that is, to configure the content of the filter items in the initial data matching model, and then send a data acquisition request to the server. After receiving the data acquisition request, the server can filter data from the tagged takeover data according to the values ​​of the filter items in the target data matching model, thereby automatically retrieving the required takeover data from massive amounts of takeover data. It is evident that this solution improves data acquisition efficiency compared to the existing technology of manually retrieving the desired takeover data from massive amounts of takeover data.

[0073] Optionally, in another embodiment of this disclosure, the number of filtering items is multiple, and the content configuration process includes: configuring candidate content and configuring priorities;

[0074] Accordingly, in this embodiment, the data filtering process for the tagged takeover data based on the filtering items of the target data matching model in step S103 above may include:

[0075] Based on the priority of this filter, the takeover data whose labels match the filter of the target data matching model are selected.

[0076] In this implementation, there are multiple filter options. When configuring filter options on the client side, users can select candidate content from the candidate content of each filter option to serve as the value of that filter option, thus completing the candidate content configuration. Furthermore, the priority of these multiple filter options can be set. Based on the priority of the filter options, takeover data whose tags match the filter options of the target data matching model are filtered out; that is, takeover data whose content represented by the tags is the same as the value of the filter option is filtered out. It can be understood that takeover data can have multiple tags. When filtering from takeover data using multiple filter options, the tags of a single takeover data entry can match multiple filter options. Since there is a priority among these multiple filter options, the filter option with the highest priority is matched first, and that takeover data entry is used as the filter result corresponding to the highest priority filter option. Therefore, by setting the priority of multiple filter options, users can obtain the filter results corresponding to multiple filter options.

[0077] For example, in one specific implementation, filtering takeover data whose labels match the filter items of the target data matching model based on the priority of the filter item may include steps C1-C2:

[0078] C1: Select the filter that has not been used and has the highest priority from multiple filter options as the filter to be used;

[0079] C2, from the unfiltered tagged takeover data, filter the takeover data whose tags match the candidate content configured for the filter to be used, and return to execute the step of selecting the unused and highest priority filter from the multiple filters as the filter to be used.

[0080] In this implementation, firstly, based on the priority of the set filter items from high to low, the highest priority filter item that has not been used is selected from multiple filter items as the filter item to be used. Then, from the takeover data with tags, takeover data with tags that match the candidate content configured for the filter item to be used is selected. Next, filter items with priorities after the filter item to be used are determined as new filter items to be used, and the data filtering process continues from the unfiltered tagged takeover data. Thus, according to the priority from high to low, takeover data with tags that match the filter item to be used is filtered sequentially.

[0081] As can be seen, this solution can quickly filter out takeover data with the same tag and filter item values ​​from massive amounts of takeover data. It can also flexibly set multiple filter items and their priorities according to user needs, so that users can obtain the filter results corresponding to multiple filter items according to their priorities.

[0082] To better understand this solution, the method provided in this disclosure embodiment will be illustrated below with a specific example.

[0083] The existing management and usage patterns of autonomous driving scenario data neglect the semantic information of the scenario itself, such as the behavior information of the driver vehicle and obstacle vehicles. In other words, scenario data management is typically based on business product lines or testing objectives, failing to provide targeted and valuable scenario data for iterations in various algorithm directions. Users must repeatedly trace the semantics of the scenario based on the original information, which increases the cost of algorithm iteration. Furthermore, the lack of unified scenario semantic rules necessitates manual intervention in the scenario processing flow, along with secondary processing in post-processing workflows to obtain the scenario data required by users, increasing data acquisition time and labor costs.

[0084] Based on the above issues, this example, drawing on practical experience in managing autonomous driving scenario libraries, focuses on takeover data. It can quickly classify takeover data automatically according to the scenario semantic information of the data, and generate a set of takeover data with certain scenario semantics for subsequent targeted algorithm iteration testing. This improves the relevance and efficiency of algorithm verification and reduces the time spent on data post-processing labeling and cleaning.

[0085] This example, based on takeover data from a road test scenario, can be divided into two main technical implementations. The first part constructs a classification rule base based on scene semantics (corresponding to the labeled takeover data mentioned above) and a data matching model; the second part, based on the classification rule base and in conjunction with the data matching model, performs automated semantic matching and segmentation during scene transitions, forming a set of scenes with specific semantics, thus preparing data for testing and verification of the subsequent algorithm for more accurate use.

[0086] The classification rule base, through a deep understanding of the semantic features of the scene, such as the behavioral characteristics of the main vehicle and the operational characteristics of the obstacle vehicle, combined with the needs of different algorithm modules for takeover-type data, ultimately builds a complete set of classification rule bases. For example... Figure 2 As shown, the classification rule base covers various elements of the scenario, such as vehicle behavior, road structure, road topology, obstacle categories, obstacle behavior, traffic lights, etc. To facilitate different semantic scenarios for various users, the rules in the classification rule base are divided according to data coverage, including first-level labels, second-level labels, and third-level labels. Each level is further subdivided into different semantic labels, ultimately establishing a classification rule base that can fully cover the semantics of the scenario. Users can configure and generate data matching models according to their data requirements for different algorithm directions. These data matching models can consist of multiple labels with priorities set, and, in conjunction with an automatic processing pipeline, can automatically generate the datasets required for different algorithm directions according to the user's data needs.

[0087] like Figure 3 As shown, in practical applications, initial takeover data can be obtained from data sources such as road tests and operations. Then, semantic summarization processing is performed on the obtained initial takeover data, i.e., manual or automated annotation. Next, according to the user-constructed data matching model, data is filtered from the classification rule base to obtain a classification dataset (corresponding to the processing results above), which serves as input data for algorithm simulation testing. In other words, based on the classification rule base and the user-constructed data matching model, a funnel classification process based on takeover data labels is automatically implemented. This mainly includes three processes: scene semantic label annotation, simulation test task verification, and automated classification processing.

[0088] Semantic Tagging: Initial takeover category data is semantically tagged manually (corresponding to the tagging process described above), resulting in tagged takeover category data. Manual tagging minimizes errors from automatic scene semantic mining and improves the accuracy of scene tagging. By manually replaying scene information, semantic-level scene tags can be assigned based on the classification rule base, preparing for subsequent scene classification.

[0089] Simulation test task verification: Using labeled takeover data as input to the simulation test algorithm, a simulation test is conducted to further verify the simulation effect of the road test algorithm. Manual labelers can perform secondary labeling of the takeover data based on the verification results.

[0090] Automated classification processing: After the simulation test verification task, if the manually labeled results meet expectations, a verification pass mark will be added. This operation will trigger the subsequent scene automated classification process. Automatic matching, classification, and inclusion into the test set are performed according to the type and priority of semantic tags, and automatic regression of new algorithms under that semantic tag is also supported.

[0091] As can be seen, this solution, through automated processing, avoids the tedious data retrieval operations of users from massive amounts of takeover data. It can quickly customize takeover data for algorithm iteration according to user needs, thereby fully exploring the value of takeover data. Compared with traditional manual data retrieval methods, this solution has a higher degree of automation and more effective data processing, and can accurately and quickly construct takeover datasets required for simulation testing of different algorithm directions.

[0092] Corresponding to the embodiments of the above methods, this disclosure also provides a data processing system, such as... Figure 4 As shown, it includes server 410 and client 420;

[0093] The server 410 is configured to receive a data acquisition request for a target business sent by a client; in response to receiving the data acquisition request, obtain a target data matching model set for the target business; perform data filtering processing on the takeover-type data with tags based on the filtering items of the target data matching model; and feed back the processing result of the data filtering processing to the client; wherein, the target business is a business for using takeover-type data for data processing; and the content represented by the tags belongs to the candidate content of the filtering items;

[0094] The client 420 is used to receive the processing results fed back by the server.

[0095] For details on the specific implementation of the server and client functions included in the system, please refer to the relevant content in the above method embodiments, which will not be repeated here.

[0096] Corresponding to the embodiments of the above methods, this disclosure also provides a data processing apparatus applied to a server, such as... Figure 5 As shown, the device includes:

[0097] The receiving module 510 is used to receive a data acquisition request sent by the client for a target service; wherein the target service is a service for data processing using takeover-type data;

[0098] The acquisition module 520 is used to obtain the target data matching model set for the target business in response to receiving the data acquisition request;

[0099] The filtering module 530 is used to perform data filtering processing on the takeover data with tags based on the filtering items of the target data matching model; wherein the content represented by the tags belongs to the candidate content of the filtering items.

[0100] The feedback module 540 is used to feed back the processing results of the data filtering to the client.

[0101] Optionally, the tags of the takeover data represent the reason for the takeover of the data.

[0102] Optionally, obtaining the target data matching model set for the target business includes:

[0103] Obtain the target data matching model set in the client for the target service;

[0104] The target data matching model is configured in the client in the following ways:

[0105] The target configuration interface is output in the client; wherein, the target configuration interface is an interface for configuring the content of the filter items of the initial data matching model;

[0106] Based on the target configuration interface, the content obtained by performing content configuration processing on the filter items is obtained, and the target data matching model is obtained.

[0107] Optionally, the method for obtaining the tagged takeover data includes:

[0108] Retrieves the initial takeover class data from the specified data source;

[0109] Based on the semantic features of the initial takeover data, the initial takeover data is labeled to obtain labeled takeover data.

[0110] Optionally, the number of filter items is multiple, and the content configuration processing includes: configuring candidate content and configuring priorities; the filter module includes:

[0111] The filtering submodule is used to filter takeover data whose labels match the filtering items of the target data matching model based on the priority of the filtering items.

[0112] Optionally, the filtering submodule is specifically used for:

[0113] From the multiple filter options, select the filter option that has not been used and has the highest priority as the filter option to be used;

[0114] From the unfiltered tagged takeover data, filter the takeover data whose tags match the candidate content configured by the filter to be used, and return to the step of selecting the unused and highest priority filter from the multiple filter options as the filter to be used.

[0115] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0116] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0117] An electronic device provided in this disclosure may include:

[0118] At least one processor; and

[0119] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described data processing methods.

[0120] This disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described data processing methods.

[0121] The present disclosure provides a computer program product containing instructions that, when run on a computer, causes the computer to perform the steps of any of the data processing methods described in the above embodiments.

[0122] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0123] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded into random access memory (RAM) 603 from storage unit 608. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0124] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0131] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0132] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method, applied on a server side, the method comprising: Receive a data acquisition request sent by a client for a target service; wherein the target service is a service for data processing using managed data; In response to receiving the data acquisition request, a target data matching model set for the target business is obtained; wherein, the target data matching model is a model obtained after content configuration processing of the filtering items of the initial data matching model; the content configuration processing includes: configuration of candidate content and configuration of priority; Based on the filtering options of the target data matching model, data filtering processing is performed on the takeover category data with tags; wherein, the content represented by the tags belongs to the candidate content of the filtering options; The processing results of the data filtering are fed back to the client. The filtering process based on the target data matching model, which filters the labeled takeover data, includes: From the multiple filter options, select the filter option that has not been used and has the highest priority as the filter option to be used; From the unfiltered tagged takeover data, filter the takeover data whose tags match the candidate content configured by the filter to be used, and return to the step of selecting the unused and highest priority filter from the multiple filter options as the filter to be used.

2. The method according to claim 1, wherein, The labels for the takeover data represent the reasons for the takeover of the data.

3. The method according to claim 1, wherein, The step of obtaining the target data matching model set for the target business includes: Obtain the target data matching model set in the client for the target service; The target data matching model is configured in the client in the following ways: The target configuration interface is output in the client; wherein, the target configuration interface is an interface for configuring the content of the filter items of the initial data matching model; Based on the target configuration interface, the content obtained by performing content configuration processing on the filter items is obtained, and the target data matching model is obtained.

4. The method according to claim 1, wherein, The methods for obtaining the tagged takeover data include: Retrieves the initial takeover class data from the specified data source; Based on the semantic features of the initial takeover data, the initial takeover data is labeled to obtain labeled takeover data.

5. A data processing system, comprising a client and a server; The server is configured to receive a data acquisition request for a target business sent by a client; in response to receiving the data acquisition request, obtain a target data matching model set for the target business; and perform data filtering processing on the takeover data with tags based on the filtering items of the target data matching model. The processing results of the data filtering are fed back to the client; wherein, The target business is a business that uses takeover-type data for data processing; the content represented by the label belongs to the candidate content of the filter item; The target data matching model is the model obtained after processing the filtering items of the initial data matching model with content configuration; the content configuration process includes: configuring candidate content and configuring priorities; The client is used to receive the processing results fed back by the server; The filtering process based on the target data matching model, which filters the labeled takeover data, includes: From the multiple filter options, select the filter option that has not been used and has the highest priority as the filter option to be used; From the unfiltered tagged takeover data, filter the takeover data whose tags match the candidate content configured by the filter to be used, and return to the step of selecting the unused and highest priority filter from the multiple filter options as the filter to be used.

6. A data processing apparatus, applied to a server, the apparatus comprising: A receiving module is used to receive a data acquisition request sent by a client for a target service; wherein the target service is a service for data processing using takeover-type data; The acquisition module is used to respond to receiving the data acquisition request and obtain a target data matching model set for the target business; wherein, the target data matching model is a model obtained after content configuration processing of the filtering items of the initial data matching model; the content configuration processing includes: configuration of candidate content and configuration of priority; The filtering module is used to perform data filtering processing on the takeover-type data with tags based on the filtering items of the target data matching model; wherein the content represented by the tags belongs to the candidate content of the filtering items; The feedback module is used to send the processing results of the data filtering to the client. Specifically, the filtering module is used for: From the multiple filter options, select the filter option that has not been used and has the highest priority as the filter option to be used; From the unfiltered tagged takeover data, filter the takeover data whose tags match the candidate content configured by the filter to be used, and return to the step of selecting the unused and highest priority filter from the multiple filter options as the filter to be used.

7. The apparatus according to claim 6, wherein, The labels for the takeover data represent the reasons for the takeover of the data.

8. The apparatus according to claim 6, wherein, The step of obtaining the target data matching model set for the target business includes: Obtain the target data matching model set in the client for the target service; The target data matching model is configured in the client in the following ways: The target configuration interface is output in the client; wherein, the target configuration interface is an interface for configuring the content of the filter items of the initial data matching model; Based on the target configuration interface, the content obtained by performing content configuration processing on the filter items is obtained, and the target data matching model is obtained.

9. The apparatus according to claim 6, wherein, The methods for obtaining the tagged takeover data include: Retrieves the initial takeover class data from the specified data source; Based on the semantic features of the initial takeover data, the initial takeover data is labeled to obtain labeled takeover data.

10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-4.

Images