A data slicing method and related apparatus

Abstract

The application discloses a data segmentation method and related device, which is used in the data labeling scene to realize the segmentation of private data, reduce the private information in a single data slice, and reduce the risk of data privacy leakage. According to the target correlation relationship, the application segments the data to be labeled, so that the sub-data in each target data slice does not have a target correlation relationship. When the common disclosure of the sub-data with the target correlation relationship leads to the risk of privacy leakage, the data segmentation method of the application can reduce the risk of data privacy leakage during data labeling, effectively protecting data privacy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data labeling, in particular to a data segmentation method and related device. BACKGROUND

[0002] With the rapid development of deep learning technology in the field of artificial intelligence (AI), data labeling technology combined with deep learning has also developed greatly. Data labeling can refer to classifying unlabeled text, video, image and other data to obtain classification labels.

[0003] Because data labeling requires a lot of manpower, enterprises usually give the data to be labeled to a third party for labeling. This improves the labeling efficiency, but also increases the risk of privacy leakage. In the case of increasing public concern about personal privacy protection, the data labeling link should strengthen the control of the risk of privacy leakage.

[0004] In the prior art, the original data is segmented into multiple data slices of a preset size or a random size, so that each data slice contains only part of the data, thereby reducing the risk of privacy leakage; the multiple data slices are labeled, and then the complete labeling information is obtained by splicing the slice labels corresponding to the original data; and finally, the complete labeling information and the original data are associated.

[0005] However, the data slices obtained by using the existing segmentation method may still contain private information in part of the original data, thereby having a high risk of privacy leakage in the data labeling link. SUMMARY

[0006] The present application provides a data segmentation method and related device, which can more thoroughly separate the data with privacy relationships in the data to be labeled in the data labeling scenario, reduce the private information in each data slice obtained after segmenting the data to be labeled, and reduce the risk of privacy leakage in the data labeling link.

[0007] The first aspect of the present application provides a data segmentation method, which can be applied to a computer device, specifically to a terminal, a server or other computer devices with data processing capability. Hereinafter, the application in the terminal is taken as an example for description.

[0008] The data segmentation method is applied to a data labeling scenario, and specifically includes:

[0009] First, the terminal obtains data to be labeled, which includes multiple sub-data, each sub-data being used to describe an attribute of a target object, and at least two sub-data in the multiple sub-data having a target association relationship.

[0010] The target object refers to objective things that can be distinguished from each other.

[0011] When two or more sub-data describe a set of associated privacy attributes of the same target object, it can be considered that the two or more sub-data have a target association relationship. The associated privacy attribute is a preset attribute, which can be obtained by identifying the sub-data through a related deep learning model.

[0012] Then, according to the target association relationship, the terminal cuts the to-be-labeled data to obtain a plurality of target data slices.

[0013] Each target data slice contains part of the plurality of sub-data, and the sub-data contained in each target data slice do not have the target association relationship.

[0014] The terminal can determine the sub-data having the target association relationship through a deep learning model corresponding to the data type of the to-be-labeled data, and cut the to-be-labeled data through a plurality of cutting methods according to the sub-data having the target association relationship to obtain the plurality of target data slices.

[0015] Finally, the terminal sends the plurality of target data slices to the labeling side, so that the labeling side labels the plurality of target data slices.

[0016] The present application cuts the to-be-labeled data according to the target association relationship, so that the sub-data contained in each target data slice obtained do not have the target association relationship. When the common disclosure of the sub-data having the target association relationship leads to the risk of privacy leakage, the data cutting method of the present application can reduce the risk of data privacy leakage during data labeling, and effectively protect data privacy.

[0017] In a possible implementation, the target data slice includes a first data slice and a second data slice, and the first sub-data in the first data slice and the second sub-data in the second data slice have the target association relationship.

[0018] In a possible implementation, the labeling side includes a plurality of labeling devices; and sending the plurality of target data slices to the labeling side includes sending the first data slice and the second data slice to different labeling devices, respectively.

[0019] The labeling device is a single computer device that can be independently operated by a single labeling personnel.

[0020] In the present application, the terminal sends the first data slice and the second data slice to different labeling devices, respectively, so that the sub-data in the target data slice received by the same labeling device do not have the target association relationship, thereby effectively reducing the risk of privacy leakage.

[0021] In a possible implementation, the sending of the plurality of target data slices to the labeling side comprises: sending the first data slice to the labeling side at a first time point; and sending the second data slice to the labeling side at a second time point, the first time point and the second time point being different time points.

[0022] In the present application, the terminal sends the first data slice and the second data slice at different time points, so that the target association relationship between the first sub-data and the second sub-data is more concealed, and the risk of privacy leakage is reduced.

[0023] In a possible implementation, the time interval between the first time point and the second time point is greater than a preset time threshold.

[0024] In the present application, by setting the time threshold, the time interval between the receiving of the first data slice and the second data slice by the labeling side is long enough, and the risk of privacy leakage is further reduced.

[0025] In a possible implementation, before the splitting of the to-be-labeled data, the method further comprises: obtaining, by a target recognition model, information of a preset category corresponding to each sub-data in the to-be-labeled data; and after the splitting of the to-be-labeled data to obtain the plurality of target data slices, the method further comprises: sending auxiliary labeling information of the target data slice to the labeling side, so that the labeling side labels the target data slice based on the auxiliary labeling information of the target data slice, the auxiliary labeling information comprising information of the preset category corresponding to each sub-data in the target data slice.

[0026] In the present application, the terminal can determine the type of the target recognition model and the specific information category of the information of the preset category according to the type of the to-be-labeled data and the requirement of data labeling.

[0027] In the present application, after obtaining the plurality of target data slices, the terminal can obtain information of the preset category corresponding to the plurality of target data slices, and take the information of the preset category corresponding to each target data slice as the auxiliary labeling information of the target data slice.

[0028] In the present application, the terminal can identify the sub-data in the to-be-labeled data by the target recognition model, and obtain the information of the preset category corresponding to the sub-data.

[0029] In the present application, the information of the preset category corresponding to the sub-data in the target data slice is sent to the labeling side as auxiliary labeling information, which helps the labeling personnel to understand the information of the preset category of the to-be-labeled data, so that the labeling side can label based on the information of the preset category when labeling the target data slice, and the labeling efficiency is effectively improved.

[0030] In a possible implementation, the preset type of information is natural language information or semantic information.

[0031] The application sends the natural language information or semantic information of the sub-data in the target data slice to the labeling side as auxiliary labeling information, so that the labeling side can label based on the corresponding natural language information or semantic information when labeling the target data slice, and the labeling efficiency is effectively improved.

[0032] In a possible implementation, the target data is segmented according to the target association relationship, including: the terminal determines, from the target data, sub-data that exist in the target association relationship by using a target recognition model; and the target data is segmented according to the sub-data that exist in the target association relationship.

[0033] In a possible implementation, the target data is segmented according to the target association relationship to obtain a plurality of target data slices, including: target regions in which each sub-data exists in the target data are identified, the sub-data being complete data that describes an attribute of a target object; and the target data is segmented according to target regions in which the sub-data that exist in the target association relationship exist, to obtain a plurality of target data slices, each target data slice including one or more complete sub-data.

[0034] The region in which the sub-data exists is a region occupied by the sub-data.

[0035] After the sub-data that exist in the target association relationship are determined, the terminal can determine target regions in which the sub-data that exist in the target association relationship exist, and segment the target data into a plurality of target data slices according to the target regions.

[0036] In the application, the target regions are acquired first, and then the target data is segmented according to the target regions in which the sub-data that exist in the target association relationship exist, so that one or more complete sub-data are included in the target data slices obtained by segmentation, the data integrity in the data slices is guaranteed, and the accuracy of data labeling is improved.

[0037] In a possible implementation, the plurality of target data slices include a third data slice and a fourth data slice; the third data slice includes first sub-data, and the fourth data slice includes second sub-data; and the target data is segmented according to the target association relationship, including: the target data is initially segmented to obtain an initial data slice; and the initial data slice is segmented to obtain the third data slice and the fourth data slice, based on the initial data slice including the first sub-data and the second sub-data, and the target association relationship existing between the first sub-data and the second sub-data.

[0038] In the present application, before the to-be-labeled data is divided according to the target association relationship, an initial division can be performed first to obtain initial data slices, and then the sub-data existing the target association relationship in the initial data slices is obtained, and finally the division based on the target association relationship is performed. Through the means of twice division, the terminal can identify the sub-data existing the target association relationship from the sub-data contained in each initial data slice, without identifying from all sub-data of the to-be-labeled data, thereby effectively reducing the operation amount of the terminal and improving the division efficiency.

[0039] In another possible implementation, before the initial division of the to-be-labeled data to obtain the initial data slices, the method comprises: obtaining target regions where each sub-data in the to-be-labeled data is located; and the initial division of the to-be-labeled data to obtain the initial data slices comprises: performing the initial division of the to-be-labeled data according to the target regions to obtain the initial data slices containing one or more complete sub-data; and the division of the initial data slices comprises: performing the division of the initial data slices according to the target region where the first sub-data is located and the target region where the second sub-data is located.

[0040] In a possible implementation, after the auxiliary labeling information of the target data slice is sent to the labeling side, the method further comprises: receiving a plurality of feedback messages returned by the labeling side, each feedback message being used for feeding back whether the auxiliary labeling information of a target data slice is correct; calculating an error parameter of the auxiliary labeling information according to the plurality of feedback messages; and outputting an alarm information if the error parameter meets an alarm condition.

[0041] In the present application, the error parameter of the auxiliary labeling information is calculated by using the feedback messages to determine whether the information of the preset category identified by the target recognition model is accurate enough, and the alarm information is outputted when it is detected that the information of the preset category is not accurate enough, so as to trigger manual access to adjust and retrain the target recognition model.

[0042] In a possible implementation, each feedback message is used for feeding back whether the auxiliary labeling information of a sub-data is correct.

[0043] In a possible implementation, the alarm condition is that a difference between an error ratio of the auxiliary labeling information and an expected error rate is greater than an alarm threshold; and the calculation of the error parameter of the auxiliary labeling information according to the plurality of feedback messages comprises: calculating a ratio of a number of feedback messages feeding back auxiliary labeling errors to a total number of the feedback messages to obtain the error ratio.

[0044] In the present application, the terminal can identify the semantic information of a certain number of sample data by using the target recognition model, and compare the identification result with the correct semantic information of the corresponding sample data to obtain the expected error rate; and the alarm threshold is a preset tolerable fluctuation value.

[0045] In another possible implementation, the alarm condition is that the number of feedback messages that assist in labeling errors reaches a preset statistical threshold.

[0046] In a possible implementation, the method further includes: receiving a split error message returned by the labeling side; and outputting alarm information.

[0047] The terminal may split data incorrectly when splitting data, so that the target data slice contains incomplete sub-data; the labeling side can return a split error message according to the target data slice with incomplete sub-data, and the terminal outputs alarm information according to the split error message.

[0048] In this application, the terminal outputs alarm information when receiving a split error message, so that the staff accesses to re-split the data to be labeled with split errors, thereby ensuring the accuracy of data labeling.

[0049] In a possible implementation, the sub-data includes key sub-data and non-key sub-data; before the multiple target data slices are sent to the labeling side, the method further includes: adding special marks to the key sub-data in the multiple target data slices, the special marks being used to distinguish the key sub-data from the non-key sub-data for the labeling personnel of the labeling side; and the sending of the multiple target data slices to the labeling side includes: sending the multiple target data slices with the special marks to the labeling side.

[0050] In this application, by adding special marks to the key sub-data in the target data slice, the labeling personnel can distinguish the key sub-data from the non-key sub-data when labeling the target data slice using a labeling device, thereby prompting the labeling personnel to label the key sub-data and the non-key sub-data according to flexible standards, so as to improve labeling quality.

[0051] The second aspect of the application provides a data splitting device, including:

[0052] An acquisition unit is configured to acquire data to be labeled, the data to be labeled including multiple sub-data, each sub-data being used to describe an attribute of a target object, and at least two sub-data in the multiple sub-data having a target association relationship;

[0053] A splitting unit is configured to split the data to be labeled according to the target association relationship, to obtain multiple target data slices; and

[0054] A sending unit is configured to send the multiple target data slices to a labeling side, so that the labeling side labels the multiple target data slices.

[0055] In a possible implementation, the target data slice includes a first data slice and a second data slice, and a first sub-data in the first data slice and a second sub-data in the second data slice have the target correlation.

[0056] In a possible implementation, the labeling side includes a plurality of labeling devices, and the sending unit is specifically configured to send the first data slice and the second data slice to different labeling devices.

[0057] In a possible implementation, the sending unit is specifically configured to send the first data slice to the labeling side at a first time point and send the second data slice to the labeling side at a second time point, and the first time point and the second time point are different time points.

[0058] In a possible implementation, a time interval between the first time point and the second time point is greater than a preset time threshold.

[0059] In a possible implementation, the obtaining unit is further configured to obtain, by using a target recognition model, information of a preset category corresponding to each sub-data in the data to be labeled, and the sending unit is further configured to send auxiliary labeling information of the target data slice to the labeling side, so that the labeling side labels the target data slice based on the auxiliary labeling information of the target data slice, and the auxiliary labeling information includes the information of the preset category corresponding to each sub-data in the target data slice.

[0060] In a possible implementation, the information of the preset category is natural language information or semantic information.

[0061] In a possible implementation, the splitting unit is specifically configured to identify a target region where each sub-data in the data to be labeled is located, each sub-data is complete data describing an attribute of a target object, split the data to be labeled according to the target region where the sub-data having the target correlation is located, and obtain the plurality of target data slices, each target data slice including one or more complete sub-data.

[0062] In a possible implementation, the plurality of target data slices include a third data slice and a fourth data slice, the third data slice includes a third sub-data, and the fourth data slice includes a fourth sub-data, the splitting unit is further configured to perform initial splitting on the data to be labeled to obtain an initial data slice, and perform splitting on the initial data slice to obtain the third data slice and the fourth data slice based on the initial data slice including the third sub-data and the fourth sub-data and the target correlation existing between the third sub-data and the fourth sub-data.

[0063] In a possible implementation, the apparatus further includes: a receiving unit configured to receive a plurality of feedback messages returned by the labeling side, each of the feedback messages being used to feed back whether the auxiliary labeling information of one of the target data slices is correct; a calculating unit configured to calculate an error parameter of the auxiliary labeling information according to the plurality of feedback messages; and an alarming unit configured to output an alarm information when the error parameter calculated by the calculating unit meets an alarm condition.

[0064] In a possible implementation, the alarm condition is that a difference between the error proportion of the auxiliary labeling information and an expected error rate is greater than an alarm threshold; and the calculating unit is specifically configured to: calculate a ratio of a number of feedback messages that feed back the auxiliary labeling errors to a total number of the feedback messages to obtain the error proportion.

[0065] The third aspect of the present application provides a computer device, comprising:

[0066] a processor and a memory;

[0067] The memory stores instructions or codes;

[0068] The processor is configured to communicate with the memory and execute the instructions or codes in the memory to perform the method of the first aspect.

[0069] The fourth aspect of the present application provides a computer readable storage medium, which includes instructions, when the instructions are run on a computer, cause the computer to perform the method of the first aspect.

[0070] The fifth aspect of the present application provides a computer program product, which includes computer readable instructions, when the computer readable instructions are run on a computer device, cause the computer device to perform the method of the first aspect.

[0071] The sixth aspect of the present application provides a chip system, which includes at least one processor and an interface, the interface is used to receive data and / or signals, and the at least one processor is used to support the computer device to realize the functions involved in the first aspect or any possible implementation manner of the first aspect. In a possible design, the chip system can further include a memory, the memory is used to save necessary program instructions and data of the computer device. The chip system can be composed of a chip, or can include the chip and other discrete devices.

[0072] The solutions provided by the second aspect to the sixth aspect are used to realize or cooperate to realize the method provided by the first aspect, and thus can achieve the same or corresponding beneficial effects as the first aspect. Here, no longer be described in detail. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 A system architecture diagram of a data segmentation method provided for an embodiment of the present application is shown in FIG. 1.

[0074] Figure 2 A flow diagram of a data segmentation method provided for an embodiment of the present application is shown in FIG. 2.

[0075] Figure 3 A diagram of to-be-labeled data provided for an embodiment of the present application is shown in FIG. 3.

[0076] Figure 4 A diagram of a target data slice provided for an embodiment of the present application is shown in FIG. 4.

[0077] Figure 5 A flow diagram of another data segmentation method provided for an embodiment of the present application is shown in FIG. 5.

[0078] Figure 6 A diagram of an initial data slice provided for an embodiment of the present application is shown in FIG. 6.

[0079] Figure 7 A diagram of another initial data slice provided for an embodiment of the present application is shown in FIG. 7.

[0080] Figure 8 A flow diagram of dynamically evaluating slice information quality provided for an embodiment of the present application is shown in FIG. 8.

[0081] Figure 9 A structural diagram of a data segmentation apparatus provided for an embodiment of the present application is shown in FIG. 9.

[0082] Figure 10 A structural diagram of a computer device provided for an embodiment of the present application is shown in FIG. 10.

[0083] Figure 11 A structural diagram of another computer device provided for an embodiment of the present application is shown in FIG. 11. DETAILED DESCRIPTION

[0084] In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application are described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Those skilled in the art can know that, as new application scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0085] The terms "first", "second", and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and embodiments of the application are capable of functioning in other sequences than those explicitly described or illustrated herein. Furthermore, the terms "comprise", "have" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, article, or apparatus that comprises a list of steps or units can not necessarily be limited to those steps or units, but can include additional steps or units not expressly listed or inherent to such process, method, article, or apparatus. The naming or numbering of steps in the methods presented in the present application does not imply a mandatory order of execution, and the execution order of the steps named or numbered can be changed according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of units appearing in the present application is a logical division, and in actual application, it can have another division manner, for example, a plurality of units can be combined or integrated into another system, or some features can be ignored or not executed, in addition, the coupling or direct coupling or communication connection between the units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between the units can be electrical or other similar forms, which are not limited in the present application. In addition, the units or sub-units described as separate components can or can not be physically separated, and can or can not be physical units, or can be distributed into a plurality of circuit units, and part or all of the units can be selected according to actual needs to achieve the purpose of the present application.

[0086] Artificial intelligence (AI) is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0087] In any application of artificial intelligence, a neural network model, such as a deep neural network (DNN) model or a convolutional neuron network (CNN) model, is involved. An initial neural network model is trained using sample data of different domains or business scenarios, and a target neural network model applicable to the business scenario is obtained. The sample data can be image data, voice data, or text data, and the type of the sample data is determined according to the domain to which the neural network model is applicable. For example, when the neural network model is used in the field of automatic driving, the sample data can be various image data in a traffic scene, such as images of buildings around an automatic driving vehicle, images of pedestrians, images of surrounding vehicles, images of ground markings, and images of traffic lights. When the neural network model is used in intelligent security or smart city, the sample data can be various image data of a city, such as images of blocks in the city. When the neural network model is used in other business scenarios, the sample data is image, audio, or text data of the corresponding business scenario.

[0088] In order to endow or teach computers with human judgment and understanding, and enable computers to have recognition ability similar to humans, the sample data used for training the neural network model needs to be annotated, so as to improve the understanding and cognition of computers on image, audio, or text data.

[0089] It can be understood that training a neural network model requires a large amount of data, and accordingly, the large amount of data needs to be annotated. In the data annotation process, each annotator responsible for annotation work can access a large amount of data.

[0090] To protect the security of private information as much as possible in the annotation process, an embodiment of the present application provides a data splitting method and related apparatus, which can more thoroughly separate data with private relationships in the data to be annotated in the scenario of data annotation, reduce private information in each data slice obtained after splitting the data to be annotated, and reduce the risk of private information leakage in the data annotation link.

[0091] The data splitting process is part of the training of the neural network model, and can be performed in the system architecture 100 shown in FIG. 1. Figure 1

[0092] Referring to FIG. 1, the system architecture 100 includes a data splitting apparatus 110, a data annotation apparatus 120, and a neural network model training apparatus 130. Figure 1 ​The embodiment of the present application provides a system architecture 100. The data acquisition device 150 is used for acquiring labeled sample data for neural network model training and storing into the database 140. The sample data can be understood by referring to the introduction of the sample data in the previous paragraph, and the description is not repeated here. The training device 130 generates a target neural network model / rule 101 based on the labeled sample data maintained in the database 140. How the training device 130 obtains the target neural network model / rule 101 based on the labeled sample data will be described in detail below. The target neural network model / rule 101 can identify to-be-labeled data or segment to-be-labeled data, for example.

[0093] The data acquisition device 150 is also used for acquiring unlabeled sample data and storing into the database 140. The unlabeled sample data can be labeled by the segmentation device 110 and the labeling device 120, and therefore, the unlabeled sample data can also be referred to as to-be-labeled data.

[0094] The target neural network model / rule 101 obtained by the training device 130 can be applied in different systems or devices. Figure 1 In the embodiment, the segmentation device 110 is configured with an I / O interface 112. The segmentation device 110 can send the target data slice obtained by segmenting the to-be-labeled data by the computing module 111 to the labeling device 120 through the I / O interface 212. The labeling device 120 can also return the labeling information obtained by labeling the target data slice to the segmentation device through the I / O interface 212.

[0095] The "labeling personnel" can label data through the labeling device 120.

[0096] The segmentation device 110 can call data, codes and the like in the data storage system 160, or store data, instructions and the like into the data storage system 160.

[0097] The segmentation device 110 associates the labeling information returned by the labeling device 120 and the to-be-labeled data by the computing module 111, obtains the labeled sample data, and sends the labeled sample data to the database 140.

[0098] More deeply, the training device 130 can generate corresponding target neural network models / rules 101 based on sample data in different labeling scenarios for different targets, so as to provide better results for users.

[0099] It is worth noting that, Figure 1 The embodiment of the present application provides a system architecture, Figure 1 The positional relationship between the devices, apparatuses, modules and the like shown in the embodiment does not constitute any limitation. For example, in the embodiment, Figure 1In some embodiments, the data storage system 160 is an external storage relative to the slicing device 110, and in other embodiments, the data storage system 160 can be placed in the slicing device 110.

[0100] The system architecture 100 can be deployed on a computer device such as a server, a virtual machine, and a terminal device. For example, the terminal device can be a mobile phone, a pad, a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc.

[0101] In Figure 1 Based on the system architecture shown, the present application further provides a data slicing method. Please refer to Figure 2 , Figure 2 for a flowchart of the data slicing method in one embodiment. As Figure 2 shown, the data slicing method includes the following steps 201-203.

[0102] Step 201, the slicing device acquires to-be-labeled data.

[0103] The to-be-labeled data includes a plurality of sub-data, each sub-data is complete data describing an attribute of a target object, and there is a target association relationship between at least two sub-data in the plurality of sub-data.

[0104] In this embodiment, the sub-data with the target association relationship has a privacy leakage risk for the target object when belonging to the same data slice.

[0105] The to-be-labeled data can be text, image, or audio.

[0106] Whether there is the target association relationship between the sub-data is determined by the attribute described by the sub-data, and the attribute described by the sub-data with the target association relationship can be referred to as an associated privacy attribute. The associated privacy attribute is determined by a privacy scenario involved by the to-be-labeled data, and the specific associated privacy attribute can be manually inputted and set, or can be obtained and set through a network database.

[0107] As shown in Figure 3 FIG. 1 is a schematic diagram of a to-be-labeled data in a financial scenario, which is a financial text image containing data such as "Name: Zhang San" and "Phone: ZZZ". Figure 3

[0108] Among them, "Zhang San" and "ZZZ" are sub-data describing the "name" and "phone" attributes of the target object Zhang San, and "name" and "mobile phone" are sub-data describing the "key-value" attributes of the table material.

[0109] Further, for example, the preset associated privacy attributes include "name" and "phone", and because "Zhang San" describes the attribute "name" and "XXX" describes the attribute "mobile phone", "Zhang San" and "XXX" are sub-data that exist in the target association relationship.

[0110] It should be noted that plural in this document refers to two or more.

[0111] Among them, the segmentation device can obtain the to-be-labeled data from the database of the data acquisition device or other computer devices through wired or wireless means.

[0112] Step 202, the segmentation device segments the to-be-labeled data according to the target association relationship to obtain a plurality of data slices.

[0113] Among them, each target data slice contains part of the sub-data in the plurality of sub-data, and there is no target association relationship between the sub-data contained in each target data slice.

[0114] Among them, the segmentation device first determines the sub-data in the to-be-labeled data that exists in the target association relationship according to the target association relationship, and then segments the to-be-labeled data according to the sub-data that exists in the target association relationship.

[0115] In one possible implementation, the process of determining the sub-data in the to-be-labeled data that exists in the target association relationship can be: the segmentation device first uses a target recognition model to obtain information of a preset type corresponding to the to-be-labeled data, then extracts semantic information that exists in the target association relationship from the information of the preset type corresponding to the to-be-labeled data through a target extraction model, and finally determines the sub-data that exists in the target association relationship according to the semantic information that exists in the target association relationship.

[0116] Specifically, the information of the preset type can be natural language information or semantic information, or other information that can indicate the specific meaning of the data.

[0117] ​In the embodiments of the present application, the target recognition model for identifying the information of the preset category, the named entity recognition model for identifying the sub-data, and the target extraction model for identifying the association relationship between the sub-data mentioned in the present application are all deep learning models that have been trained, and the present application does not limit the way of obtaining the aforementioned trained deep learning models.

[0118] In another possible implementation, the area where each sub-data of the to-be-labeled data is located is fixed within a certain range, for example Figure 3 The financial text image shown is a structured text image, and even if the specific information in it changes, the area corresponding to the specific information in the image will still be in the preset area; in addition, the area where the sub-data is located can also be fixed for text data and audio data, for example, audio recorded in the form of questions and answers, and a fixed time is set for each question and answer session.

[0119] In this case, the segmentation device can determine the attributes described by each sub-data and the association relationship between the sub-data in advance; after obtaining the to-be-labeled data, the sub-data in the to-be-labeled data that exist in the target association relationship are directly determined according to the target association relationship.

[0120] After determining the sub-data in the to-be-labeled data that exist in the target association relationship, the segmentation device can segment the to-be-labeled data according to the area where the sub-data that exist in the target association relationship are located.

[0121] In a possible implementation, the segmentation device segments the to-be-labeled data according to the area boundary of the sub-data that exist in the target association relationship, to obtain a plurality of target data slices.

[0122] Please refer to Figure 3 and Figure 4 , Figure 4 for a schematic view of the plurality of target data slices obtained according to the to-be-labeled data in Figure 3 . Figure 3 The association privacy attributes that exist in the target association relationship in Figure 4 are “name and phone number”, “name and remittance amount”, and “remittance amount and transaction note”, therefore, the segmentation device segments according to the area boundary of the sub-data that describe these association privacy attributes, i.e., “Zhang San”, “ZZZ”, “USD 500,000.01”, and “education and training expenses”, to obtain the plurality of target data slices in .

[0123] In another possible implementation, the segmentation device can also cut according to the area where the sub-data that exist in the target association relationship are located, so that the area where each sub-data that exist in the target association relationship is located is cut out separately as a target data slice, and the to-be-labeled data after cutting is taken as a target data slice.

[0124] In addition, the target data slice can be destructively divided into a plurality of target data sub-slices, and the target data sub-slices are sent to the labeling side. In this way, the target data sub-slices are not complete, and the target data sub-slices are not complete, and the target data sub-slices are not complete.

[0125] Step 203, the cutting device sends the plurality of target data slices to the labeling side.

[0126] In order to further protect data privacy and avoid the labeling personnel of the labeling side from easily obtaining or associating the contact of the data in each target data slice, the application embodiment further provides a distribution scheme.

[0127] The labeling side can include a labeling device and a transfer device, and can be a terminal, a server or other computer device with data processing capability. Different labeling devices can be multiple different computer devices belonging to the same labeling service provider, or multiple different computer devices belonging to different labeling service providers.

[0128] It can be understood that when the cutting device sends the target data slice to the transfer device, the transfer device forwards the target data slice to the labeling device for data labeling.

[0129] In one possible implementation, the target data slice of the same data to be labeled includes a first data slice and a second data slice, and the first sub-data in the first data slice and the second sub-data in the second data slice have a target association relationship. In this case, the following distribution scheme can be executed.

[0130] In the first distribution scheme, the labeling side includes multiple labeling devices, and the cutting device sends the target data slice to the labeling devices. The cutting device can send the first data slice and the second data slice to different labeling devices.

[0131] Under this distribution scheme, the labeling personnel are difficult to determine the two labeling devices corresponding to the received first data slice and the received second data slice, and find the first data slice and the second data slice from the numerous data slices received from the two labeling devices. Therefore, the distribution scheme can effectively reduce the risk of privacy leakage in the data labeling link.

[0132] In the second distribution scheme, the cutting device sends the target data slice at different times. The cutting device can send the first data slice and the second data slice at a first time point and a second time point, and the time interval between the first time point and the second time point is greater than a preset time threshold.

[0133] Specifically, the time threshold can be 3 hours, 5 hours or 12 hours.

[0134] Under this distribution scheme, the annotator obtains the first sub-data and the second sub-data at two time points respectively, for example, obtains the first sub-data name "Zhang San" at 9 o'clock in the morning, and obtains the second sub-data mobile phone number "XXX" at 9 o'clock in the evening, and the annotator is difficult to perceive the association between the first sub-data and the second sub-data across a certain length of time. Therefore, the distribution scheme can effectively reduce the risk of privacy leakage in the data annotation link.

[0135] It can be understood that the above two distribution schemes can be combined and applied.

[0136] It can be understood that the method provided by the present application cuts the to-be-annotated data according to the sub-data that exists in the target association relationship, so that each data slice obtained does not contain sub-data that exists in the target association relationship, and then the annotator is difficult to identify or perceive the target association relationship between the sub-data, thereby achieving the purpose of protecting privacy. In accordance with this principle, the information sent by the cutting device to the annotation device does not contain information that can directly or indirectly indicate that the sub-data exists in the target association relationship.

[0137] The present application cuts the to-be-annotated data according to the target association relationship, so that each target data slice obtained does not contain sub-data that exists in the target association relationship; when the common disclosure of sub-data that exists in the target association relationship leads to the risk of privacy leakage, the data cutting method provided by the present application can reduce the risk of data privacy leakage during data annotation, and effectively protect data privacy.

[0138] In order to better understand the data cutting method provided by the embodiments of the present application, the data cutting method provided by the embodiments of the present application will be described in detail below based on the system architecture shown in Figure 1 , in combination with the overall data annotation and training process. For details, please refer to Figure 5 , Figure 5 The flowchart of another data cutting method provided by the embodiments of the present application.

[0139] 501. The cutting device obtains to-be-annotated data.

[0140] Step 501 in the present embodiment is similar to step 201 in the foregoing embodiment shown in Figure 2 , and details are not repeated here.

[0141] Similar to the foregoing embodiment shown in Figure 2 , in the present embodiment, the sub-data that exists in the target association relationship has a privacy leakage risk with respect to the target object when it belongs to the same data slice.

[0142] 502. The cutting device identifies a target region in the to-be-annotated data.

[0143] The target region refers to a region occupied by each sub-data in the data to be labeled, and each sub-data corresponds to a target region.

[0144] The sub-data is data that is focused on in the current data labeling scenario, that is, a part that is likely to be labeled in the data to be labeled. For example, text image data in a financial table image, facial feature image data in a person image, or audio data containing a human voice period.

[0145] It can be understood that the target region can be an image region composed of pixel coordinates in image data, or a text paragraph in text data or an audio paragraph in audio data.

[0146] The segmentation device can first obtain information of a preset category corresponding to all data in the data to be labeled, and then determine the sub-data in the data to be labeled through the information of the preset category.

[0147] Specifically, the segmentation device can first obtain natural language information or semantic information corresponding to all data in the data to be labeled through a target recognition model, and then determine data corresponding to semantic information describing the attributes of a target object, that is, sub-data, according to the natural language information or semantic information corresponding to all data. Finally, the target recognition model or a clustering algorithm is used to recognize the region occupied by the sub-data, that is, the target region.

[0148] The natural language information refers to spoken language and written language in daily life, and the semantic information refers to information with certain meaning that can eliminate data uncertainty.

[0149] For the case where the data to be labeled is text data, the sub-data is image data of text data, and the sub-data is voice data, the target recognition model can recognize all data in the data to be labeled, and "translate" or "convert" the all data into corresponding natural language information, so as to further extract semantic information and determine the sub-data. Of course, the target recognition model can also directly recognize the semantic information corresponding to all data in the data to be labeled.

[0150] It can be understood that the target recognition model can not involve the recognition of semantic information when recognizing natural language information.

[0151] For example, for a financial text image, the segmentation device can use a general text detection and text recognition model as a target recognition model, recognize the shape of the characters in the financial text image, obtain the natural language information corresponding to the text data in the financial text image, and obtain the target region corresponding to the text data.

[0152] For the image type data, the target recognition model identifies semantic information in the image and determines the corresponding sub-data.

[0153] In another possible implementation, the area where each sub-data of the data to be annotated is located is fixed within a certain range, and the segmentation device can directly obtain the preset target area in the data to be annotated.

[0154] 503. The segmentation device performs initial segmentation on the data to be annotated to obtain initial data slices.

[0155] The segmentation device can perform initial segmentation on the data to be annotated according to the target area, or can perform initial segmentation in other ways, for example, the prior art, which segments the data to be annotated into multiple data slices of random sizes.

[0156] As shown in FIG. 5A, Figure 6 As shown in FIG. 5B, Figure 6 is a schematic diagram of multiple initial data slices obtained by segmenting the data to be annotated using the prior art. The segmentation method of segmenting the data into multiple data slices of random sizes will have an overlapping area between the multiple data slices to ensure that the final annotation result can completely contain all the contents in the data.

[0157] As shown in FIG. 5A, Figure 7 As shown in FIG. 5B, Figure 6 is a schematic diagram of multiple initial data slices obtained by segmenting the data to be annotated according to the target area. The segmentation method according to the target area can make the sub-data in the data slice complete, which can on the one hand ensure the integrity of the data in the data slice and improve the annotation quality, and on the other hand enable the annotation side to not annotate the repeated data contents and improve the annotation efficiency.

[0158] It can be understood that steps 502 and 503 are not necessary steps, and the segmentation device can not perform steps 502 and 503 and directly perform step 504.

[0159] 504. The segmentation device determines the sub-data with the target association relationship in the initial data slice.

[0160] The method for determining the sub-data with the target association relationship in the initial data slice in step 504 in the embodiment is similar to the method for determining the sub-data with the target association relationship in the data to be annotated in step 202 in the embodiment shown in FIG. 2, and details are not described herein. Figure 2

[0161] ​Specifically, taking a financial text image as an example, the segmentation device can acquire natural language information corresponding to the text of the financial text image; when it is needed to confirm the sub-data in the initial data slice of the financial text image that has the target correlation relationship, the segmentation device can identify specific named entities, i.e., the sub-data, from the natural language information in the initial data slice through a named entity recognition model, and then determine the relationship between the sub-data in the initial data slice through a target extraction model. When it is determined through the relationship extraction model that the sub-data in the initial data slice has the target correlation relationship, the segmentation device can further determine the sub-data that has the target correlation relationship.

[0162] When steps 502 and 503 are not performed, the segmentation device will directly determine the sub-data in the to-be-labeled data that has the target correlation relationship.

[0163] 505. The segmentation device segments the initial data slice according to the sub-data that has the target correlation relationship to obtain a plurality of target data slices, and sends the target data slices to the labeling device.

[0164] In the embodiment, the target region corresponds to the region in which the sub-data in the above-mentioned Figure 2 In the embodiment, the method for segmenting the initial data slice according to the sub-data that has the target correlation relationship in step 505 is similar to the method for segmenting the to-be-labeled data according to the sub-data that has the target correlation relationship in step 202 in the above-mentioned Figure 2

[0165] Each sub-data is complete data that describes one attribute of a target object, for example, a sub-data describes a "date" attribute, and the sub-data should describe "year-month-day", and it is not complete to describe only "year" or "month". Correspondingly, the complete sub-data refers to data that completely describes one attribute of a target object.

[0166] The target region is a region occupied by one sub-data, i.e., each target region includes one complete sub-data; therefore, each target data slice obtained by segmenting the initial data slice according to the target region includes one or more complete sub-data.

[0167] When the initial data slice does not include the sub-data that has the target correlation relationship, the initial data slice is a target data slice.

[0168] When steps 502 and 503 are not performed, the segmentation device can directly segment the to-be-labeled data according to the sub-data that has the target correlation relationship.

[0169] ​In a possible implementation, the slicing device adds special marks to the key sub-data in the target data slice, and the special marks are used to distinguish the key sub-data from the non-key sub-data by the annotators on the annotation side.

[0170] The key sub-data is sub-data corresponding to a preset key attribute, and the acceptance standard of the key sub-data is higher. Therefore, the annotators will perform data annotation on the key sub-data in a higher standard after seeing the key sub-data marked with the special marks.

[0171] The key attribute refers to an annotation object that needs to be focused on in the current annotation task.

[0172] For example, the demand side is researching contact lenses, and the annotation object that needs to be focused on is an eye image in a face image. The attribute of the eye image can be set as the key attribute. When the target recognition model recognizes the sub-data corresponding to the key attribute, the terminal adds special marks to the sub-data.

[0173] After obtaining the target data slice, the slicing device sends the target data slice to the annotation device. The annotation device is equivalent to the annotation side in the foregoing Figure 2 The method for sending the target data slice in step 505 in this embodiment is similar to the method for sending the target data slice in step 202 in the foregoing Figure 2 embodiment, and details are not described herein again.

[0174] 506. The slicing device sends auxiliary annotation information of the target data slice to the annotation device.

[0175] After obtaining the plurality of target data slices, the slicing device can obtain the preset category information of the sub-data in each target data slice according to the preset category information of the sub-data obtained in step 502 or step 504, and use the preset category information as part of the auxiliary annotation information of the corresponding target data slice.

[0176] The auxiliary annotation information can further include position information of the sub-data, data attribute information of the sub-data, and other information that is helpful for the annotators to understand the sub-data.

[0177] It can be understood that step 506 can be performed before the action of sending the target data slice in step 505, or can be performed after the action of sending the target data slice.

[0178] It can be understood that step 506 is not a necessary step, and the annotation device can directly perform data annotation according to the target data slice in step 505.

[0179] It should be noted that the specific steps of the data segmentation method provided in the embodiments of the present application include steps 501 to 506, and steps 509 and 510. In order to facilitate the understanding of the data segmentation method provided in the embodiments of the present application, the other steps in the specific embodiments are described below.

[0180] 507、The labeling device labels data according to the target data slice, and returns the labeling information obtained by labeling to the segmentation device.

[0181] The labeling device can directly label data according to the target data slice, or label data according to the target data slice and the corresponding auxiliary labeling information.

[0182] The semantic information is important knowledge information in the data labeling link. Through the semantic information, the correspondence between the data itself and the knowledge concept can be established, and then in the model training link, the machine can better understand the correspondence between the data and the knowledge. Therefore, the labeling result of the data is closely related to the semantic information of the data, and in some cases, the labeling result of the data is the corresponding semantic information.

[0183] In addition to the semantic information, the labeling personnel can also directly understand the semantics in the natural language information in the auxiliary labeling information and perform corresponding data labeling.

[0184] Therefore, through the natural language information or the semantic information about the sub-data in the target data slice in the auxiliary labeling information, the labeling personnel can refer to the corresponding natural language information or semantic information for labeling when using the labeling side device to label the sub-data in the target data slice. Further, the natural language information or the semantic information can be directly used as the pre-labeling result of the corresponding target data slice, and the labeling side only needs to confirm or modify the pre-labeling result.

[0185] Specifically, in the case of labeling according to the target data slice and the corresponding auxiliary labeling information, the labeling personnel who controls the labeling device can refer to the natural language information or the semantic information in the auxiliary labeling information to label the corresponding target data slice, and can also directly use the semantic information as the pre-labeling information of the corresponding target data slice.

[0186] For example, when the pre-labeling information is consistent with the expected labeling information, the labeling personnel can confirm the semantic information as the labeling information through the labeling device; when the pre-labeling information is inconsistent with the expected labeling information, the labeling personnel can modify the natural language information or the semantic information through the labeling device, and use the modified natural language information or semantic information as the labeling information.

[0187] When the labeling personnel completes the labeling of a target data slice, the labeling device can return a feedback message to the splitting device, where the feedback message is used to feed back whether the auxiliary labeling information of the target data slice is correct.

[0188] Specifically, if the natural language information or semantic information in the auxiliary labeling information can correctly describe the data content in the corresponding target data slice, or the natural language information or semantic information can be directly used as the labeling result of the corresponding target data slice, the labeling side returns a feedback message about the correctness of the auxiliary labeling information; if the natural language information or semantic information makes an incorrect description of the data content in the corresponding target data slice, the labeling side returns a feedback message about the incorrectness of the auxiliary labeling information.

[0189] In a specific possible implementation, one feedback message is used to feed back whether the auxiliary labeling information of one sub-data in a target data slice is correct.

[0190] In another possible implementation, when the labeling personnel finds that a target data slice contains incomplete or unrecognizable data due to splitting, the labeling personnel can send a splitting error message to the splitting device.

[0191] 508、The splitting device obtains a training sample according to the labeling information and the corresponding to-be-labeled data, and sends the training sample to the training device.

[0192] The splitting device splices the labeling information returned by the labeling device and belonging to the same to-be-labeled data to obtain complete labeling information of the to-be-labeled data, and associates the complete labeling information with the corresponding to-be-labeled data to obtain the training sample.

[0193] 509、The splitting device calculates an error ratio of the slice information, and sends the error ratio of the key sub-data to the training device.

[0194] The terminal can calculate a plurality of error parameters about the auxiliary labeling information, where the plurality of error parameters correspond to a plurality of preset alarm conditions; when the calculated error parameter satisfies an alarm condition, it indicates that the natural language information or semantic information obtained by using the current target recognition model cannot meet the requirements of auxiliary labeling, and human intervention is needed to adjust and retrain the target recognition model.

[0195] In a possible implementation, after receiving the feedback message returned in step 507, the splitting device can perform real-time synchronous calculation with step 507, obtain an error ratio by calculating the ratio of the number of feedback messages that feed back incorrect auxiliary labeling information to the number of all feedback messages, and perform step 510 when the difference between the error ratio and an expected error rate is greater than a preset alarm threshold.

[0196] For details, please refer toFigure 8 , Figure 8 A flowchart of a process for dynamically evaluating slice information quality is provided for an embodiment of the present application. As shown in FIG. 5, after the error ratio R is calculated in real time, the slicing device further calculates the difference Δ between the error ratio R and the expected error rate E; when the difference Δ is greater than the alarm threshold T, the slicing device executes step 510 and outputs alarm information; when the difference Δ is less than the alarm threshold T, the calculation of the error ratio R is repeated. Figure 8

[0197] The expected error rate E is calculated before the error ratio R is calculated. The slicing device identifies the natural language information or semantic information of the labeled data samples in the test set through the aforementioned target recognition model, and calculates the expected error rate R by comparing with the corresponding correct natural language information or semantic information.

[0198] The slicing device can obtain the labeled samples from the database of the data collection device, and construct a test set from the labeled samples. The labeled information of the labeled samples is the corresponding correct natural language information or semantic information.

[0199] When the slice information feedback messages corresponding to the target data slices of the same to-be-labeled data are all received, the slicing device can calculate the error ratio of the key sub-data and send the error ratio of the key sub-data to the training device.

[0200] It can be understood that when the number of feedback messages that feedback auxiliary labeling errors is less than the triggering threshold, even if the error ratio is greater than the alarm threshold, no alarm information is output, so as to avoid frequent output of alarm information when the sample amount of slice information feedback messages of a certain to-be-labeled data is too small. The triggering threshold can be determined according to the number of target data slices of the to-be-labeled data.

[0201] Other error parameters, for example, the number of feedback messages that feedback auxiliary labeling errors is counted, and when the number parameter reaches a preset statistical threshold, the slicing device executes step 510.

[0202] It can be understood that step 509 is not necessary to execute step 510.

[0203] 510、The slicing device outputs alarm information.

[0204] When the difference Δ between the error ratio R and the expected error rate E is greater than the alarm threshold T, the slicing device outputs alarm information to alert that the recognition accuracy of the target recognition model is too high or too low, and human intervention is needed to adjust the target recognition model or the expected error rate E.

[0205] ​When receiving the split error message sent by the labeling device in step 507, the splitting device can output an alarm message to alert that the current splitting algorithm or splitting model is wrong and needs manual intervention to re-split the corresponding data to be labeled.

[0206] It can be understood that this step 510 is not necessary to perform step.

[0207] 511、The training device adjusts the loss function of the to-be-trained model according to the error ratio of the key sub-data.

[0208] The training device adjusts the loss function of the to-be-trained model according to the error ratio of the key sub-data obtained by the splitting device. When the error ratio of a certain key sub-data is low, the training device can reduce the corresponding weight in the loss function; otherwise, the weight is increased.

[0209] 512、According to the training sample, the to-be-trained model after adjusting the loss function is trained.

[0210] The training device uses the training sample sent by the splitting device to train the to-be-trained model after adjusting the loss function, so as to obtain a deep learning model more accurate in recognizing key sub-data.

[0211] The data splitting method in the embodiments of the present application is described above, and the data splitting device in the embodiments of the present application is described below. Please refer to Figure 9 An embodiment of the data splitting device 900 in the embodiments of the present application includes:

[0212] The acquisition unit 901 is configured to acquire to-be-labeled data, wherein the to-be-labeled data includes a plurality of sub-data, each of the sub-data is used to describe an attribute of a target object, and there is a target association relationship between at least two of the plurality of sub-data.

[0213] The splitting unit 902 is configured to split the to-be-labeled data according to the target association relationship to obtain a plurality of target data slices; and

[0214] The sending unit 903 is configured to send the plurality of target data slices to a labeling side, so that the labeling side performs data labeling on the plurality of target data slices.

[0215] In the embodiments of the present application, the to-be-labeled data acquired by the acquisition unit 901 is split by the splitting unit 902 according to the target association relationship, so that the sub-data contained in each target data slice sent by the sending unit 903 to the labeling side does not have the target association relationship, thereby effectively protecting data privacy.

[0216] In a possible implementation, the target data slice includes a first data slice and a second data slice, and a first sub-data in the first data slice and a second sub-data in the second data slice have the target correlation.

[0217] In a possible implementation, the labeling side includes a plurality of labeling devices, and the sending unit 903 is specifically configured to send the first data slice and the second data slice to different labeling devices.

[0218] In a possible implementation, the sending unit 903 is specifically configured to send the first data slice to the labeling side at a first time point and send the second data slice to the labeling side at a second time point, and the first time point and the second time point are different time points.

[0219] In a possible implementation, a time interval between the first time point and the second time point is greater than a preset time threshold.

[0220] In a possible implementation, the obtaining unit 901 is further configured to obtain, by using a target recognition model, information of a preset category corresponding to each sub-data in the data to be labeled, and the sending unit 903 is further configured to send auxiliary labeling information of the target data slice to the labeling side, so that the labeling side labels the target data slice based on the auxiliary labeling information of the target data slice, and the auxiliary labeling information includes the information of the preset category corresponding to each sub-data in the target data slice.

[0221] In a possible implementation, the information of the preset category is natural language information or semantic information.

[0222] In a possible implementation, the splitting unit 902 is specifically configured to identify a target region where each sub-data in the data to be labeled is located, and split the data to be labeled according to the target region where the sub-data having the target correlation is located, to obtain the plurality of target data slices, and each target data slice includes one or more complete sub-data.

[0223] In a possible implementation, the plurality of target data slices include a third data slice and a fourth data slice, the third data slice includes a third sub-data, and the fourth data slice includes a fourth sub-data, and the splitting unit 902 is further configured to perform initial splitting on the data to be labeled to obtain an initial data slice, and perform splitting on the initial data slice to obtain the third data slice and the fourth data slice based on the initial data slice including the third sub-data and the fourth sub-data and the third sub-data and the fourth sub-data having the target correlation.

[0224] In one possible implementation, the device further includes: a receiving unit 904, configured to receive multiple feedback messages returned by the annotation side, each feedback message being used to provide feedback on whether the auxiliary annotation information of a target data slice is correct; a calculation unit 905, configured to calculate error parameters of the auxiliary annotation information based on the multiple feedback messages; and an alarm unit 906, configured to output alarm information when the error parameters calculated by the calculation unit meet the alarm conditions.

[0225] In one possible implementation, the alarm condition is that the difference between the error ratio of the auxiliary annotation information and the expected error rate is greater than an alarm threshold; the calculation unit 905 is specifically used to: calculate the ratio of the number of feedback messages that report the auxiliary annotation errors to the total number of feedback messages, to obtain the error ratio.

[0226] The data segmentation device 900 provided in this application embodiment can be understood by referring to the relevant content in the foregoing data segmentation method embodiment section, and will not be repeated here.

[0227] like Figure 10 The diagram shown illustrates a possible logical structure of a computer device 1000 provided in an embodiment of this application. The computer device 1000 includes a processor 1001, a communication interface 1002, a memory 1003, and a bus 1004. The processor 1001 may include a CPU, or at least one of a CPU, a GPU, an NPU, and other types of processors. The processor 1001, the communication interface 1002, and the memory 1003 are interconnected via the bus 1004. In embodiments of this application, the processor 1001 is used to control and manage the operations of the computer device 1000; for example, the processor 1001 is used to execute... Figure 2 as well as Figure 5 The steps described herein and / or other processes used in the techniques described herein. Communication interface 1002 is used to support communication by computer device 1000. Memory 1003 is used to store program code and data of computer device 1000.

[0228] The processor 1001 can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic device, transistor logic, hardware components, or any combination thereof, which can implement or execute the various exemplary logical blocks, modules, and circuits described in connection with the disclosure. The processor can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The bus 1004 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 10 only one bus or one type of bus is represented by a thick line, but this does not indicate that there is only one bus or only one type of bus.

[0229] As shown in Figure 11 , the computer device 1100 provided by the embodiments of the present application can have a possible logical structure diagram. The computer device 1100 includes a hardware layer 1101 and a virtual machine (VM) layer 1102, and the VM layer can include one or more VMs. The hardware layer 1101 provides hardware resources for the VMs and supports the running of the VMs. The functions and processes of the VMs related to the present application can be understood by referring to the descriptions of the above Figure 2 or Figure 5 . The hardware layer 1101 includes hardware resources such as a processor, a communication interface, and a memory. The processor can include a CPU, or a combination of a CPU, a GPU, and an NPU.

[0230] In another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores computer execution instructions. When at least one processor of a device executes the computer execution instructions, the device executes the data splitting method described in the above Figure 2 or Figure 5 embodiments.

[0231] In another embodiment of the present application, a computer program product is also provided, and the computer program product includes computer execution instructions stored in a computer readable storage medium. At least one processor of a device can read the computer execution instructions from the computer readable storage medium, and the at least one processor executes the computer execution instructions to make the device execute the data splitting method described in the above Figure 2 or Figure 5 embodiments.

[0232] In another embodiment of the present application, a chip system is also provided, which includes at least one processor and an interface for receiving data and / or signals, and the at least one processor is configured to support the implementation of the above Figure 2 Or Figure 5 The data splitting method described in some embodiments. In a possible design, the chip system can further include a memory for storing necessary program instructions and data of the computer device. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0233] The above describes the embodiments of the present application in detail, and the steps in the method of the embodiments of the present application can be sequentially scheduled, combined or deleted according to actual needs; the modules in the device of the embodiments of the present application can be divided, combined or deleted according to actual needs.

[0234] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of each process does not mean the execution order, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0235] The term "and / or" herein is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents that the front and rear associated objects are in an "or" relationship.

[0236] It should be understood that in the embodiments of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that the determination of B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0237] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0238] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the division of the above-described device embodiments is only a logical function division, and there can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0239] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0240] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in a unit. The integrated unit can be implemented in a form of hardware, or in a form of software functional unit.

Claims

1. A data slicing method, characterized by, The method comprises: obtaining to-be-labeled data, the to-be-labeled data comprising a plurality of sub-data, each of the sub-data being used to describe an attribute of a target object, and a target correlation relationship existing between at least two of the sub-data; segmenting the to-be-labeled data according to the target correlation relationship to obtain a plurality of target data slices, wherein the target data slices comprise a first data slice and a second data slice, and a first sub-data in the first data slice and a second sub-data in the second data slice have the target correlation relationship; and sending the plurality of target data slices to a labeling side so that the labeling side performs data labeling on the plurality of target data slices; before the segmenting the to-be-labeled data, the method further comprises: obtaining, by a target recognition model, information of a preset category corresponding to each of the sub-data in the to-be-labeled data; after the segmenting the to-be-labeled data to obtain the plurality of target data slices, the method further comprises: sending auxiliary labeling information of the target data slices to the labeling side so that the labeling side labels the target data slices based on the auxiliary labeling information of the target data slices, the auxiliary labeling information comprising information of a preset category corresponding to each of the sub-data in the target data slices.

2. The method of claim 1, wherein, The labeling side comprises a plurality of labeling devices; and the sending the plurality of target data slices to the labeling side comprises: sending the first data slice and the second data slice to different labeling devices respectively.

3. The method of claim 1, wherein, The sending the plurality of target data slices to the labeling side comprises: sending the first data slice to the labeling side at a first time point; sending the second data slice to the labeling side at a second time point, the first time point and the second time point being different time points.

4. The method of claim 1, wherein, The information of the preset category is natural language information or semantic information.

5. The method according to any one of claims 1 to 4, characterized in that, The segmenting the to-be-labeled data according to the target correlation relationship to obtain the plurality of target data slices comprises: identifying target regions in which each of the sub-data in the to-be-labeled data is located, each of the sub-data being complete data describing an attribute of a target object; segmenting the to-be-labeled data according to target regions in which the sub-data having the target correlation relationship are located to obtain the plurality of target data slices, each of the target data slices comprising one or more complete sub-data.

6. The method according to any one of claims 1 to 4, characterized in that, The plurality of target data slices further comprise a third data slice and a fourth data slice; the third data slice comprises a third sub-data, and the fourth data slice comprises a fourth sub-data. The segmenting the to-be-labeled data according to the target correlation relationship to obtain the plurality of target data slices comprises: performing initial segmentation on the to-be-labeled data to obtain an initial data slice; based on the initial data slice comprising the third sub-data and the fourth sub-data and the target correlation relationship existing between the third sub-data and the fourth sub-data, segmenting the initial data slice to obtain the third data slice and the fourth data slice.

7. The method of claim 1, wherein, After the sending of the auxiliary labeling information of the target data slice to the labeling side, the method further comprises: receiving a plurality of feedback messages returned by the labeling side, each of the feedback messages being used for feeding back whether the auxiliary labeling information of one of the target data slices is correct; calculating an error parameter of the auxiliary labeling information according to the plurality of feedback messages; if the error parameter meets an alarm condition, outputting alarm information.

8. The method of claim 7, wherein, The alarm condition is that a difference between an error ratio of the auxiliary labeling information and an expected error rate is greater than an alarm threshold; and the calculating of the error parameter of the auxiliary labeling information according to the plurality of feedback messages comprises: calculating a ratio of a number of feedback messages that feed back errors of the auxiliary labeling to a total number of the feedback messages, to obtain the error ratio.

9. A data slicing apparatus, characterized by, comprises: an acquisition unit, configured to acquire to-be-labeled data, the to-be-labeled data comprising a plurality of sub-data, each of the sub-data being used for describing one attribute of a target object, and at least two of the sub-data having a target association relationship therebetween; a slicing unit, configured to slice the to-be-labeled data according to the target association relationship, to obtain a plurality of target data slices, wherein the target data slice comprises a first data slice and a second data slice, and a first sub-data in the first data slice and a second sub-data in the second data slice have the target association relationship therebetween; and a sending unit, configured to send the plurality of target data slices to a labeling side, so that the labeling side performs data labeling on the plurality of target data slices; the acquisition unit is further configured to: acquire, through a target recognition model, information of a preset category corresponding to each sub-data in the to-be-labeled data; the sending unit is further configured to: send auxiliary labeling information of the target data slice to the labeling side, so that the labeling side labels the target data slice based on the auxiliary labeling information of the target data slice, the auxiliary labeling information comprising information of the preset category corresponding to each sub-data in the target data slice.

10. The apparatus of claim 9, wherein, The labeling side comprises a plurality of labeling devices; and the sending unit is specifically configured to send the first data slice and the second data slice to different labeling devices respectively.

11. The apparatus of claim 9, wherein, The sending unit is specifically configured to: send the first data slice to the labeling side at a first time point; send the second data slice to the labeling side at a second time point, the first time point and the second time point being different time points.

12. The apparatus of claim 9, wherein, The information of the preset category is natural language information or semantic information.

13. The apparatus of any one of claims 9-12, wherein, The slicing unit is specifically configured to: identify a target region where each sub-data in the to-be-labeled data is located, each of the sub-data being complete data describing one attribute of a target object; slice the to-be-labeled data according to a target region where the sub-data having the target association relationship therebetween is located, to obtain the plurality of target data slices, each of the target data slices comprising one or more complete sub-data.

14. The apparatus of any one of claims 9-12, wherein, The plurality of target data slices further include a third data slice and a fourth data slice; the third data slice includes third sub-data, and the fourth data slice includes fourth sub-data; and the splitting unit is further configured to: perform initial splitting on the to-be-labeled data to obtain initial data slices; based on the initial data slices including the third sub-data and the fourth sub-data and the target association relationship existing between the third sub-data and the fourth sub-data, split the initial data slices to obtain the third data slice and the fourth data slice.

15. The apparatus of claim 9, wherein, The apparatus further includes: a receiving unit configured to receive a plurality of feedback messages returned by the labeling side, each of the feedback messages being used to feed back whether the auxiliary labeling information of one of the target data slices is correct; a computing unit configured to compute an error parameter of the auxiliary labeling information according to the plurality of feedback messages; an alarming unit configured to output an alarm information when the error parameter computed by the computing unit meets an alarm condition.

16. The apparatus of claim 15, wherein, The alarm condition is that a difference between an error ratio of the auxiliary labeling information and an expected error rate is greater than an alarm threshold; and the computing unit is specifically configured to: compute a ratio of a number of feedback messages that feed back the auxiliary labeling errors to a total number of the feedback messages to obtain the error ratio.

17. A computer device, comprising: comprise: a processor and a memory; the memory has stored therein instruction operations or codes; the processor is configured to communicate with the memory and execute the instruction operations or codes in the memory to perform the method in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the method in any one of claims 1 to 8.

19. A computer program product, characterised in that, comprise computer readable instructions that, when executed on a computer device, cause the computer device to perform the method in any one of claims 1 to 8.

Images