An intelligent labeling and cleaning method and system for AI training big data

Abstract

The application relates to the technical field of data labeling and cleaning, and provides a big data intelligent labeling and cleaning method and system for AI training, which comprises the following steps: synchronously collecting original samples to be processed, initial labeling labels and cursor staying tracks of labelers, performing space mapping on the positions of the staying segments in the tracks and the content areas of the corresponding samples in the original samples to be processed to generate an attention offset unit; meanwhile, the samples are split into multiple layers of content blocks according to spatial arrangement, and the content blocks are matched with label semantic ranges layer by layer to generate a semantic coverage gap unit. The surface compliance and the actually semantic implicit loss of the silent drift labeling samples are identified by comparing the directions of the two, and the labels are modified according to the label semantic boundary extended by the cursor staying area. After the samples that cannot be modified are removed, a cleaned data set is output. Therefore, the application breaks through the limitation of the prior art that only explicit labeling errors can be identified, can eliminate the hidden danger of implicit label semantic loss, and improves the AI training data quality and the model edge feature recognition capability.

Description

Technical Field

[0001] This invention relates to the field of data annotation and cleaning technology, and more specifically, to a method and system for intelligent annotation and cleaning of big data for AI training. Background Technology

[0002] The content in this section only provides background information related to this invention and may not constitute prior art.

[0003] With the large-scale application of artificial intelligence technology, the quality of training data labeling directly determines the generalization ability and recognition accuracy of AI models. Data labeling and cleaning have become core and fundamental steps in the AI ​​R&D process. Currently, the industry has established a relatively mature governance system for cleaning and quality inspection of labeled data. Quality control generally revolves around explicit labeling anomalies, covering typical scenarios such as incorrect labeling categories, missed labeling objects, duplicate sample removal, and labeling format verification. Data screening and correction are typically accomplished through methods such as cross-labeling verification, model pre-labeling verification, and rule-based sampling quality inspection.

[0004] However, current technologies consistently focus on the explicit correctness of the annotation results themselves, neglecting the implicit annotation quality issues caused by human factors during the annotation process. In actual annotation work, annotators are affected by unconscious physiological and state factors such as visual fatigue and distraction, which can lead to a deviation between the annotation operation and actual visual attention. This results in abnormal samples where the annotation results are compliant in form but have implicit missing semantic boundaries. These samples do not have obvious mislabeling or omission characteristics and cannot be identified and filtered through conventional quality inspection, cross-checking, and model validation methods. However, they continuously introduce implicit noise into the training data, causing the model to learn edge features and boundary regions with bias. Over the long term, this will significantly reduce the model's recognition upper limit and generalization stability.

[0005] Currently, there is no effective solution in the industry for identifying and correcting this type of implicit annotation drift problem. This kind of hidden data quality risk has become a bottleneck restricting the improvement of training data quality for high-precision AI models, and corresponding technical means are urgently needed to solve it. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for intelligent annotation and cleaning of large datasets for AI training, in order to improve the aforementioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows:

[0007] Firstly, this application provides a method for intelligent annotation and cleaning of big data for AI training, including:

[0008] The system collects the original set of samples to be processed, the corresponding set of initial labels, and the set of cursor trajectories during the annotator's work. The original set of samples to be processed is an uncleaned set of samples used for AI model training. The set of initial labels is generated by the annotator after completing the annotation of the original set of samples to be processed. The set of cursor trajectories records all cursor positions and durations of the annotator in each sample annotation interface.

[0009] The set of cursor dwell trajectories is split, and all cursor dwell segments within a single task are extracted with the annotation start and submission of a single sample as the boundary. The position coordinates of each dwell segment are spatially mapped to the content area of ​​the corresponding sample in the original sample set to be processed, and attention shift unit corresponding to each sample is generated.

[0010] For each sample in the original set of samples to be processed, the content is split into multiple layers of content blocks according to the spatial arrangement order of the content. Each layer of content blocks corresponds to the first level of content granularity. The multiple layers of content blocks are matched layer by layer with the semantic coverage of the corresponding initial label, and the content blocks not covered by the label semantics are marked to generate the semantic coverage gap unit corresponding to each sample.

[0011] The attention shift unit is compared with the semantic coverage gap unit of the corresponding sample. When the distribution of the attention shift unit and the semantic coverage gap unit meets the preset overlap condition, the corresponding sample is marked as a silent drift labeled sample. After traversing all samples, a set of silent drift labeled samples is obtained.

[0012] For each sample in the silent drift annotation sample set, based on the cursor dwell area recorded by the attention shift unit and the content block corresponding to the semantic coverage gap unit, the semantic coverage range of the initial annotation label is extended so that the semantic boundary of the label fits the boundary of the content block, thus completing the annotation correction; samples that still cannot fit the boundary of the content block after extension are removed from the original sample set to be processed, and the cleaned annotation sample set is output.

[0013] Furthermore, the position coordinates of each pause segment are spatially mapped to the content region of the corresponding sample in the original sample set to be processed. Specifically, this includes:

[0014] Establish a baseline coordinate system for a single sample content area. Delineate the content coordinate system with the corner of the content area as the origin. Convert the screen coordinates of the cursor hovering segment into relative coordinates in the coordinate system, so that all hovering positions correspond to specific points within the sample content area, thus completing the spatial mapping.

[0015] Furthermore, attention shift units are generated for each sample, specifically including:

[0016] Spatial clustering is performed on all cursor dwell segments within a single task. Based on the proximity of the dwell positions and the continuity of the dwell time, multiple attention clusters are divided. The region with the highest density of dwell points and the longest cumulative dwell time in each attention cluster is determined as the core distribution region of the attention cluster.

[0017] The core distribution area of ​​each attention cluster is spatially compared with the reference target area corresponding to the initial label in the corresponding sample. The reference target area is obtained through target semantic recognition or target image recognition. The deviation direction and coverage of the core distribution area relative to the reference target area are used as the orientation attribute of the offset unit, and the relative cumulative degree of dwell time within the cluster is used as the intensity attribute of the offset unit. The orientation attribute, intensity attribute and the corresponding core distribution area together constitute a complete attention offset unit.

[0018] Furthermore, semantic coverage gap units are generated for each sample, specifically including:

[0019] Following the spatial arrangement of each sample content, the content is split layer by layer according to the layout separation boundary between content elements, forming a content block hierarchy with multi-level granularity from coarse to fine.

[0020] The semantic coverage of the initial label corresponding to each sample is matched layer by layer with each content block in the content block hierarchy, and the content block covered by the semantic coverage is identified as the semantic coverage area.

[0021] At the same granularity level, continuous content blocks outside the semantic coverage area are marked as semantic gap blocks;

[0022] The distribution range and location of all semantic gap blocks are used as the semantic coverage gap unit corresponding to each sample.

[0023] Furthermore, the content elements are further divided layer by layer according to the layout boundaries, specifically including:

[0024] Identify the layout boundaries in the sample content. Layout boundaries include paragraph boundaries, line break boundaries, indentation boundaries, blank area boundaries, and separator boundaries.

[0025] Based on the preset boundary strength rules, the separation strength of each layout boundary is determined, from strongest to weakest: paragraph boundary, line break boundary, indentation boundary, blank area boundary, and separator boundary. The sample content is split by the paragraph boundary to obtain the top-level content block. Then, each top-level content block is split by the line break boundary to obtain the intermediate-level content block. Subsequently, the indentation boundary, blank area boundary, and separator boundary are used to split the content step by step towards finer granularity until the preset finest granularity is reached.

[0026] All content blocks at all levels are organized into a tree structure according to their inclusion relationship, forming a multi-level content block hierarchy with coarse to fine granularity.

[0027] Furthermore, the distribution of attention-shifting units and semantic coverage gap units satisfies a preset overlap condition, specifically including:

[0028] Extract the projection region of the core distribution region in the attention shift unit onto the sample content, and use it as the shift coverage region;

[0029] Spatially merge all semantic gap blocks recorded by the semantic coverage gap unit to obtain the gap distribution area;

[0030] When the maximum distance between the boundary of the offset coverage area and the boundary of the gap distribution area is less than the span of the finest granular content block in the content block hierarchy in the layout direction, and the spatial overlap ratio of the two reaches the preset overlap threshold, the preset overlap condition is confirmed to be met.

[0031] Furthermore, the semantic coverage of the initial annotation labels is extended, specifically including:

[0032] Starting from the original semantic boundaries of the initial annotation tags, content blocks are matched outward layer by layer along the distribution direction of semantic gap blocks;

[0033] Each time the extension reaches a new content block, it checks whether the cumulative cursor dwell time of the corresponding block meets the effective attention standard. If it does, the block is included in the semantic coverage of the tag, and the verification continues to the outermost block. If the current block does not meet the effective attention standard, the extension stops, and the boundary of the outermost content block that has been included is used as the corrected semantic boundary of the tag.

[0034] Furthermore, samples that still cannot fit the content block boundaries after extension include:

[0035] When the semantic boundary of the tag extends to the boundary of the finest content block, there are still semantic gap blocks that are not covered, and the sample is judged as not fitting.

[0036] Secondly, this application also provides a big data intelligent labeling and cleaning system for AI training, including:

[0037] The multi-source data acquisition module is used to acquire the original set of samples to be processed, the corresponding set of initial labels, and the set of cursor trajectories during the annotator's work. The original set of samples to be processed is an unwashed set of samples used for AI model training. The set of initial labels is generated by the annotator after completing the annotation of the original set of samples to be processed. The set of cursor trajectories records all cursor positions and durations of the annotator in each sample annotation interface.

[0038] The trajectory space mapping module is used to split the set of cursor dwell trajectories. Taking the annotation start and submission of a single sample as the boundary, it extracts all cursor dwell segments within a single task, and performs spatial mapping between the position coordinates of each dwell segment and the content area of ​​the corresponding sample in the original set of samples to be processed, generating attention offset units corresponding to each sample.

[0039] The semantic gap identification module is used to split each sample in the original sample set into multiple content blocks along the spatial arrangement of the content. Each content block corresponds to a first-level content granularity. The module matches the semantic coverage of the multiple content blocks with the semantic coverage of the corresponding initial label layer by layer, marks the content blocks not covered by the label semantics, and generates the semantic coverage gap unit corresponding to each sample.

[0040] The silent drift determination module is used to compare the orientation of the attention shift unit with the semantic coverage gap unit of the corresponding sample. When the distribution of the attention shift unit and the semantic coverage gap unit meets the preset overlap condition, the corresponding sample is marked as a silent drift labeled sample. After traversing all samples, a silent drift labeled sample set is obtained.

[0041] The annotation cleaning and correction module is used to extend the semantic coverage of the initial annotation labels for each sample in the silent drift annotation sample set, based on the cursor dwell area recorded by the attention shift unit and the content block corresponding to the semantic coverage gap unit, so that the semantic boundary of the label fits the boundary of the content block, thus completing the annotation correction; samples that still cannot fit the boundary of the content block after extension are removed from the original sample set to be processed, and the cleaned annotation sample set is output.

[0042] Thirdly, this application also provides an electronic device, including:

[0043] Memory, used to store computer programs;

[0044] A processor for implementing the method steps as described in the first aspect when executing the computer program.

[0045] The beneficial effects of this invention are as follows:

[0046] This invention uses the cursor's trajectory, reflecting the true attention distribution, as the core clue. It combines semantic hierarchy matching of sample content to locate implicit annotation biases. By simultaneously collecting the original sample to be processed, the initial annotation labels, and the cursor's trajectory during the annotation process, and using the annotation initiation and submission of a single sample as boundaries to split the cursor's position into segments, spatially mapping the position to the sample content area to generate attention shift units. Each sample is then split into multi-level content blocks according to its content spatial arrangement, and matched layer by layer with the semantic coverage of the initial label to mark semantic coverage gap units not covered by the label. Subsequently, by comparing the cursor's offset direction with the distribution direction of the semantic gap blocks, silent drift annotation samples that appear compliant but actually have implicit semantic deficiencies are identified. Finally, based on the actual attention area where the cursor rests and the corresponding semantic gap content block, the label semantic boundary is extended to complete the annotation correction. Samples that still cannot fit the content block boundary after extension are removed and a cleaned set of labeled samples is output. This can overcome the limitation of existing label cleaning technology, which can only identify explicit errors such as mislabeling and omissions. It can effectively capture the silent label drift problem caused by the coupling of unconscious attention shift of the labeler and semantic block gaps, eliminate the hidden label semantic missing risks in training data, significantly improve the labeling quality of AI training data, and thus enhance the model's ability to recognize edge features. Attached Figure Description

[0047] Figure 1 A flowchart illustrating a big data intelligent labeling and cleaning method for AI training provided by this invention;

[0048] Figure 2 This is a schematic diagram of the cursor rest point, core distribution area, and attention cluster in this invention;

[0049] Figure 3 This is a schematic diagram of the attention shift unit, core distribution region, and reference target region in this invention;

[0050] Figure 4 This is a schematic diagram of the attention shift unit, core distribution region, reference target region, and semantic coverage gap unit in this invention;

[0051] Figure 5 This is a schematic diagram of a big data intelligent labeling and cleaning system for AI training provided by the present invention;

[0052] Figure 6 This is a schematic diagram of an electronic device provided by the present invention.

[0053] In the diagram: 201, Multi-source data acquisition module; 202, Trajectory space mapping module; 203, Semantic gap recognition module; 204, Silent drift determination module; 205, Annotation cleaning and correction module; 301, Processor; 302, Memory. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0055] like Figure 1 As shown in the embodiment of the present invention, a big data intelligent labeling and cleaning method for AI training is proposed, comprising:

[0056] S101, collect the original set of samples to be processed, the corresponding initial label set, and the cursor trajectory set during the annotator's operation; the original set of samples to be processed is an uncleaned set of samples used for AI model training, the initial label set is generated by the annotator after completing the annotation of the original set of samples to be processed, and the cursor trajectory set records all cursor positions and durations of the annotator in each sample annotation interface.

[0057] Specifically, the method collects the original set of samples to be processed, the corresponding set of initial labels, and the set of cursor trajectories during the annotation process. The aim is to simultaneously collect three types of data: sample content, annotation results, and work behavior, thereby establishing a correlation mapping system of "content carrier - annotation results - work behavior". The principle is that the annotation operator's operation behavior will implicitly reflect the cognitive bias and attention fluctuation during the annotation process, which can serve as an independent circumstantial dimension for judging annotation quality. Moreover, it can uncover deep annotation risks without additional manual review, reducing the manpower cost of quality inspection in batch annotation scenarios.

[0058] Wherein, the original set of samples to be processed is the set of unwashed samples used for AI model training, and its expression is: , For the sample size, For the first The set of original samples to be processed is available. This set can be retrieved in batches through the business data source interface, covering text, image, or formatted document samples required for the corresponding AI task. All samples have not undergone annotation quality verification or dirty data removal, and retain their original collection state completely. Taking the AI ​​training scenario of contract information extraction as an example, the original sample set to be processed may contain 10,000 unfiltered scanned contract text documents. The samples contain various content elements such as body text, signatures, remarks, headers and footers, and have uncleaned features such as content redundancy and inconsistent formatting. Its purpose is to preserve the complete layout and content structure of the original samples, providing a unified benchmark for subsequent spatial coordinate mapping and semantic block splitting, ensuring that the cleaning process adapts to the original data form of real business, and improving the model training generality of the cleaned samples.

[0059] The initial label set is generated by the annotator after labeling the original sample set to be processed, and its expression is: , For the first The initial label corresponding to each sample The sample size is specified in the provided text. This set is generated by annotators who, through a standardized annotation platform, individually select target regions and input semantic tags for each sample. Each sample corresponds to at least one annotation tag, which fully records the target semantic range and category attributes identified by the annotator. Continuing with the contract scenario mentioned above, the initial annotation tags are the text region and category tag corresponding to the "contract amount" marked by the annotator. The principle is to use the manual annotation results as a benchmark for subsequent corrections, and to verify the accuracy and completeness of this benchmark through behavioral data, thereby anchoring the starting boundary of annotation corrections and ensuring that the correction process does not deviate from the core semantic direction of the original annotations.

[0060] The cursor stop trajectory set records all cursor positions and durations observed by the annotator within each sample annotation interface; its expression is: , This represents the total number of global cursor rest points. For the stationary trajectory, the first The data collection is achieved using a built-in mouse event monitoring component on the annotation platform's front end. The collection begins with the loading of a single sample annotation interface and ends with the submission button being pressed, recording the cursor's screen coordinates and corresponding timestamps frame by frame. Continuing with the above scenario, when an annotator browses contract text, the coordinates and duration of the cursor's repeated stops in disputed clause areas are fully recorded. The principle is that the cursor's position corresponds to the annotator's visual attention focus, and the duration of the stop corresponds to the level of attention, indirectly reflecting the annotator's varying cognitive investment in different areas of the sample. Its beneficial effect is that it obtains implicit behavioral data during the annotation process, providing objective quantitative evidence for subsequent identification of annotation omissions and attention shifts.

[0061] It should be noted that the cursor in this invention includes a screen pointer cursor controlled by the annotator through mouse operation, and a gaze cursor generated on the annotation interface screen by real-time acquisition of the user's eye gaze point using eye-tracking technology (this technology is existing and will not be elaborated here). Preferably, the cursor generated on the screen using eye-tracking technology is preferred. Compared to the mouse pointer, which requires indirect hand operation to reflect the area of ​​attention, the cursor generated by eye-tracking directly corresponds to the annotator's actual gaze point, thus better matching the actual distribution of human visual attention. This effectively eliminates the deviation caused by the inconsistency between mouse operation and the gaze point, thereby improving the accuracy of subsequent attention clustering analysis and annotation offset determination. Furthermore, using mouse event monitoring requires the annotator to move the mouse sequentially according to their viewing position on the data (including images or text), ensuring that the mouse movement and the user's viewing order are consistent.

[0062] S102, the set of cursor dwell trajectories is split, and all cursor dwell segments within a single task are extracted with the annotation start and submission of a single sample as the boundary. The position coordinates of each dwell segment are spatially mapped to the content area of ​​the corresponding sample in the original sample set to be processed, and attention shift unit corresponding to each sample is generated.

[0063] Specifically, the set of cursor stop trajectories is split, and all cursor stop segments within a single task are extracted, with the start and submission of annotation for a single sample as the boundary. In practice, the annotation platform's front-end event listening mechanism writes a task start timestamp when the annotation interface for a single sample is rendered, and a task end timestamp when the annotator clicks the submit button. Based on these two timestamps, the globally continuous set of cursor stop trajectories is sliced, and the independent stop segment for each sample is stored according to its unique identifier. The expression is as follows:

[0064] (1)

[0065] In the formula, For the first The set of independent cursor-stop segments corresponding to each sample; For the first The timestamp of the start of the annotation of each sample; For the first The submission timestamp of the labeled sample; For the global number The timestamp of the cursor's position; This is the set of global cursor positions.

[0066] The principle behind this approach is that the original cursor trajectory is a continuous temporal data stream across samples. The annotation behaviors of different samples are sequentially connected on the timeline. Without task boundary splitting, subsequent spatial mapping and attention analysis will suffer from cross-sample data confusion. By using annotation initiation and submission as natural boundary splitting, behavioral data can be bound one-to-one with individual samples. This ensures a strict correspondence between the attention analysis object and the sample from the data source, eliminating analysis errors caused by cross-task data crosstalk and providing a precise single-task data foundation for subsequent spatial mapping.

[0067] The position coordinates of each pause segment are spatially mapped to the content region of the corresponding sample in the original sample set to generate an attention shift unit for each sample. The specific implementation of the spatial mapping is as follows:

[0068] A baseline coordinate system is established for each sample content area, with the corner of the content area as the origin. In practical implementation, the top-left corner of the sample content display area is selected as the origin. A two-dimensional relative coordinate system matching the size of the sample content area is constructed, with the horizontal axis pointing to the right and the vertical axis pointing downwards. The horizontal and vertical axes of the coordinate system are equal to the pixel width and height of the content area, respectively. The principle is that the original screen coordinates of the cursor are affected by display environment factors such as browser window size, page scroll distance, and scaling ratio, and cannot directly correspond to the specific content position inside the sample. Establishing an independent content relative coordinate system can isolate the interference of external display variables, thereby keeping the relative position of the cursor position and the sample content constant. This ensures that the trajectory data collected under different terminals and display ratios have a unified comparison benchmark, guaranteeing the consistency of subsequent spatial comparisons.

[0069] Then, the screen coordinates of the cursor's hovering segment are converted to relative coordinates in a coordinate system, so that all hovering positions correspond to specific points within the sample content area, completing the spatial mapping. Specifically, the absolute screen coordinates of the top-left corner of the current sample content area and the area's width and height parameters are first obtained through the front-end interface. The absolute screen coordinates of each cursor hovering point are then subtracted from the horizontal and vertical coordinates of the top-left corner of the content area, converting them into relative coordinates in the content coordinate system. Invalid cursor positions falling outside the content area boundaries are filtered out. The conversion formula is as follows:

[0070] (2)

[0071] (3)

[0072] In the formula, For the first Sample No. The screen coordinates of each stop point; For the first The screen coordinates of the top left corner of the sample content display area; No. Sample No. The relative coordinates of the cursor's rest point in the content coordinate system.

[0073] The effective point filtering condition is:

[0074] (4)

[0075] (5)

[0076] In the formula, For the first The pixel width of the sample content area; For the first The pixel height of the sample content area.

[0077] The principle is to convert absolute screen coordinates to relative content coordinates through coordinate translation, mapping the cursor's position on the screen to the position of the sample content. Subsequently, specific content elements such as text lines and image blocks within the sample can be directly matched based on the relative coordinates, without the need for additional adaptation to the display parameter differences of different devices, thus reducing the complexity of spatial matching.

[0078] After completing the spatial mapping, the attention shift unit generation stage begins:

[0079] First, spatial clustering is performed on all cursor dwell segments within a single task. Multiple distinct attention clusters are created based on the proximity of the dwell positions and the continuity of the dwell duration, such as... Figure 2 As shown. In specific implementation, density clustering logic is used. A spatial proximity threshold and a temporal continuity threshold are pre-set. All dwell points are traversed, and when the coordinate distance between two adjacent dwell points is less than the spatial proximity threshold and the timestamp interval is less than the temporal continuity threshold, the two points are grouped into the same cluster. The expression for the spatial distance between adjacent dwell points is as follows:

[0080] (6)

[0081] In the formula, For the first The and the first +1 spatial coordinate spacing between adjacent stopping points; For the first Sample No. The relative x-coordinate of each stop point; For the first Sample No. +1 relative x-coordinate of the stop point; For the first Sample No. The relative ordinates of each stop point; For the first Sample No. +1 relative ordinate of the stop point.

[0082] like and but and Belonging to the same attention cluster, among which, The preset spatial proximity threshold; For the first Sample No. Timestamps of each stop point; For the first Sample No. +1 timestamp of the stop point; The preset time continuity threshold; For the first Sample No. One stop; For the first Sample No. +1 stop; The sample number; This refers to the sequence number of the stay point within a single sample.

[0083] After traversal, multiple discrete attention clusters are obtained, each corresponding to a content area continuously focused on by the annotator. The principle behind this is that the annotator's visual attention does not uniformly cover the entire sample, but rather concentrates in blocks within specific content areas. Furthermore, genuine attention dwell is accompanied by continuous point recording within a short period. Through clustering constrained by both spatial proximity and temporal continuity, the distribution blocks of the annotator's attention can be accurately reconstructed. This effectively eliminates transient points passed through during cursor movement, retaining only the blocks representing genuine attention input, thus improving the accuracy of subsequent offset determination.

[0084] Secondly, the region with the highest density of dwell points and the longest cumulative dwell time in each attention cluster is identified as the core distribution region of the attention cluster, such as... Figure 2 As shown. In the specific calculation, within the coverage area of ​​each attention cluster, a fixed-size sliding window traverses all dwell points, simultaneously counting the number of dwell points and the total dwell time within each window. The window coverage area with the highest combined score of these two indicators is taken as the core distribution area of ​​that attention cluster. Its expression is as follows:

[0085] (7)

[0086] (8)

[0087] (9)

[0088] In the formula, The score is based on a combination of the two indicators; For sliding windows Density of dwell points within; For sliding windows The number of cursor rest points contained within; The width of the sliding window; The height of the sliding window; The maximum dwell point density of all sliding windows within a single cluster; For sliding windows Cumulative length of stay within; For the first Sample No. One cursor stop point; For the first Sample No. Duration of stay at each stop; This represents the maximum cumulative dwell time of all sliding windows within a single cluster.

[0089] The principle behind this is that there is a gradient difference in the level of attention within an attention cluster. The core region is where the annotator's cognitive investment is highest and their gaze lingers the longest, best reflecting their true focus. Using the core region to represent the attention position of the entire cluster can reduce the interference of scattered points at the cluster edge on the orientation determination, and improve the stability of subsequent offset comparison results.

[0090] Then, the core distribution area of ​​each attention cluster is spatially compared with the baseline target area corresponding to the initial labeled label in the corresponding sample. For example... Figure 3 As shown. In the specific comparison, the initial annotation labels corresponding to the target semantics or target image are first extracted using a pre-trained target semantic recognition model or target image recognition model (both of which can use existing technologies, which will not be elaborated here). The boundary coordinates of the target semantic region (or target image region) marked by the label are used as the reference target region. The center coordinate difference and the region overlap ratio between each attention cluster core region and the reference target region are calculated one by one to clarify the spatial positional relationship between the two.

[0091] The formula for the difference in center coordinates is as follows:

[0092] (10)

[0093] (11)

[0094] In the formula, For the first Sample No. The difference in the center of the abscissa between the core region of each cluster and the benchmark target region; For the first Sample No. The difference in the center of the vertical coordinate between the core region of each cluster and the benchmark target region; For the first Sample No. The left boundary of the core region of each cluster is relative to the x-coordinate; For the first Sample No. The right boundary of the core region of each cluster is relative to the x-coordinate; For the first The left boundary of the target region of the sample relative to the x-coordinate; For the first The right boundary relative to the x-coordinate of the target region of the sample; For the first Sample No. The upper boundary of the core region of each cluster is relative to the vertical coordinate; For the first Sample No. The lower boundary of the core region of each cluster is relative to the ordinate. For the first The upper boundary of the target area of ​​the sample is relative to the ordinate. For the first The lower boundary of the target area of ​​the sample is relative to the ordinate.

[0095] The formula for the percentage of overlapping regions is as follows:

[0096] (12)

[0097] In the formula, For the first Sample No. The percentage of overlap between the core region of each cluster and the benchmark target region; The overlapping area between the core area and the benchmark target area; The total area of ​​the baseline target region; The sample number; This is the cluster number.

[0098] The formula for the area of ​​the benchmark region is as follows:

[0099] (13)

[0100] The principle of spatial orientation comparison is to use the manually annotated target area as a reference and compare the positional difference between the actual point of attention and the annotated target area. This can identify whether the annotator's attention has deviated from the target area that should be annotated. Establishing a spatial correlation between work behavior data and annotation result data provides quantifiable orientational evidence for the implicit judgment of annotation quality.

[0101] Finally, the orientation and coverage of the core distribution area relative to the benchmark target area are used as the orientation attribute of the offset unit, and the relative cumulative duration of stay within the cluster is used as the intensity attribute. The orientation attribute, intensity attribute, and corresponding core distribution area together constitute a complete attention offset unit. It should be noted that the orientation and intensity attributes are used to characterize the fatigue level of the annotator; either attribute can be used for determination, or a weighted sum can be used for comprehensive determination. The attention offset unit is displayed on the screen as follows: Figure 3 As shown in 'a', the deviation direction is categorized into four directions: up, down, left, and right. The coverage area records the boundary difference between the core region and the baseline target region. The intensity attribute is the proportion of the total dwell time of this cluster to the total dwell time of a single sample; a higher proportion indicates a stronger degree of attention shift. The formula for the intensity attribute of the attention shift unit is as follows:

[0102] (14)

[0103] In the formula, For the first Sample No. The offset intensity attribute corresponding to each attention cluster; For the first Sample No. One cursor stop point; For the first The first sample A cluster of attention; For the first Sample No. Duration of stay at each stop; For the first The total dwell time of all cursor positions in the sample; The sample number; Cluster number; This refers to the sequence number of the stay point within a single sample.

[0104] The principle is to quantify the attention shift features in a structured way from two dimensions: orientation and intensity. This can fully describe the specific shape of the annotator's attention deviating from the annotation target, forming a standardized shift feature unit. This facilitates accurate orientation matching with semantic coverage gap units, effectively improving the recognition accuracy of silent drift annotation samples.

[0105] Taking the contract information extraction scenario as an example, when the annotator annotates a contract sample with the "contract amount" label, the original global cursor trajectory is divided into independent dwell segments according to the interface loading time and submission time of the sample (i.e., all cursor points in the dwell segment are invalid). After coordinate transformation, all valid cursor points correspond to the relative positions within the contract text area. Through clustering, three attention clusters are divided. The core distribution area of ​​one cluster is located directly below the benchmark amount annotation area, with the deviation direction being downward. The coverage area exceeds the height of two lines of text in the benchmark area. The total dwell time of this cluster accounts for 18% of the total dwell time of the sample. The above-mentioned position and intensity attributes together constitute the attention offset unit corresponding to the contract sample.

[0106] S103. For each sample in the original sample set to be processed, split it into multiple layers of content blocks according to the spatial arrangement of the content. Each layer of content blocks corresponds to the first level of content granularity. Match the multiple layers of content blocks with the semantic coverage of the corresponding initial label layer by layer, mark the content blocks that are not covered by the label semantics, and generate the semantic coverage gap unit corresponding to each sample.

[0107] Specifically, this step starts from the original structural dimension of the sample content and establishes a standardized content granularity comparison system to provide a unified physical benchmark for the identification of annotation coverage gaps, avoiding inaccurate gap judgment caused by subjective boundary differences in manual annotation; through multi-level granularity hierarchical matching, it can quickly locate the approximate location of the gap at the macro level and accurately lock the specific boundary of the gap at the micro level, taking into account both recognition efficiency and accuracy. Moreover, based on the content's own layout structure, it can be divided into blocks without relying on a deep semantic model to complete the structured decomposition, and can be adapted to various types of annotation samples such as text and formatted documents.

[0108] Specifically, generating semantic coverage gap units for each sample includes:

[0109] The first step involves splitting the content layer by layer along the spatial arrangement of each sample, based on the layout boundaries between content elements, forming a multi-level content block hierarchy with varying granularity from coarse to fine. Here, "along the spatial arrangement" means following the original reading layout order of the sample content from top to bottom and left to right, ensuring that the block splitting logic is consistent with the human visual browsing order and providing a unified spatial reference system for subsequent positional comparison with the cursor's attention trajectory. In implementation, the layout structure information of the sample content can be extracted using a layout analysis engine to identify different levels of layout separation features. The splitting operation is performed sequentially according to the separation strength from high to low. Each level of splitting generates a corresponding content block of one level of granularity, ultimately forming a multi-level granular hierarchy covering everything from entire paragraphs to short sentences. The principle behind this is that layout separation boundaries are natural physical markers for semantic segmentation of content. Higher separation strength indicates stronger semantic independence of the content on both sides. Layer-by-layer splitting based on this principle can accurately match the semantic hierarchy of the content, achieving content structuring and grading without complex semantic understanding algorithms. This low computational cost allows for rapid processing of large batches of samples.

[0110] This involves splitting content elements layer by layer based on the layout boundaries between them, specifically including:

[0111] First, the layout boundaries in the sample content are identified. These boundaries include paragraph boundaries, line break boundaries, indentation boundaries, white space boundaries, and separator boundaries. In practical implementation, for text samples, line breaks, paragraph separators, and indentation spaces can be identified through character parsing. For layout documents and image samples, page layout analysis algorithms can identify paragraph spacing, line spacing, white space, and separator lines, extracting the position coordinates of all five types of boundaries. These five types of boundaries cover all layout separation forms in regular documents, from macro to micro, fully supporting multi-level granularity splitting needs and adapting to document layout styles in most business scenarios. There is no need to customize splitting rules for specific formats, improving the method's versatility.

[0112] Secondly, based on preset boundary strength rules, the separation strength of each layout boundary is determined, from strongest to weakest: paragraph boundary, line break boundary, indentation boundary, blank area boundary, and separator boundary. The sample content is split using paragraph boundaries to obtain top-level content blocks. Then, within each top-level content block, line break boundaries are used to further split it to obtain intermediate-level content blocks. Subsequently, indentation boundaries, blank area boundaries, and separator boundaries are used sequentially to progressively refine the separation to a finer granularity, until the preset finest granularity is reached. The preset boundary strength rules are set based on the degree of semantic separation of the content; paragraph boundaries separate different semantic paragraphs, exhibiting the strongest semantic independence and therefore the highest strength; separators only separate semantic elements within short sentences, resulting in the lowest strength. During actual splitting, the process strictly follows the order of strength from highest to lowest. Each level of splitting is only performed within the block generated at the previous level and cannot cross-level blocks. The preset finest granularity can be flexibly set according to the business scenario. By splitting the content blocks in descending order of separation strength, the hierarchical inclusion relationship of the content blocks can be strictly matched with the semantic inclusion relationship, avoiding logical errors such as fine-grained blocks spanning multiple content segments. The resulting content block hierarchical structure is clear, and the parent-child block inclusion relationship is explicit, providing a regular structural foundation for subsequent layer-by-layer matching.

[0113] Finally, after the content blocks are split, all levels are organized into a tree structure according to their inclusion relationships, forming a multi-level content block hierarchy with coarse to fine granularity. In implementation, the top-level content block serves as the root node of the tree, and each finer-grained sub-block serves as a child node of its corresponding parent block. Each node stores the boundary coordinates, position number, and granularity level information of the corresponding block. The tree structure can intuitively represent the hierarchical inclusion relationship of content blocks, facilitating quick querying of the parent and child blocks of any block, supporting rapid positioning during layer-by-layer matching, effectively improving the efficiency of content block retrieval and comparison, and also facilitating the subsequent expansion of blocks outward layer by layer along the tree structure when extending the semantic range of tags.

[0114] The second step is to match the semantic coverage of the initial label corresponding to each sample with each content block in the content block hierarchy layer by layer, and identify the content block covered by the semantic coverage as the semantic coverage area. In specific implementation, first extract the boundary coordinates of the target area corresponding to the initial label in the content coordinate system, and compare the spatial inclusion relationship between the block boundary and the label coverage boundary from top to bottom starting from the top content block. When the entire spatial range of a certain content block is within the label semantic coverage range, the block is completely identified as the semantic coverage area, and there is no need to continue to split and compare to a finer granular level. That is, when the following formula (15) is satisfied, the first Sample No. The first level of particle size Content blocks (i.e.) ) was identified as a semantic coverage area.

[0115] (15)

[0116] In the formula, For the first Sample No. Level 1 The left boundary of each content block relative to its x-coordinate; For the first Sample No. Level 1 The right boundary of each content block relative to the x-coordinate; For the first Sample No. Level 1 The upper boundary of each content block relative to the vertical coordinate; For the first Sample No. Level 1 The lower boundary of each content block relative to the vertical coordinate; For the first The left boundary of the target region of the sample relative to the x-coordinate; For the first The right boundary relative to the x-coordinate of the target region of the sample; For the first The upper boundary of the target area of ​​the sample is relative to the ordinate. For the first The lower boundary of the target region of the sample is relative to the ordinate. For the first Sample No. The first level of particle size Each content block; This refers to the particle size level.

[0117] Then, by adopting a coarse-to-fine layer-by-layer matching strategy, the matching range can be quickly narrowed down by first using coarse-grained blocks, and then the verification of overlapping blocks can be refined to a fine-grained level. This ensures the accuracy of coverage determination and reduces unnecessary fine-grained comparison operations. It can accurately align the semantic boundaries of manually annotated blocks with the physical boundaries of content blocks, providing a clear dividing benchmark for subsequent gap identification and avoiding misjudgment caused by blurred boundaries.

[0118] The third step involves marking continuous content blocks outside the semantic coverage area as semantic gap blocks at the same granularity level. After identifying the semantic coverage area at the current level, all content blocks at that level are traversed, and blocks not included in the semantic coverage area are filtered out. Then, adjacent and continuously arranged blocks are merged and marked according to their spatial arrangement to form independent semantic gap blocks. Scattered single non-contiguous blocks are also marked as gap blocks separately. Continuously arranged uncovered blocks usually correspond to a complete semantic omission range. Merging and marking them reduces the number of fragmented gaps, making it easier to perform overall orientation comparison with attention shift units later. Integrating scattered uncovered content into structured gap units clarifies the spatial distribution range of gaps, avoids gap fragmentation due to excessively fine granularity, and improves the efficiency of subsequent orientation matching.

[0119] The fourth step involves using the distribution range and location of all semantic gap blocks as the semantic coverage gap unit for each sample. The distribution range refers to the overall boundary coordinates of the semantic gap block in the content coordinate system, and the location refers to the vertical, horizontal, and directional orientation of the gap block relative to the initial labeled area. After integrating these two types of information into standardized semantic coverage gap units, a one-to-one orientation verification can be performed with the previously generated attention shift units. The relationship between the attention shift units and the semantic coverage gap units is as follows: Figure 4 As shown, by structurally encapsulating the spatial attributes of the gap, feature units aligned with the dimension of the attention shift unit can be formed, providing a unified comparison interface for subsequent dual-dimensional joint determination of silent drift samples, realizing standardized output of labeled gap features, eliminating feature differences under different samples and different labeling types, and ensuring the universality of subsequent comparison logic.

[0120] Taking contract information extraction as an example, the content of a contract sample is first divided into three top-level content blocks: the contract header, the main clauses, and the signature page, using paragraph boundaries. Within the main clauses block, it is further divided into thirteen intermediate-level line blocks using line break boundaries. Within each line block, it is further divided into clause item blocks using indentation boundaries, then into sentence blocks using blank area boundaries, and finally into the finest-grained short sentence blocks using separator boundaries. All blocks are organized into a multi-level tree structure. For the "contract amount" tag marked by the annotator, its semantic coverage only includes the amount figure in the third line of the main text. After layer-by-layer matching, the intermediate-level third-line block is completely covered. At the sentence granularity level, the currency description clause below the amount figure in the third line and the amount agreement prefix clause above it are not covered by the tag, and the two clauses are arranged consecutively. Therefore, they are marked as two semantic gap blocks. The distribution range coordinates of the two gaps and their upper and lower positions relative to the annotation area together constitute the semantic coverage gap unit of this sample.

[0121] S104, compare the orientation of the attention shift unit with the semantic coverage gap unit of the corresponding sample. When the distribution of the attention shift unit and the semantic coverage gap unit meets the preset overlap condition, mark the corresponding sample as a silent drift annotation sample. After traversing all samples, a set of silent drift annotation samples is obtained.

[0122] Specifically, this step is based on the core feature of silent drift, "the correspondence between attention landing point and annotation omission space", and establishes a two-dimensional spatial verification logic for behavioral data and content data. It can identify the implicit annotation defects of "seeing but not annotating" without manual review. Unlike the obvious omissions in conventional annotation, it can explore the deep quality inspection blind spots caused by the cognitive drift of annotators and improve the quality inspection coverage depth in batch annotation scenarios.

[0123] The distribution of attention-shifting units and semantic coverage gap units satisfies a preset overlap condition, specifically including:

[0124] The first step is to extract the projection area of ​​the core distribution region in the attention shift unit onto the sample content, which is then used as the shift coverage area. In practical implementation, the coordinates of the four sides of the core distribution region in the content coordinate system are directly retrieved, and the corresponding rectangular area of ​​the sample content plane is determined as the shift coverage area. This area corresponds to the content range in the sample where the annotator's attention is highly focused and deviates from the initial annotation target. This process uses the clustered and filtered core region as the comparison benchmark, which can eliminate the fluctuation interference of scattered transient cursor positions, ensure the stability of the comparison benchmark, and avoid misjudgment of orientation matching caused by random passing points.

[0125] The second step involves spatially merging all semantic gap blocks recorded by the semantic coverage gap unit to obtain the gap distribution area. During implementation, the boundary coordinates of all gap blocks are read, and the maximum outward coordinate values ​​of each corresponding boundary are taken along both the horizontal and vertical directions. These are then merged to form a single, continuous rectangular gap distribution area, corresponding to all semantic omissions not covered by the initial label. Integrating fragmented gaps into a continuous area simplifies the subsequent spatial comparison calculation logic, fully covers the spatial range of all missing content, avoids matching omissions caused by scattered gaps, and improves the execution efficiency and coverage completeness of the comparison process.

[0126] When the maximum distance between the boundary of the offset coverage area and the boundary of the gap distribution area is less than the span of the finest-grained content block in the content block hierarchy in the layout direction, and the spatial overlap ratio of the two reaches a preset overlap threshold, the two are confirmed to meet the preset overlap condition. In practice, first calculate the coordinate difference between the corresponding boundaries of the two areas, and take the maximum absolute value of all differences as the maximum boundary distance; the layout direction is taken as the original reading layout direction of the sample content from top to bottom; the span of the finest-grained content block is directly retrieved from the line height parameter corresponding to the bottom node of the content block hierarchy, and then the spatial overlap ratio of the two areas is checked sequentially to see if it reaches the preset overlap threshold. Setting an overlap threshold can exclude mismatch scenarios where the locations are similar but actually misaligned, ensuring that the matching accuracy is strictly aligned with the content splitting accuracy.

[0127] Taking the contract information extraction scenario as an example, the offset coverage area of ​​a certain contract sample corresponds to the range of the currency description sentence below the amount label area. The merged gap distribution area also corresponds to the range of the currency description sentence. The maximum distance between the two boundaries is less than the row height of the finest granularity short sentence block, and the two areas completely contain each other. Therefore, it is determined that the overlap condition is met, and the sample is marked as a silent drift label sample.

[0128] S105: For each sample in the silent drift annotation sample set, based on the cursor dwell area recorded by the attention shift unit and the content block corresponding to the semantic coverage gap unit, the semantic coverage range of the initial annotation label is extended so that the semantic boundary of the label fits the boundary of the content block, thus completing the annotation correction; samples that still cannot fit the boundary of the content block after extension are removed from the original sample set to be processed, and the cleaned annotation sample set is output.

[0129] Specifically, the annotation correction in this step is based on the fact that attention-shifted units and semantic gap units have been confirmed to overlap in silent drift samples. This means that the annotator did indeed observe the missed area but did not perform annotation. Therefore, objective behavioral data can be used to drive automated compensation of label boundaries. By directly stretching the label boundaries towards the effectively observed missed content blocks, rather than re-annotating, implicit annotation defects in a large number of samples can be repaired with extremely low computational cost, avoiding secondary manual review and significantly improving the integrity of the annotation data.

[0130] In practical implementation, the semantic coverage of the initial annotation labels is extended, specifically including:

[0131] The first step is to match content blocks outward layer by layer along the distribution direction of semantic gap blocks, starting from the original semantic boundary of the initial labeled tag. In practice, the location of the gap is first extracted from the semantic coverage gap unit to determine the direction of extension; then, based on the previously constructed tree-like content block hierarchy, starting from the block immediately adjacent to the current boundary of the tag, the blocks to be verified are extracted layer by layer in order from near to far and from coarse to fine, so that the extension path strictly follows the original layout structure of the sample and avoids crossing semantic gaps.

[0132] The second step involves verifying whether the cumulative cursor dwell time at each content block reaches the effective attention standard. Specifically, the effective attention standard is set as follows: Count all cursor dwell points from attention offset units within the content block's coordinate range, calculate the ratio of the total dwell time to the total dwell time of the sample, and if this ratio exceeds a preset effective attention threshold, such as 5%, then the effective attention standard is met. The formula for calculating the percentage of dwell time is as follows:

[0133] (16)

[0134] In the formula, For content blocks The percentage of time the cursor stays within the text; For the first Sample No. One cursor stop point; This refers to a single content block to be verified. For the first Sample No. Duration of stay at each stop; For the first The total dwell time of all cursor positions in the sample. If Then the block If the criteria for effective attention are met, it can be included in the labeling scope; This is the preset effective attention threshold.

[0135] The principle is that only when the dwell time reaches a certain intensity can it be proven that the area is the landing point of the labeler's cognitive processing rather than an unconscious scan. This is used as a criterion for whether to add a label, which can prevent accidentally swept areas from being mistakenly included in the label semantics and ensure that the correction results are highly consistent with the actual attention distribution.

[0136] The third step involves incorporating the block into the semantic coverage of the tag if the criteria are met, and continuing to validate outer blocks. If the current block does not meet the effective attention criteria, the expansion stops, and the boundary of the outermost content block currently included is used as the corrected semantic boundary of the tag. At this point, the tag boundary naturally aligns with the physical boundary of the effectively followed content block, accurately filling the semantic gap caused by silent drift. Furthermore, the corrected tags remain aligned with the content layout structure, preserving semantic integrity. Simultaneously, samples that still cannot align with the content block boundary after expansion are removed from the original set of samples to be processed, resulting in a cleaned and annotated sample set.

[0137] Samples that still cannot fit the content block boundary after extension include those where, when the semantic boundary of the tag is extended to the finest-grained content block boundary, there are still uncovered semantic gap blocks; these samples are deemed unfit. In practice, after the extension operation, the semantic coverage gap units are traversed. If gap blocks not covered by any extended tags are found, and the current boundary has reached the finest-grained block of the tree hierarchy, it indicates that the sample contains under-focused but genuinely missed content. This represents a situation of severe cognitive drift or lack of annotation intent on the part of the annotator; forced correction would introduce erroneous semantics, therefore these samples are removed.

[0138] Continuing with the above scenario, in a contract sample, the annotator only selected the number "100,000" for the "Contract Amount," omitting the currency description clause "(RMB Yuan)" directly below. The attention shift unit showed that the annotator's dwell time in this currency clause area reached 12%, exceeding the effective attention threshold, and the semantic coverage gap unit indicated that the gap was located below the label. During extension, the matching proceeded from the lower boundary of the label down to the clause block. After verifying that the dwell time percentage met the standard, it was included, and the lower boundary of the label aligned with the lower boundary of the "(RMB Yuan)" clause, completing the correction. If the capitalized amount "One Hundred Thousand Yuan Only" below was also omitted, and the dwell time percentage of this block was only 2%, failing to meet the standard, then even after extending the label to the finest-grained clause boundary, there was still an uncovered gap block. Therefore, the sample was deemed unsuitable for fitting and was discarded. The final output set of cleaned labeled samples includes both automatically corrected silent drift samples and dirty data that cannot be reliably repaired, ensuring the high purity and consistency of the training data.

[0139] like Figure 5 As shown, based on the same inventive concept, this embodiment provides a big data intelligent labeling and cleaning system for AI training, including:

[0140] The multi-source data acquisition module 201 is used to acquire the original set of samples to be processed, the corresponding initial label set, and the cursor trajectory set during the annotator's operation. The original set of samples to be processed is an uncleaned set of samples used for AI model training. The initial label set is generated by the annotator after completing the annotation of the original set of samples to be processed. The cursor trajectory set records all cursor positions and durations of the annotator in each sample annotation interface.

[0141] The trajectory space mapping module 202 is used to split the set of cursor dwell trajectories, extract all cursor dwell segments within a single task with the annotation start and submission of a single sample as the boundary, and perform spatial mapping between the position coordinates of each dwell segment and the content area of ​​the corresponding sample in the original sample set to be processed, thereby generating the attention offset unit corresponding to each sample.

[0142] The semantic gap identification module 203 is used to split each sample in the original sample set to be processed into multiple layers of content blocks along the spatial arrangement order of the content. Each layer of content blocks corresponds to a first-level content granularity. The multi-layer content blocks are matched layer by layer with the semantic coverage of the corresponding initial label, and the content blocks not covered by the label semantics are marked to generate the semantic coverage gap unit corresponding to each sample.

[0143] The silent drift determination module 204 is used to compare the orientation of the attention shift unit with the semantic coverage gap unit of the corresponding sample. When the offset orientation of the cursor coincides with the distribution orientation of the semantic gap, the corresponding sample is marked as a silent drift labeled sample. After traversing all samples, a set of silent drift labeled samples is obtained.

[0144] The annotation cleaning and correction module 205 is used to extend the semantic coverage of the initial annotation label for each sample in the silent drift annotation sample set, based on the cursor dwell area recorded by the attention shift unit and the content block corresponding to the semantic coverage gap unit, so that the semantic boundary of the label fits the boundary of the content block, thus completing the annotation correction; samples that still cannot fit the boundary of the content block after extension are removed from the original sample set to be processed, and the cleaned annotation sample set is output.

[0145] like Figure 6 As shown, based on the same inventive concept, this embodiment provides an electronic device, including:

[0146] Memory 302 is used to store computer programs;

[0147] Processor 301 is used to implement the method steps as described in the first aspect when executing a computer program.

[0148] Based on the same inventive concept, this embodiment provides a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method steps as described in the first aspect.

[0149] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0150] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent annotation and cleaning of big data for AI training, characterized in that, include: Collect the original set of samples to be processed, the corresponding set of initial labels, and the set of cursor trajectories during the annotator's work process; The original set of samples to be processed is an unwashed set of samples used for AI model training. The initial label set is generated by the annotator after the original set of samples to be processed is annotated. The cursor trajectory set records all cursor positions and durations of the annotator in each sample annotation interface. The set of cursor dwell trajectories is split, and all cursor dwell segments within a single task are extracted with the annotation start and submission of a single sample as the boundary. The position coordinates of each dwell segment are spatially mapped to the content area of ​​the corresponding sample in the original set of samples to be processed, and an attention offset unit corresponding to each sample is generated. For each sample in the original set of samples to be processed, multiple content blocks are split along the spatial arrangement of the content, and each content block corresponds to a first-level content granularity. The semantic coverage of the multi-layer content blocks is matched layer by layer with the semantic coverage of the corresponding initial label, and the content blocks not covered by the label semantics are marked to generate the semantic coverage gap unit corresponding to each sample. The attention shift unit is compared with the semantic coverage gap unit of the corresponding sample. When the distribution of the attention shift unit and the semantic coverage gap unit meets the preset overlap condition, the corresponding sample is marked as a silent drift labeled sample. After traversing all samples, a set of silent drift labeled samples is obtained. For each sample in the silent drift annotation sample set, based on the cursor dwell area recorded by the attention offset unit and the content block corresponding to the semantic coverage gap unit, the semantic coverage range of the initial annotation label is extended so that the semantic boundary of the label fits the boundary of the content block, thus completing the annotation correction; samples that still cannot fit the boundary of the content block after extension are removed from the original sample set to be processed, and the cleaned annotation sample set is output.

2. The method for intelligent annotation and cleaning of big data for AI training according to claim 1, characterized in that, The step of spatially mapping the position coordinates of each pause segment to the content region of the corresponding sample in the original sample set to be processed specifically includes: Establish a reference coordinate system for a single sample content area. Delineate the content coordinate system with the corner of the content area as the origin. Convert the screen coordinates of the cursor hovering segment into relative coordinates under the aforementioned coordinate system, so that all hovering positions correspond to specific points within the sample content area, thus completing spatial mapping.

3. The method for intelligent annotation and cleaning of big data for AI training according to claim 1, characterized in that, The generation of the attention shift unit corresponding to each sample specifically includes: Spatial clustering is performed on all cursor dwell segments within a single task. Based on the proximity of the dwell positions and the continuity of the dwell time, different attention clusters are divided. The region with the highest density of dwell points and the longest cumulative dwell time in each attention cluster is determined as the core distribution region of the attention cluster. The core distribution area of ​​each attention cluster is spatially compared with the reference target area corresponding to the initial label in the corresponding sample. The reference target area is obtained through target semantic recognition or target image recognition. The deviation direction and coverage area of ​​the core distribution area relative to the reference target area are used as the orientation attribute of the offset unit, and the relative cumulative degree of dwell time within the cluster is used as the intensity attribute of the offset unit. The orientation attribute, the intensity attribute, and the corresponding core distribution area together constitute a complete attention offset unit.

4. The method for intelligent annotation and cleaning of big data for AI training according to claim 1, characterized in that, The generation of semantic coverage gap units corresponding to each sample specifically includes: Following the spatial arrangement of each sample content, the content is split layer by layer according to the layout separation boundary between content elements, forming a content block hierarchy with multi-level granularity from coarse to fine. The semantic coverage of the initial label corresponding to each sample is matched layer by layer with each content block in the content block hierarchy, and the content block covered by the semantic coverage is identified as the semantic coverage area. At the same granularity level, continuous content blocks outside the semantic coverage area are marked as semantic gap blocks; The distribution range and location of all the semantic gap blocks are used as the semantic coverage gap unit corresponding to each sample.

5. The big data intelligent labeling and cleaning method for AI training according to claim 4, characterized in that, The process of splitting content elements layer by layer based on layout boundaries specifically includes: Identify the layout separation boundaries in the sample content, including paragraph boundaries, line break boundaries, indentation boundaries, blank area boundaries, and separator boundaries; Based on preset boundary strength rules, the separation strength of each layout separation boundary is determined, in descending order of strength: paragraph boundary, line break boundary, indentation boundary, blank area boundary, and separator boundary. The sample content is split using the paragraph boundary to obtain the top-level content block. Then, each top-level content block is split using the line break boundary to obtain the intermediate-level content block. Subsequently, the indentation boundary, the blank area boundary, and the separator boundary are used sequentially to split the content into finer granularities until the preset finest granularity is reached. The content blocks of all levels are organized into a tree structure according to their inclusion relationship, forming the content block hierarchy with a multi-level granularity from coarse to fine.

6. The method for intelligent annotation and cleaning of big data for AI training according to claim 4, characterized in that, The distribution of the attention shift units and the semantic coverage gap units satisfies a preset overlap condition, specifically including: Extract the projection region of the core distribution region in the attention offset unit onto the sample content, and use it as the offset coverage region; The semantic gap blocks recorded by the semantic coverage gap unit are spatially merged to obtain the gap distribution area; When the maximum distance between the boundary of the offset coverage area and the boundary of the gap distribution area is less than the span of the finest granular content block in the layout direction at the content block level, and the spatial overlap ratio of the two reaches a preset overlap threshold, the preset overlap condition is confirmed to be met.

7. The method for intelligent annotation and cleaning of big data for AI training according to claim 4, characterized in that, The extension of the semantic coverage of the initial labeled tags specifically includes: Starting from the original semantic boundaries of the initial annotation tags, content blocks are matched outward layer by layer along the distribution direction of semantic gap blocks; Each time the extension reaches a content block, it checks whether the cumulative cursor dwell time of the block reaches the effective attention standard. If it does, the block is included in the semantic coverage of the tag, and the verification continues to the outermost block. If the current block does not reach the effective attention standard, the extension stops, and the boundary of the currently included outermost content block is used as the corrected semantic boundary of the tag.

8. The big data intelligent labeling and cleaning method for AI training according to claim 4, characterized in that, The samples that still cannot fit the content block boundaries after extension specifically include: When the semantic boundary of the tag extends to the boundary of the finest content block, there are still semantic gap blocks that are not covered, and the sample is determined to be unfit.

9. A big data intelligent labeling and cleaning system for AI training, based on the big data intelligent labeling and cleaning method for AI training as described in claim 1, characterized in that, include: The multi-source data acquisition module is used to acquire the original set of samples to be processed, the corresponding initial label set, and the cursor trajectory set during the annotator's operation. The original set of samples to be processed is an unwashed set of samples used for AI model training. The initial label set is generated by the annotator after labeling the original set of samples to be processed. The cursor trajectory set records all cursor positions and durations of the annotator in each sample labeling interface. The trajectory space mapping module is used to split the set of cursor dwell trajectories, extract all cursor dwell segments within a single task with the annotation start and submission of a single sample as the boundary, and spatially map the position coordinates of each dwell segment with the content area of ​​the corresponding sample in the original set of samples to be processed to generate an attention offset unit corresponding to each sample. The semantic gap identification module is used to split each sample in the original sample set to be processed into multiple content blocks along the spatial arrangement order of the content, with each content block corresponding to a first-level content granularity. The semantic coverage of the multi-layer content blocks is matched layer by layer with the semantic coverage of the corresponding initial label, and the content blocks not covered by the label semantics are marked to generate the semantic coverage gap unit corresponding to each sample. The silent drift determination module is used to compare the orientation of the attention shift unit with the semantic coverage gap unit of the corresponding sample. When the distribution of the attention shift unit and the semantic coverage gap unit meets the preset overlap condition, the corresponding sample is marked as a silent drift labeled sample. After traversing all samples, a silent drift labeled sample set is obtained. The annotation cleaning and correction module is used to extend the semantic coverage of the initial annotation label for each sample in the silent drift annotation sample set, based on the cursor dwell area recorded by the attention offset unit and the content block corresponding to the semantic coverage gap unit, so that the semantic boundary of the label fits the boundary of the content block, thus completing the annotation correction; samples that still cannot fit the boundary of the content block after extension are removed from the original sample set to be processed, and the cleaned annotation sample set is output.

Images