Large model labeling method based on data labeling rule

Abstract

The invention relates to the field of data processing, in particular to a data annotation rule-based large model annotation method, which comprises the following steps of: acquiring a plurality of pieces of data to be annotated based on a database, determining type information and annotation requirements according to the data to be annotated, and creating a data set to be annotated according to the data to be annotated; constructing an annotation rule set according to the type information and annotation requirements; calculating a matching degree score vector of each piece of to-be-labeled data according to the labeling rule set; and according to the matching degree score vector, selecting a labeling rule with the highest matching degree score to label the to-be-labeled data # imgabs0 # so as to obtain standard data. And through the constructed labeling rule set, a matching degree score vector # imgabs 1 # is calculated by using a calculation formula, and a rule with the highest matching degree score is selected for labeling, so that # imgabs 2 # during labeling of the to-be-labeled data is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a large model annotation method based on data annotation rules. Background Art

[0002] In the field of artificial intelligence, the performance of large models is highly dependent on high-quality labeled data. For example, the patent application with publication number CN119961605A and invention name "A data labeling method and system based on a large language model" discloses data preparation: collecting raw data and preprocessing; model selection and loading: selecting and loading a suitable large model, the large model is used to perform preliminary labeling on the preprocessed data; automatic labeling: using the loaded large model to automatically label the preprocessed data, and filtering and optimizing the preliminary labeling results; manual verification and correction: verifying the results of the automatic labeling and correcting the incorrect labeling; labeling result integration and output: integrating the corrected labeling results and outputting the labeling data set in an appropriate format; model optimization and iteration: optimizing and iterating the large model according to the labeling results. In this invention, the labeling results are automatically generated by the large language model. Although the efficiency of labeling has been improved to a certain extent, it lacks a refined analysis of the features of the data to be labeled. When processing data to be labeled with complex semantics or structures, it is difficult to ensure the accuracy of the labeling.

[0003] In summary, the technical problem actually solved by the present invention is how to improve the accuracy of labeling the data to be labeled. Summary of the Invention

[0004] In order to overcome the technical defect of the prior art that lacks refined analysis of data features and leads to low annotation accuracy, the purpose of the present invention is to provide a large model annotation method based on data annotation rules, by constructing a set of annotation rules. , use the calculation formula to calculate the matching score vector Select the rule with the highest matching score for annotation, thereby improving the accuracy of annotating the data to be annotated. .

[0005] The present invention discloses a large model annotation method based on data annotation rules, comprising the steps of: Obtain some data to be labeled based on the database, determine the type information and labeling requirements based on the data to be labeled, and create a data set based on some data to be labeled ,in is the number of data to be labeled; Construct annotation rule sets based on type information and annotation requirements , where m is the number of labeling rules; Calculate the matching score vector of each data to be labeled according to the labeling rule set R ,in, Represented as data to be labeled and marking rules The matching score of , The calculation formula is: Among them, p represents the number of features of each data to be labeled, Expressed as the weight of the k-th feature, is the data to be labeled calculated based on the kth feature and marking rules The matching function of According to the matching score vector Select the annotation rule with the highest matching score Treating labeled data Marking is performed to obtain standard data, where .

[0006] Preferably, a set of annotation rules is constructed based on type information and annotation requirements. The step also includes verifying the annotation rule set, including the following steps: Select some unlabeled data as verification data to create a verification data set , where l is the number of verification data; For the validation data set Each verification data in , , based on the above method to calculate the matching score vector , and select the annotation rule with the highest matching score Verification data Annotation is performed to obtain verification annotation data, and the annotation of the verification annotation data is the verification annotation result; Verify the annotation results and the validation data set The actual annotation comparison is performed to calculate the annotation accuracy. The annotation accuracy calculation formula is: in, Expressed as an indicator function, when the labeling result is consistent with the true labeling ,otherwise ;

[0007] The labeling accuracy Acc is compared with the preset labeling accuracy threshold T. When the labeling accuracy Acc is less than the labeling accuracy threshold T, the labeling rule set R is adjusted until the labeling accuracy Acc is greater than or equal to the labeling accuracy threshold T.

[0008] Preferably, the method of adjusting the labeling rule set R includes: Add new annotation rules based on the validation data set Improperly labeled validation data To create new type information and add it to the annotation rule set middle; Modify existing labeling rules and modify the conditions or weights of rules with low labeling accuracy; Delete invalid annotation rules and delete the annotation rules that are not selected in the verification or cause a large number of incorrect annotations from the annotation rule set R.

[0009] Preferably, when treating the labeled data After marking, the following steps are also included: From the standard data dataset Select some standard data as the evaluation sample set , where q represents the number of standard data selected as evaluation samples; For the evaluation sample set Each standard data , manually annotated by at least two people to obtain a manually annotated set , where k represents the number of manual annotations; Calculating manual annotation consistency scores , the calculation formula is: in, It is expressed as the number of combinations of 2 selected from k manual annotation results. is the indicator function, when and When consistent ,otherwise ; Calculate the evaluation sample set Consistency score ; Compare the consistency score C with the preset consistency score threshold Comparison, when the consistency score C is less than the consistency score threshold The marked results are reviewed and corrected.

[0010] Preferably, after creating the to-be-annotated data set D according to the plurality of to-be-annotated data, the method further includes preprocessing the to-be-annotated data; When the data to be annotated is text, preprocessing includes text cleaning, removing noise characters, special symbols, stop words, and text segmentation; When the data to be labeled is an image, preprocessing includes image denoising, normalization, and size adjustment; When the data to be labeled is audio, preprocessing includes audio noise reduction, format conversion, and sampling rate adjustment; When the data to be labeled is video, preprocessing includes video frame extraction and video frame rate adjustment.

[0011] Preferably, text cleaning uses regular expressions to remove special symbols; common words are removed through a stop word list; and text segmentation uses a word segmentation algorithm to divide the text into words or phrases.

[0012] Preferably, image denoising adopts median filtering and Gaussian filtering algorithms; image normalization maps image pixels to the range of 0~1 or 0~255; image resizing adopts interpolation algorithm to adjust the image to a preset size to preprocess the data to be labeled for the image.

[0013] Preferably, audio noise reduction adopts Wiener filtering and spectral subtraction algorithm; audio format conversion converts the format of the data to be labeled as audio into WAV or MP3 format; audio sampling rate adjustment adjusts the audio sampling rate of the data to be labeled as audio to a standard value of 44.1kHz or 16kHz.

[0014] After adopting the above technical solution, compared with the existing technology, the present invention constructs a set of annotation rules. , use the calculation formula to calculate the matching score vector The rule with the highest matching score is selected for annotation, thereby improving the accuracy of annotation of the data to be annotated. More specifically, a set of annotation rules R is constructed based on the type information and annotation requirements of the data to be annotated, and a mathematical calculation formula is used to calculate the matching score vector between the data to be annotated and each annotation rule. , and select the rule with the highest matching score for annotation, quantify the annotation process, avoid subjective judgment bias, make the annotation more in line with the actual characteristics of the data to be annotated, and effectively solve the annotation error problem caused by the general annotation rules and lack of refined analysis in the existing technology, thereby improving the accuracy of annotation of the data to be annotated; by verifying the data set The labeling rule set R is verified and adjusted, so that labeling rules can be dynamically added, modified or deleted according to the labeling accuracy, so as to timely adapt to the distribution changes of the data to be labeled and the new types of data to be labeled, so as to improve the effectiveness and applicability of the labeling rules; the quality of the standard results is evaluated to judge the reliability of the labeling results, prevent erroneous labels from entering the large model for training, and effectively improve the effect and performance of large model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of the steps of a large model annotation method based on data annotation rules of the present invention; Figure 2 For Figure 1 Schematic diagram of the steps for validating the labeling rule set in . DETAILED DESCRIPTION

[0016] The advantages of the present invention are further described below with reference to the accompanying drawings and specific embodiments.

[0017] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0018] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0020] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0021] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal communication between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0022] In the following description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0023] The present invention discloses a large model annotation method based on data annotation rules, comprising the steps of: Obtain some data to be labeled based on the database, determine the type information and labeling requirements based on the data to be labeled, and create a data set based on some data to be labeled ,in The number of data to be labeled; build a set of labeling rules based on type information and labeling requirements , where m is the number of labeling rules; calculate the matching score vector of each data to be labeled according to the labeling rule set R ,in, Represented as data to be labeled and marking rules The matching score of , The calculation formula is: Where p represents the number of features of each data to be labeled. Expressed as the weight of the k-th feature, is the data to be labeled calculated based on the kth feature and marking rules Matching function; according to the matching score vector Select the annotation rule with the highest matching score Treating labeled data Marking is performed to obtain standard data, where .

[0024] See Figure 1 As shown, in this embodiment, a large model annotation method based on data annotation rules is described in detail, including the following steps: Step S100: Acquire a number of data to be labeled based on a database, and the source database for acquisition includes but is not limited to one or more of local files, storage devices, etc. The acquisition method includes but is not limited to remotely acquiring the data to be labeled through an API interface, a web crawler, or a data transmission protocol; or, acquiring the data to be labeled in real time through sensors, devices, or data streams. After acquiring the data to be labeled, the type information and labeling requirements of the data to be labeled will be determined based on the acquired data to be labeled. The determined type information is which of the data to be labeled is text, image, audio, video, etc., and the determined labeling requirements refer to the subsequent tasks of labeling the data to be labeled. The labeling requirements are to determine the specific requirements and goals for marking, classifying, annotating or structuring the labeled data, that is, to clarify "what needs to be done to the labeled data", "how to do it", and "to what extent" when labeling the labeled data. A set of data to be labeled will be created based on the acquired data to be labeled. , so that the acquired individual data to be annotated are formed into a set, enabling batch annotation when annotating the data to be annotated, thereby improving annotation efficiency. For example, in some embodiments, when annotating each data to be annotated individually, it is necessary to frequently switch tasks, load files, and adjust annotation parameters, which is time-consuming and error-prone. If there are 100 data to be annotated, the above operations need to be repeated 100 times. If the 100 data to be annotated are created into a data set to be annotated, only one batch annotation operation is required, significantly reducing duplication of work and improving annotation efficiency. It should be noted that, in the set of data to be labeled D, d represents one data to be labeled, and n represents the total number of data to be labeled. For example, in some embodiments, the data set D to be labeled is a large number of text paragraphs collected from social media and news websites; in the field of computer vision, the data set D to be labeled may be a collection of images collected by a camera or obtained from a public image database. Step S200: Construct a set of annotation rules based on the determined type information and annotation requirements , where r represents a labeling rule and m represents the total number of labeling rules. In this step, it is emphasized that the construction of the labeling rule set R must be strictly based on the type information of the data to be labeled and the labeling requirements to ensure the pertinence and flexibility of each labeling rule. The construction process of the labeling rule set R includes analyzing the characteristics of each data to be labeled, extracting key information, and formulating a number of corresponding labeling rules in combination with the labeling requirements. In this embodiment, medical image labeling will be used as an example for a detailed description. If the type information of the data to be labeled is X-rays, and the labeling requirement is to label lung diseases, then the labeling rule set R constructed according to the type information and labeling requirements should include specific rules such as "If ground-glass shadows appear in the image, it is labeled as early lung disease" and "If there are massive high-density shadows, it is labeled as lung tumors" to ensure that each labeling rule can accurately correspond to the characteristics of lung diseases and avoid mislabeling or omissions. Step S300: In this step, each data to be labeled in the data set D to be labeled ( ), calculate the matching score vector according to the labeling rule set R , among which Represented as data to be labeled and marking rules The matching score is calculated as follows: In this calculation formula, p represents the number of features of each data to be labeled. Expressed as the weight of the k-th feature, is the data to be labeled calculated based on the kth feature and marking rules The matching function is used to calculate the matching score of the annotation rule of the data to be annotated in a quantitative way. In this embodiment, the image annotation is described in detail. Assume that p=3, that is, considering the three features of the image: color, shape and texture, 、 、 The weights corresponding to the three features are: 、 、 is the corresponding matching function. For an image to be labeled and the labeling rules for "identifying cats" , Maybe calculate the similarity between the fur color in the image and common cat colors, Determine how well the outline of the object in the image matches the shape of the cat, Evaluate the pixels of hair texture. Obtained by weighted summation , a matching score is calculated for each data to be labeled and all the labeling rules, and finally a matching score vector is formed , in order to realize the transformation of the labeling process from empirical judgment to quantitative analysis. Step S400: Calculate the matching score vector obtained in step S300 , select the annotation rule with the highest matching score , treat labeled data Marking is done to obtain standard data. It should be noted that , that is, select the labeling rule with the highest matching degree with the data to be labeled from all labeling rules. In this step, the matching score vector calculated in the previous step is , through the "arg max" operator, accurately filter out the data to be labeled from m labeling rules The annotation rule with the highest matching degree , and annotate with this annotation rule. For example, in some embodiments, for a certain text data , its matching score vector , which correspond to the three annotation rules "sports news", "financial news" and "entertainment news", respectively. It shows that the matching score of the annotation rule of "financial news" is the highest. Sure , that is, select the second labeling rule and label the text as "Financial News." In this embodiment, this optimal selection of labeling rules avoids arbitrary and subjective labeling, ensuring that each piece of data to be labeled receives the label that best fits its characteristics, significantly improving the accuracy and reliability of labeling.

[0025] Furthermore, a set of annotation rules is constructed based on type information and annotation requirements. The step also includes verifying the annotation rule set, including the following steps: selecting a number of data to be annotated as verification data to create a verification data set , where l is the number of verification data; for the verification data set Each verification data in , , based on the above method to calculate the matching score vector , and select the annotation rule with the highest matching score Verification data Perform annotation to obtain verification annotation data; combine the verification annotation results with the verification data set The actual annotation comparison is performed to calculate the annotation accuracy. The annotation accuracy calculation formula is: .in, Expressed as an indicator function, when the labeling result is consistent with the true labeling ,otherwise ; Compare the labeling accuracy Acc with the preset labeling accuracy threshold T. When the labeling accuracy Acc is less than the labeling accuracy threshold T, adjust the labeling rule set R until the labeling accuracy Acc is greater than or equal to the labeling accuracy threshold T.

[0026] See Figure 2 As shown, in this embodiment, the large model annotation method based on data annotation rules will be described in detail again. Step S200 described in the above embodiment also includes the step of verifying the annotation rule set, which specifically includes the following: Step S210: Randomly select a certain proportion of the data to be labeled from the acquired data to be labeled as verification data, and then create a verification data set based on the selected verification data. , where l is the number of verification data, ensuring the representativeness and diversity of the verification data. Step S220: Verify the data set created in step S210 Each verification data in Specifically, the method provided in the above embodiment verifies the data set Each verification data in Perform annotation to obtain validation annotation data. Step S230: Combine the verification annotation data obtained in the above step with the verification data set The actual annotation is compared to calculate the annotation accuracy. It should be noted that the annotation accuracy calculation formula is in, Expressed as an indicator function, when the verification annotation result is consistent with the true annotation ,otherwise It should be noted that the true annotations described in this embodiment are obtained by accurate manual annotation. For example, in some embodiments, there are three entities after manual annotation of a certain text, but there are two entities after annotation using the above method. Then, the annotation accuracy rate is obtained by calculating the ratio of the number of verified annotation results that are consistent with the true annotation results to the total number of verified data. Specifically, in a news text processing project, the annotation task is to identify the three types of entities, "person name", "place name" and "institution name", from the news article. There is a piece of news text as verification data: "A attended a technical seminar held by Y company in X place". After manual real annotation, this text has three real entities "A (person name)", "X (place name)" and "Y company (institution name)". After annotation using the annotation rule set R, the verification annotation result is "A (person name)", "X (place name)" and "Y (institution name)". Therefore, the verification annotation result only annotates "Y company" with Y, and does not fully annotate "Y company", so the annotation is wrong. If such a verification data set There are 10 such news texts in total. After annotating them one by one and comparing them with the real annotations, it was found that the verification annotation results of 7 paragraphs were completely consistent with the real annotations. Then the number of verification annotation results and real annotation results is 7, and the total number of verification data is 10, so the annotation accuracy is 70%. Step S240: Evaluate the effectiveness of the labeling rule set R based on the labeling accuracy. For example, a labeling accuracy threshold T is preset, and the labeling accuracy calculated in step S230 is compared with the labeling accuracy threshold T. If the labeling accuracy is less than the labeling accuracy threshold T, it is considered that the labeling rule set R needs to be adjusted until the labeling accuracy Acc is greater than or equal to the labeling accuracy threshold T. For example, the labeling accuracy threshold T is set to 85%. When the labeling accuracy Acc is 70%, the labeling rule set R needs to be adjusted, and the labeling rules need to be added or modified. It should be noted that when the labeling accuracy threshold is 50%, it means that the accuracy of the labeling rule set R is applicable. It should also be noted that the setting of the labeling accuracy threshold T should be adjusted in combination with the actual application scenarios and needs. The specific numerical values ​​described in this embodiment are for explanation only and do not represent the standard in all cases.

[0027] Furthermore, the method of adjusting the annotation rule set R includes: adding new annotation rules, based on the verification data set Create new type information for incorrectly annotated validation data and add it to the annotation rule set In the process of labeling, the existing labeling rules are modified, and the conditions or weights of the rules with low labeling accuracy are modified; invalid labeling rules are deleted, and the labeling rules that are not selected in the verification or cause a large number of incorrect labeling are deleted from the labeling rule set R.

[0028] In this embodiment, a detailed description is given of adjusting the labeling rule set R when the labeling accuracy Acc is less than the labeling accuracy threshold T. The adjustment methods include but are not limited to adding new labeling rules, modifying existing labeling rules, and deleting invalid labeling rules.

[0029] Adding new annotation rules is mainly for verifying the data set Improperly labeled validation data Thus, new type information is created, and new annotation rules are created according to the created new type information to be added to the annotation rule set R, thereby supplementing and improving the annotation rule set R so that it can correct the inaccurately labeled rules.

[0030] Modify the existing labeling rules. Analyze the labeling rules with low labeling accuracy in the validation data set R in detail, and adjust their conditions or weights to improve labeling accuracy.

[0031] Deleting invalid annotation rules means removing those annotation rules that do not play a role in the verification process or cause incorrect annotations, ensuring the efficiency and accuracy of the annotation rule set R.

[0032] Furthermore, when dealing with labeled data After labeling, the following steps are also included: Select some standard data as the evaluation sample set , where q represents the number of standard data selected as evaluation samples; for the evaluation sample set Each standard data , manually annotated by at least two people to obtain a manually annotated set , where k represents the number of manual annotations; calculate the consistency score of manual annotations , the calculation formula is: .in, It is expressed as the number of combinations of 2 selected from k manual annotation results. is the indicator function, when and When consistent ,otherwise ; Calculate the evaluation sample set Consistency score ; Compare the consistency score C with the preset consistency score threshold Comparison, when the consistency score C is less than the consistency score threshold The marked results are reviewed and corrected.

[0033] In this embodiment, the large model annotation method based on data annotation rules will be described in detail again. After step S400 of the above embodiment, the annotation results are evaluated, which specifically includes the following steps: Step S500: From the dataset of labeled data Select some labeled data to create an evaluation sample set , where q represents the number of labeled data selected as evaluation samples. It should be noted that random sampling is used when selecting labeled data to ensure the randomness and representativeness of the evaluation samples and avoid bias caused by subjective selection. Step S600: For the evaluation sample set Each standard data in At least two people independently annotate to obtain a set of manually annotated images. , where k represents the number of manual annotations. In this embodiment, we will take the image annotation task as an example to describe it. Assume that the evaluation sample set There is an image data labeled "cat" , arrange three labelers to label it again, and the manual labeling set is . Which may appear = "cat", =cat, = "dog". Step S700: According to the calculation formula Calculate the manual annotation consistency score, where It is expressed as the number of combinations of 2 selected from k manual annotation results. is the indicator function, when and When consistent ,otherwise In the example of the three manual annotations above, k=3, =3, that is, there are 3 combinations of two. Indicator function Determine whether each group is consistent, if consistent, it is recorded as 1, if not, it is recorded as 0. In the above example, and Consistent, and Inconsistency, and Inconsistent, then , the final manual annotation consistency score . Step S800: In this step, the manual annotation consistency score calculated in step S700 is Calculate the evaluation sample set The consistency score C is calculated as follows: Step S900: In this step, the evaluation sample set calculated in step S800 is After the consistency score C is obtained, the consistency score is compared with the preset consistency score. Compare, if , it indicates that there may be many errors or inconsistencies in the annotation, and it is necessary to review or correct the annotation results. For example, if the preset consistency score threshold is , and the calculated C = 0.33, then the annotation data needs to be re-reviewed to ensure the annotation accuracy.

[0034] Furthermore, after creating the set D of data to be annotated based on a number of data to be annotated, it also includes preprocessing the data to be annotated; when the data to be annotated is text, the preprocessing includes text cleaning, removing noise characters, special symbols, stop words, and text tokenization; when the data to be annotated is an image, the preprocessing includes image denoising, normalization, and size adjustment; when the data to be annotated is audio, the preprocessing includes audio noise reduction, format conversion, and sampling rate adjustment; when the data to be annotated is video, the preprocessing includes video frame extraction and video frame rate adjustment.

[0035] In this embodiment, the preprocessing of the data to be annotated will be described in detail. The data to be annotated can be text, image, audio, or video.

[0036] When the data to be annotated is text, the preprocessing includes text cleaning, removing noise characters, special symbols, stop words, and text tokenization. Text cleaning uses regular expressions to remove special symbols; common words are removed through a stop word list; text tokenization uses a tokenization algorithm to split the text into words or phrases. For example, in some embodiments, when using regular expressions to remove punctuation marks or some special symbols, such as "@", "#", etc.; filtering out high-frequency meaningless words such as "de", "shi", "zai", etc. through a stop word list.

[0037] When the data to be annotated is an image, the preprocessing includes image denoising, normalization, and size adjustment. Image denoising uses median filtering and Gaussian filtering algorithms; image normalization maps the image pixels to the range of 0~1 or 0~255; image size adjustment uses an interpolation algorithm to adjust the image to a preset size to preprocess the data to be annotated for the image.

[0038] When the data to be annotated is audio, the preprocessing includes audio noise reduction, format conversion, and sampling rate adjustment. Audio noise reduction uses Wiener filtering and spectral subtraction algorithms; audio format conversion converts the format of the data to be annotated as audio to the WAV or MP3 format; audio sampling rate adjustment adjusts the audio sampling rate of the data to be annotated as audio to the standard values of 44.1kHz or 16kHz.

[0039] When the data to be annotated is video, the preprocessing includes video frame extraction and video frame rate adjustment.

[0040] It should be noted that the embodiments of the present invention have better practicability and do not impose any form of limitation on the present invention. Any technician familiar with the field may use the technical content disclosed above to change or modify it into an equivalent effective embodiment. However, any modification or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A large model annotation method based on data annotation rules, characterized in that: Including steps: Acquire a number of data to be labeled based on the database, determine the type information and labeling requirements based on the data to be labeled, and create a data set to be labeled based on the data to be labeled ,in is the number of data to be labeled; Construct a labeling rule set according to the type information and the labeling requirements , where m is the number of labeling rules; Calculate the matching score vector of each of the to-be-annotated data according to the annotation rule set R ,in, Represented as the data to be labeled With the annotation rules The matching score of , The calculation formula is: Wherein, p represents the number of features of each data to be labeled, Expressed as the weight of the k-th feature, is the data to be labeled calculated based on the kth feature and marking rules The matching function of According to the matching score vector Select the annotation rule with the highest matching score The data to be labeled Marking is performed to obtain standard data, where .

2. The large model annotation method based on data annotation rules according to claim 1 is characterized in that: Constructing a labeling rule set based on the type information and the labeling requirements The step also includes verifying the annotation rule set, including the following steps: Select some of the data to be annotated as verification data to create a verification data set , where l is the number of verification data; For the validation data set Each of the verification data , , calculate the matching score vector based on the method as described in claim 1 , and select the annotation rule with the highest matching score The verification data Annotation is performed to obtain verification annotation data, where the annotation of the verification annotation data is a verification annotation result; The verification annotation results are combined with the verification data set The actual annotation comparison is performed to calculate the annotation accuracy. The annotation accuracy calculation formula is: in, Expressed as an indicator function, when the verification annotation result is consistent with the true annotation ,otherwise ; The labeling accuracy Acc is compared with a preset labeling accuracy threshold T. When the labeling accuracy Acc is less than the labeling accuracy threshold T, the labeling rule set R is adjusted until the labeling accuracy Acc is greater than or equal to the labeling accuracy threshold T.

3. The large model annotation method based on data annotation rules according to claim 2 is characterized in that: Methods for adjusting the labeling rule set R include: Add new annotation rules based on the validation data set Improperly labeled validation data To create new type information and add it to the annotation rule set middle; Modify the existing labeling rules, and modify the conditions or weights of the rules with low labeling accuracy; Deleting invalid marking rules: deleting the marking rules that are not selected in the verification or cause a large number of erroneous markings from the marking rule set R.

4. The large model annotation method based on data annotation rules according to claim 1 is characterized in that: In the data to be labeled After marking, the following steps are also included: From the standard data dataset Select some standard data as the evaluation sample set , where q represents the number of standard data selected as evaluation samples; For the evaluation sample set Each of the standard data , manually annotated by at least two people to obtain a manually annotated set , where k represents the number of manual annotations; Calculating manual annotation consistency scores , the calculation formula is: in, It is expressed as the number of combinations of 2 selected from k manual annotation results. is the indicator function, when and When consistent ,otherwise ; Calculate the evaluation sample set Consistency score ; Compare the consistency score C with the preset consistency score threshold When the consistency score C is less than the consistency score threshold The marked results are reviewed and corrected.

5. The large model annotation method based on data annotation rules according to claim 1 is characterized in that: After creating a set of data to be labeled D based on a number of data to be labeled, the method further includes preprocessing the data to be labeled; When the data to be annotated is text, the preprocessing includes text cleaning, removal of noise characters, special symbols, stop words and text segmentation; When the data to be labeled is an image, the preprocessing includes image denoising, normalization, and size adjustment; When the data to be labeled is audio, the preprocessing includes audio noise reduction, format conversion, and sampling rate adjustment; When the data to be labeled is a video, the preprocessing includes video frame extraction and video frame rate adjustment.

6. The large model annotation method based on data annotation rules according to claim 5 is characterized in that: The text cleaning uses regular expressions to remove special symbols; common words are removed through a stop word list; and the text segmentation uses a segmentation algorithm to divide the text into words or phrases.

7. The large model annotation method based on data annotation rules according to claim 5 is characterized in that: The image denoising adopts median filtering and Gaussian filtering algorithms; the image normalization maps image pixels to the range of 0~1 or 0~255; the image resizing adopts interpolation algorithm to adjust the image to a preset size to preprocess the data to be labeled which is an image.

8. The large model annotation method based on data annotation rules according to claim 5 is characterized in that: The audio noise reduction adopts Wiener filtering and spectral subtraction algorithm; the audio format conversion converts the format of the data to be labeled as audio into WAV or MP3 format; the audio sampling rate is adjusted to adjust the audio sampling rate of the data to be labeled as audio to a standard value of 44.1kHz or 16kHz.

Images