A gift sale identification model training method, a gift sale identification method and device

Abstract

This invention relates to the field of model training technology, and more particularly to a method, apparatus, and training method for a gift sale recognition model. It addresses the technical problems of existing model training methods having weak adaptability in the gift sale recognition field and relying on a certain scale of manually labeled data to start the model, leading to high training difficulty. The invention includes: acquiring a gift sale context corpus and extracting a first keyword from the corpus; matching related information from a pre-defined gift sale knowledge base based on the first keyword; extracting pre-defined format tags from the gift sale context corpus and related information; using the gift sale context corpus, related information, and pre-defined format tags as structured samples to generate a structured sample training set; calculating risk scores and correcting attribution fragments on the structured sample training set to obtain a target training set; and training a gift sale recognition model using the target training set.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for training a gift buying and selling recognition model, a gift buying and selling recognition method, and an apparatus. Background Technology

[0002] With the online transformation of e-commerce and social gift-giving, massive amounts of gift-selling text (such as product descriptions, user reviews, and customer service conversations) need to be automatically and accurately categorized, and their underlying transaction intentions or emotional attributions analyzed. Text classification, as a core subtask of natural language processing, has significant application value in areas such as e-commerce risk control, user profiling, and precision marketing.

[0003] Existing text classification methods mainly involve the training of text classification models, text classification techniques, and devices. In the training method, a training set is obtained, and any first training text is input into the text classification model. One principal probability and m additional probability are obtained through the main classification network and m supplementary classification networks, respectively. Based on the m additional probabilities of each training text, certain texts are selected from the training set to form a certainty set. Based on the prediction loss of each certain text, target texts corresponding to a certain threshold number are selected from each category in the certainty set to form a first clean set. A first noise set is formed based on the other certain texts in the certainty set besides the target texts. Using a large model, the predicted labels for each first noise text are obtained based on the first clean set. The text classification model is then trained based on each first noise text and its predicted label, as well as each first clean text and its category label.

[0004] However, all of the above solutions require a fixed set of category labels to be manually set before model training or inference. When a new category appears, the label system must be redesigned and the model retrained / adjusted, making it impossible to adapt to the gift buying and selling recognition scenario in real time. Furthermore, the model needs to be started with a certain amount of manually labeled data, which increases the difficulty of model training. Summary of the Invention

[0005] This invention provides a training method, a method and apparatus for recognizing gift transactions, and a model for gift transactions recognition. It addresses the technical problem that existing model training methods have weak adaptability in the field of gift transaction recognition and rely on a certain scale of manually labeled data to start the model, resulting in high difficulty in model training.

[0006] This invention provides a method for training a gift buying and selling recognition model, comprising: Obtain the contextual data of gift buying and selling, and extract the first keyword from the contextual data of gift buying and selling; Based on the first keyword, relevant information is matched from a pre-defined gift sales knowledge base; Extract preset format tags from the gift buying and selling context corpus and the associated information; The gift buying and selling context corpus, the associated information, and the preset format labels are used as structured samples to generate a structured sample training set; Risk scores are calculated and attribution fragments are corrected on the structured sample training set to obtain the target training set; The target training set is used to train the gift buying and selling recognition model.

[0007] Optionally, the step of extracting preset format tags from the gift buying and selling context corpus and the associated information includes: Based on a preset few-sample example, preset format labels are extracted from the gift buying and selling context corpus and the associated information through a preset large language model.

[0008] Optionally, the step of calculating risk scores and correcting attribution fragments on the structured sample training set to obtain the target training set includes: Each structured sample in the structured sample training set is split into several sample segments; Vectorize the sample fragment to obtain the sample fragment vector; The sample segment vector is matched in a preset vector library to obtain the most similar segment vector; Determine whether the first similarity between the sample segment vector and the most similar segment vector is greater than a preset threshold; If so, the sample segment corresponding to the sample segment vector is selected as the selected segment; Calculate the segment risk score of the selected segment; Based on the selected segment and the segment risk score, a target structured sample is generated; Integrate all target structured samples to obtain the target training set.

[0009] Optionally, the step of generating a target structured sample based on the selected segment and the segment risk score includes: The selected segments are sorted from largest to smallest according to their risk scores to obtain the sequence number of each selected segment; The first target field of the preset format label of the structured sample is replaced by the selected fragment with the highest sequence number of selected fragments, and the selected fragment with the highest risk score is added to the second target field of the preset format label to obtain the target structured sample.

[0010] Optionally, the preset vector library includes a risk sample fragment vector library and a non-risk sample fragment vector library; the step of generating the preset vector library includes: Obtain a training set of historical structured samples; Human-edited annotations are performed on the historical structured sample training set to generate a gift sales dataset and a non-gift sales dataset; The historical structured samples in the gift buying and selling dataset and the non-gift buying and selling dataset are divided into multiple gift fragments and non-gift fragments respectively. Extract the segments containing the second keyword from the gift segments as risky segments, and treat the non-gift segments as non-risky segments; Calculate the risk score for each risk segment, and store the risk segments and their corresponding risk scores in a preset format to obtain a risk sample segment vector library; Calculate the non-risk score for each non-risk segment, and store the non-risk segments and their corresponding non-risk scores in a preset format to obtain a non-risk sample segment vector library.

[0011] Optionally, the step of calculating the risk score for each of the risk segments includes: The risk segment is vectorized to obtain a segment vector; Clustering the fragment vectors yields multiple initial clusters; When there is overlapping data in multiple initial clusters, the overlapping data is assigned to the initial cluster with the largest data volume to obtain multiple target clusters; The total data volume of all risk segments and the cluster data volume of each target cluster are calculated. The risk score of each risk segment within each target cluster is calculated based on the cluster data volume and the total data volume.

[0012] Optionally, the step of calculating the segment risk score of the selected segment includes: Obtain the risk score of the most similar segment vector corresponding to the selected segment, and use it as the segment risk score of the selected segment.

[0013] Optionally, the step of calculating the segment risk score of the selected segment includes: The first number of first keywords contained in the structured samples corresponding to the selected fragment, and the second number of sample fragments contained therein are counted. Obtain the second similarity between each selected sample and the corresponding most similar segment vector contained in the structured sample corresponding to the selected segment; Calculate the independent risk score for each selected segment based on the second similarity corresponding to each selected sample contained in the structured sample; Calculate the average risk score of all selected segments based on the independent risk scores; Calculate the keyword violation score based on the first quantity and the second quantity; The selected segment's segment risk score is obtained by weighted summing the average risk score and the keyword violation score.

[0014] This invention also provides a gift purchase and sale identification method based on a gift purchase and sale identification model, the method comprising: Obtain the sample to be identified; The sample to be identified is input into a pre-trained gift buying and selling identification model, which outputs classification inference results. Extract classification labels from the classification reasoning results, and identify gift buying and selling behavior based on the classification labels.

[0015] The present invention also provides a training device for a gift buying and selling recognition model, comprising: The corpus and keyword extraction module is used to acquire the contextual corpus of gift buying and selling, and extract the first keyword from the contextual corpus of gift buying and selling; The related information matching module is used to match related information from a preset gift buying and selling knowledge base based on the first keyword; A preset format tag extraction module is used to extract preset format tags from the gift buying and selling context corpus and the associated information; The structured sample training set generation module is used to generate a structured sample training set by using the gift buying and selling context corpus, the associated information, and the preset format labels as structured samples. The target training set generation module is used to calculate risk scores and correct attribution fragments in the structured sample training set to obtain the target training set. The model training module is used to train a gift buying and selling recognition model using the target training set.

[0016] The present invention also provides a gift sale recognition device based on a gift sale recognition model, wherein the gift sale recognition model is trained by the training device described in claim 10, and the recognition device comprises: The sample acquisition module is used to acquire the sample to be identified. The classification reasoning result output module is used to input the sample to be identified into the pre-trained gift buying and selling recognition model and output the classification reasoning result; The gift buying and selling behavior identification module is used to extract classification labels from the classification reasoning results and identify gift buying and selling behavior based on the classification labels.

[0017] As can be seen from the above technical solutions, the present invention has the following advantages: The present invention discloses a method for training a gift buying and selling recognition model, and specifically discloses: obtaining gift buying and selling contextual corpus and extracting a first keyword from the gift buying and selling contextual corpus; matching related information from a preset gift buying and selling proprietary knowledge base based on the first keyword; extracting preset format tags from the gift buying and selling contextual corpus and related information; using the gift buying and selling contextual corpus, related information, and preset format tags as structured samples to generate a structured sample training set; calculating risk scores and correcting attribution fragments on the structured sample training set to obtain a target training set; and training the gift buying and selling recognition model using the target training set.

[0018] This invention provides precise domain knowledge support for gift buying and selling scenarios by setting up a dedicated knowledge base for gift buying and selling, thereby improving the accuracy of data analysis. At the same time, by extracting preset format labels from the contextual corpus and related information of gift buying and selling, the labels are dynamically generated according to the sample content, without the need for a preset fixed classification system. This adapts to the dynamic evolution characteristics of the gift buying and selling domain, improves the adaptability of the model in the gift buying and selling recognition field, and reduces the difficulty of model training by eliminating the need to rely on a large amount of manually standardized data. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the steps of a gift buying and selling recognition model training method provided in an embodiment of the present invention; Figure 2 A flowchart illustrating the steps of a gift buying and selling recognition model training method according to another embodiment of the present invention; Figure 3 Example diagram of calculating risk scores for risk segment clustering; Figure 4 This is a schematic diagram illustrating the acquisition of the most similar segment vector; Figure 5 Flowchart for generating target structured samples; Figure 6 A flowchart illustrating the steps of a gift purchase and sale identification method based on a gift purchase and sale identification model provided in this embodiment of the invention; Figure 7 A schematic diagram illustrating the application of a gift sales recognition model; Figure 8This is a structural block diagram of a gift buying and selling recognition model training device provided in an embodiment of the present invention; Figure 9 This is a structural block diagram of a gift purchase and sale identification device based on a gift purchase and sale identification model, provided in an embodiment of the present invention. Detailed Implementation

[0021] This invention provides a training method, a method and apparatus for identifying gift sales, and a model for gift sales recognition. These methods address the technical problems of existing model training methods having weak adaptability in the field of gift sales recognition and relying on a certain scale of manually labeled data to start the model, which leads to high difficulty in model training.

[0022] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0023] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a gift buying and selling recognition model training method provided in an embodiment of the present invention.

[0024] The present invention provides a method for training a gift buying and selling recognition model, which may specifically include the following steps: Step 101: Obtain the contextual data of gift buying and selling, and extract the first keyword from the contextual data of gift buying and selling; In this embodiment of the invention, corpora related to gift buying and selling can be pre-screened from a corpus, and the context of these corpora (e.g., the preceding and following 5 sentences) can be extracted as the gift buying and selling context corpus. Then, the first keyword related to gift buying and selling can be extracted from the gift buying and selling context corpus, such as "zhuanzhuan," "doushang," "chundou," and "mdou."

[0025] Step 102: Match relevant information from the preset gift buying and selling knowledge base based on the first keyword; To improve the accuracy of data analysis and provide precise domain knowledge support for gift buying and selling scenarios, this embodiment of the invention pre-sets a gift buying and selling proprietary knowledge base. By matching the first keyword in the pre-set gift buying and selling proprietary knowledge base, related information related to the context of gift buying and selling can be obtained.

[0026] Step 103: Extract preset format tags from the contextual corpus and related information of gift buying and selling; In one example, the default formatting tags can be JSON formatted tags.

[0027] JSON (JavaScript Object Notation) is a lightweight data exchange format used for storing and transmitting data in scenarios such as network transmission, API interfaces, and configuration files.

[0028] Existing technologies require a fixed set of category labels to be manually pre-defined before model training or inference. When a new category appears, the labeling system must be redesigned and the model retrained / adjusted, making it impossible to adapt to gift-buying and selling recognition scenarios in real time. To solve this problem, this invention extracts JSON-formatted labels from the gift-buying and selling context corpus and related information to serve as labels for the gift-buying and selling context corpus, reducing the need for manual labeling and eliminating the need to manually set new categories.

[0029] Step 104: Use the contextual corpus of gift buying and selling, related information, and preset format labels as structured samples to generate a structured sample training set; After obtaining the JSON format tags, the gift buying and selling context corpus, the related information in the gift buying and selling knowledge base, and the JSON format tags can be used as structured samples to generate a structured sample training set for subsequent training of the gift buying and selling recognition model.

[0030] Step 105: Calculate risk scores and correct attribution fragments on the structured sample training set to obtain the target training set; In this embodiment of the invention, in order to improve the quality of training samples and thus improve the effect of subsequent model training or inference, risk scores can be calculated and attribution fragments can be corrected on the structured sample training set to obtain the target training set.

[0031] Step 106: Train the gift buying and selling recognition model using the target training set.

[0032] After generating the target training set, the target training set can be used to train the gift buying and selling recognition model.

[0033] It should be noted that the gift buying and selling recognition model can be trained using any conventional model training method, and the embodiments of the present invention do not impose specific limitations on it.

[0034] This invention provides precise domain knowledge support for gift buying and selling scenarios by setting up a dedicated knowledge base for gift buying and selling, thereby improving the accuracy of data analysis. At the same time, by extracting preset format labels from the contextual corpus and related information of gift buying and selling, the labels are dynamically generated according to the sample content, without the need for a preset fixed classification system. This adapts to the dynamic evolution characteristics of the gift buying and selling domain, improves the adaptability of the model in the gift buying and selling recognition field, and reduces the difficulty of model training by eliminating the need to rely on a large amount of manually standardized data.

[0035] Please see Figure 2 , Figure 2 A flowchart illustrating the steps of a gift buying and selling recognition model training method according to another embodiment of the present invention. Specifically, it may include the following steps: Step 201: Obtain the contextual data of gift buying and selling, and extract the first keyword from the contextual data of gift buying and selling; In this embodiment of the invention, corpora related to gift buying and selling can be pre-screened from a corpus, and the context of these corpora (e.g., the preceding and following 5 sentences) can be extracted as the gift buying and selling context corpus. Then, the first keyword related to gift buying and selling can be extracted from the gift buying and selling context corpus, such as "zhuanzhuan," "doushang," "chundou," and "mdou."

[0036] Step 202: Match relevant information from the preset gift buying and selling knowledge base based on the first keyword; In one example, the default knowledge base for gift buying and selling is shown in Table 1: Table 1

[0037] Assuming the contextual corpus of gift buying and selling is as shown in the text on the left side of Table 2, and the first keyword is "bean merchant", then the associated information obtained by matching the first keyword in the gift buying and selling proprietary knowledge base is shown in the Knowledge on the right side of Table 2.

[0038] Table 2

[0039] Step 203: Extract preset format tags from the contextual corpus of gift buying and selling and related information; In this embodiment of the invention, preset format tags are extracted from the contextual corpus of gift buying and selling and related information to serve as tags for the contextual corpus of gift buying and selling, reducing the manual annotation process and eliminating the need to manually set new categories.

[0040] In one example, based on a preset few-sample example, JSON-formatted tags can be extracted from the contextual corpus of gift buying and selling and related information using a preset large language model.

[0041] In the specific implementation, a few-shot example is designed as follows: ExampleData( good sister Do you have a backpack? What are you doing """, extractions=[ Extraction extraction_class="risk-free", extraction_text="Do you have a backpack?" attributes={ "Label": 0, "Risk Category": "", "Risk type": "", "Slang terms": "", "Trading Intent": "", "External drainage": "", "illustrate": "", } ) ] ) Using the few-shot technique, a large language model can extract the JSON-formatted labels to be trained from the original samples (gift buying and selling context corpus), as shown in Table 3.

[0042] Table 3

[0043] It should be noted that the extraction process of JSON format tags from a large model using fewshot technology can refer to conventional extraction methods, and this embodiment of the invention does not impose specific limitations on this.

[0044] Step 204: Use the contextual corpus of gift buying and selling, related information, and preset format labels as structured samples to generate a structured sample training set; After obtaining the JSON format tags, the gift buying and selling context corpus, the related information in the gift buying and selling knowledge base, and the JSON format tags can be used as structured samples to generate a structured sample training set for subsequent training of the gift buying and selling recognition model.

[0045] Step 205: Divide each structured sample in the structured sample training set into several sample segments. In this embodiment of the invention, the JSON file of each structured sample in the structured sample training set contains classification results, key fragments, and result attribution. The classification results characterize whether the structured sample carries a risk of gift trading; the key fragments record the basis for determining whether the classification result is risky; and the result attribution records the most crucial basis for determining whether the classification result is risky. To ensure that the JSON format tags accurately express the classification results, key fragments, and result attribution, risk scores can be calculated and attribution fragments can be corrected on the structured sample training set.

[0046] First, the structured samples in the training set are divided into several sample segments, and risk analysis is performed on each sample segment.

[0047] Step 206: Vectorize the sample fragments to obtain sample fragment vectors; After generating sample fragments, in order to match sample fragments in a preset vector library, the sample fragments need to be vectorized to obtain sample fragment vectors.

[0048] Step 207: Match the sample segment vectors in the preset vector library to obtain the most similar segment vector; In this embodiment of the invention, the preset vector library may include a risky sample fragment vector library and a non-risky sample fragment vector library. The process of generating the vector library may include the following steps: S01, Obtain the historical structured sample training set; S02, perform human-edited annotation on the historical structured sample training set to generate a gift sales dataset and a non-gift sales dataset; S03, the historical structured samples in the gift buying and selling dataset and the non-gift buying and selling dataset are divided into multiple gift slices and non-gift slices respectively; S04, extract the segments containing the second keyword from the gift segments as risky segments, and treat the non-gift segments as non-risky segments; S05, calculate the risk score of each risk segment, and store the risk segments and their corresponding risk scores in a preset format to obtain a risk sample segment vector library; In its implementation, the vector library can be generated based on historical structured samples. First, the training set of historical structured samples is manually labeled as "gift buying" and "non-gift buying," creating gift buying and non-gift buying datasets. The samples labeled as gift buying are then segmented, and the segments containing the second keyword from the gift buying-specific knowledge base are extracted as risk segments. Finally, the risk score for each risk segment is calculated.

[0049] In one example, the steps for calculating the risk score for each risk segment may include: S051, Vectorize the risk segment to obtain the segment vector; S052, cluster the fragment vectors to obtain multiple initial clusters; S053, when there is overlapping data in multiple initial clusters, the overlapping data is divided into the initial cluster with the largest data volume to obtain multiple target clusters; S054, Calculate the total data volume of all risk segments and the cluster data volume of each target cluster; S055, calculate the risk score of each risk segment within each target cluster based on the cluster data volume and the total data volume.

[0050] In the specific implementation, when calculating the risk score of each risk segment, the risk segment is first vectorized to obtain multiple segment vectors. Then, the segment vectors are clustered to obtain multiple initial clusters. Next, it is determined whether there is overlapping data among all the initial clusters. If so, the overlapping data is assigned to the initial cluster with the largest data volume to obtain the target cluster. Then, the total data volume of all risk segments and the cluster data volume of each target cluster are counted. Based on the cluster data volume and the total data volume, the risk score of each risk segment within each target cluster is calculated.

[0051] like Figure 3 As shown, Figure 3 This is an example diagram for calculating risk scores by clustering risk segments. It includes three clusters: Cluster 1, Cluster 2, and Cluster 3. Cluster 1 overlaps with both Cluster 2 and Cluster 3. Since the data volume of Cluster 1 is greater than that of Cluster 2 and Cluster 3, the overlapping data is assigned to Cluster 1. The cluster data volumes of the three target clusters are as follows: Cluster 1 (cluster_1): 19; Cluster 2: 14; Cluster 3: 8.

[0052] The total data volume of all risk segments, total_vec_num, is: total_vec_num = cluster_1 + cluster_2 + cluster_3= 19+14+8= 41; Therefore, the risk scores of the risk segments in the three target clusters are as follows: cluster_1_score = cluster_1 / total_vec_num= 19 / 41= 0.46; cluster_1_score = cluster_2 / total_vec_num= 14 / 41= 0.34; cluster_1_score = cluster_3 / total_vec_num = 8 / 41 = 0.20.

[0053] Furthermore, the risk score of each target cluster can be enhanced by multiplying it by a coefficient alpha, such as alpha. Then, the final risk scores for each target cluster are: cluster_1_score_fix = cluster_1_score alpha = 0.92; cluster_2_score_fix = cluster_2_score alpha = 0.67; cluster_3_score_fix = cluster_3_score alpha = 0.40.

[0054] If the adjusted risk score is greater than 1, then the adjusted risk score is set to 1.

[0055] Similarly, by referring to the process of calculating the risk score of a risky segment, the non-risk score of a non-risky segment can be calculated.

[0056] Finally, each risk segment and its corresponding risk maple tree are converted into the format [index, text, vector, score] and stored in the risk sample segment vector library.

[0057] S06, calculate the non-risk score of each non-risk segment, and store the non-risk segments and their corresponding non-risk scores in a preset format to obtain a non-risk sample segment vector library.

[0058] Based on the process of generating the risk sample fragment vector library, the same principle applies to generating the non-risk sample fragment vector library.

[0059] After vectorizing the sample fragments, the most similar fragment vector can be matched in the vector library, and the similarity with the most similar fragment vector can be calculated.

[0060] like Figure 4 As shown, a structured sample can include multiple sample fragments. If the "extraction_class" of the JSON format tag is risky, the most similar fragment vector is matched from the risky sample fragment vector library. If the "extraction_class" is risk-free, the most similar fragment vector is matched from the non-risky sample fragment vector library.

[0061] Specifically, Assuming that "extraction_class" is considered risky in the current JSON format tag, the most similar sample is obtained from the sample fragment vector library, and its similarity is: text_seg_similarity = max(sim_risk_0, sim_risk_1,...,sim_risk_i); The complete data for each vector in the fragment vector library is as follows: [index,text,seg_vec,score].

[0062] Step 208: Determine whether the first similarity between the sample segment vector and the most similar segment vector is greater than a preset threshold; Step 209: If yes, select the sample segment corresponding to the sample segment vector as the selected segment; After obtaining the first similarity score (text_seg_similarity) of the most similar segment of the sample segment vector, it can be determined whether text_seg_similarity is greater than the preset threshold th_text_seg_similarity (e.g., th_text_seg_similarity = 0.5). If so, the current segment can be selected, and the vector library data with the most similar value will be added to the risk segment candidate pool. { {text_seg_i,[index,text,seg_vec,score]} } If multiple risk segments are selected, then: { {text_seg_i,[index,text,seg_vec,score]} ... {text_seg_x,[index,text,seg_vec,score]} }

[0063] Step 210: Calculate the segment risk score of the selected segment; Next, calculate the segment risk score for the selected segment.

[0064] In one example, step 210 may include: obtaining the risk score of the most similar segment vector corresponding to the selected segment, as the segment risk score of the selected segment.

[0065] In a practical implementation, the risk score of the most similar segment to the selected segment in the vector library can be obtained as the segment risk score of the selected segment.

[0066] In another example, step 210 may include the following sub-steps: S2101, count the first number of first keywords contained in the structured samples corresponding to the selected segment, and the second number of sample segments contained therein; S2102, obtain the second similarity between each selected sample and the corresponding most similar segment vector contained in the structured sample corresponding to the selected segment; S2103, Calculate the independent risk score for each selected segment based on the second similarity of each selected sample contained in the structured sample; S2104, Calculate the average risk score of all selected segments based on the independent risk scores; S2105, calculate keyword violation scores based on the first and second quantities; S2106, the weighted sum of the independent risk score and the keyword violation score yields the segment risk score for the selected segment.

[0067] In the specific implementation, assume that the structured sample has M sample segments, each containing X primary keywords. N segments are selected, and the independent risk score (score_seg) for each sample segment is: score_seg = text_seg_sim score0.

[0068] Where text_seg_sim is the second similarity between the selected sample and the corresponding most similar segment vector, and score0 is the basic risk score.

[0069] Assuming N=3, the average risk score (score_seg_avg) for all selected samples is: score_seg_avg = (score_seg_1 + score_seg_2 + score_seg_3) / N.

[0070] The keyword violation score (score_keywords) is: score_keywords = X / M.

[0071] Assuming the weights of the keywords and the vector library are a and b respectively, the segment risk score is:

[0072] score = a score_keywords + b score_seg_avg; Where: a+b = 1, such as a=0.3, b=0.7.

[0073] Step 211: Generate target structured samples based on the selected segments and their risk scores; In this embodiment of the invention, the structured sample can be modified according to the selected segment and its corresponding segment risk score to obtain the target structured sample.

[0074] In one example, step 211 may include the following sub-steps: S2111, Sort the selected segments from largest to smallest according to the segment risk score to obtain the sequence number of each selected segment; S2112, replace the first target field of the preset format label of the structured sample with the selected fragments with the highest sequence number of selected fragments, and add the selected fragments with the highest fragment risk score to the second target field of the preset format label to obtain the target structured sample.

[0075] In the specific implementation, the selected segments in the selection pool can be sorted from largest to smallest according to the segment risk score to obtain the sequence number of each selected segment. Then, the first target field (extraction_text field) of the JSON format label of the structured sample is replaced by the selected segment with the first preset number of sequence numbers. The selected segment with the highest segment risk score is added to the second target field (hitting slang field) of the JSON format label to obtain the target structured sample.

[0076] In one example, let's assume the original JSON format tags are: [{"extraction_class": "Risk", "extraction_text": "8-10 pm, reliable, is the host, he can't run away from the hall", "attributes": {"Label": 1, "Risk category": "Risk", "Risk point type": "Risk", "Hit the code": "", "Transaction intention": "Clear", "External traffic generation": "No", "Description": "Gift buying and selling behavior"}}].

[0077] The JSON format tags for adding segment risk scores and selected fields are as follows: [{"extraction_class": "risky", "extraction_text": "text_seg_j,text_seg_h,text_seg_i", "attributes": {"Label": 1, "risk category": "risky", "risk point type": "risky", "hitting slang": "text_seg_j", "transaction intent": "clear", "external traffic generation": "no", "description": "gift buying and selling behavior"},"score": "score_max"}].

[0078] The value of score_max is between 0 and 1: 0 < score_max < 1.

[0079] To facilitate understanding, the following example illustrates the process of generating target structured samples: like Figure 5 As shown, the process first obtains the structured sample to be processed and segments the text within it. Then, it selects the appropriate vector library based on the classification results in the JSON format tags. Next, it calculates the similarity between the sample segments of the structured sample and the knowledge base data, obtaining the most similar segment vector with the highest similarity. Then, it checks whether the similarity with the most similar segment vector is greater than a preset threshold. If so, it is retained, and both the selected segments and the most similar segment vector are kept in the candidate pool. Finally, the segment risk scores of the selected segments in the candidate pool are sorted, and the top N selected segments from the candidate pool are used to modify the JSON format tags of the structured sample, resulting in the target structured sample.

[0080] Step 212: Integrate all target structured samples to obtain the target training set; After obtaining all the target structured samples, they can be integrated to obtain the target training set.

[0081] Step 213: Train the gift buying and selling recognition model using the target training set.

[0082] After generating the target training set, it can be used to train a gift-selling identification model, thereby training a large model that can accurately identify gift-selling samples. Accurate identification of gift-selling samples allows for timely blocking and punishment of such activities, effectively combating illicit transactions and illegal fund transfers, preventing crime, and fulfilling platform regulatory responsibilities.

[0083] In one example, an initial gift buying and selling recognition model can be trained using an SFT model and a target training set. Then, a reinforcement learning model can be used to optimize this initial model, thereby training a new gift buying and selling recognition model. The input to this new model is the target training set, and the output is JSON-formatted labels. It should be noted that using an SFT model and a reinforcement learning model for training is a standard training method in this field, and the training process will not be elaborated upon here.

[0084] Please see Figure 6 , Figure 6 The flowchart illustrates the steps of a gift purchase and sale identification method based on a gift purchase and sale identification model, as provided in an embodiment of the present invention.

[0085] This invention provides a gift purchase and sale identification method based on a gift purchase and sale identification model, comprising: Step 601: Obtain the sample to be identified; Step 602: Input the sample to be identified into the pre-trained gift buying and selling identification model and output the classification inference result; Step 603: Extract classification labels from the classification reasoning results and identify gift buying and selling behavior based on the classification labels.

[0086] In this embodiment of the invention, text containing keywords can be obtained and input into a pre-trained gift buying and selling recognition model to output classification inference results. Then, classification labels can be extracted from the classification inference results to identify gift buying and selling behavior based on the classification labels.

[0087] The classification reasoning results can be JSON format labels. The risk category in the JSON format labels records the classification labels, which can be either risky or risk-free.

[0088] This invention accurately identifies gift buying and selling behavior through a gift buying and selling model, enabling timely blocking and punishment of such behavior. This effectively combats private gray transactions and illegal fund transfers, prevents illegal activities, and fulfills the platform's regulatory responsibilities.

[0089] Taking the classification reasoning result as a JSON-formatted label as an example, after completing the gift buying and selling recognition model, the process of recognizing gift buying and selling based on the model is as follows: Figure 7 As shown: Input the sample to be identified into the gift buying and selling identification model, and output JSON format labels. Then extract the category labels from the JSON format labels.

[0090] This invention provides precise domain knowledge support for gift buying and selling scenarios by setting up a dedicated knowledge base for gift buying and selling, thereby improving the accuracy of data analysis. At the same time, by extracting JSON format tags from the contextual corpus and related information of gift buying and selling, the tags are dynamically generated with the sample content, without the need for a pre-set fixed classification system. This adapts to the dynamic evolution characteristics of the gift buying and selling domain, improves the adaptability of the model in the gift buying and selling recognition field, and reduces the difficulty of model training by eliminating the need to rely on a large amount of manually standardized data.

[0091] Please see Figure 8 , Figure 8 This is a structural block diagram of a gift buying and selling recognition model training device provided in an embodiment of the present invention.

[0092] This invention provides a training device for a gift buying and selling recognition model, comprising: The corpus and keyword extraction module 801 is used to acquire the contextual corpus of gift buying and selling, and extract the first keyword from the contextual corpus of gift buying and selling; The related information matching module 802 is used to match related information from a preset gift buying and selling knowledge base based on the first keyword; The preset format tag extraction module 803 is used to extract preset format tags from the context corpus and related information of gift buying and selling; The structured sample training set generation module 804 is used to generate a structured sample training set by using the contextual corpus of gift buying and selling, related information and preset format labels as structured samples. The target training set generation module 805 is used to calculate risk scores and correct attribution fragments on the structured sample training set to obtain the target training set. Model training module 806 is used to train a gift buying and selling recognition model using the target training set.

[0093] In this embodiment of the invention, the preset format tag extraction module 803 includes: The preset format label extraction submodule is used to extract preset format labels from the contextual corpus and related information of gift buying and selling based on preset few sample examples and a preset large language model.

[0094] In this embodiment of the invention, the target training set generation module 805 includes: The sample fragment splitting submodule is used to split each structured sample in the structured sample training set into several sample fragments. The sample fragment vector generation submodule is used to vectorize sample fragments to obtain sample fragment vectors. The most similar segment vector acquisition submodule is used to match sample segment vectors in a preset vector library to obtain the most similar segment vector; The threshold judgment submodule is used to determine whether the first similarity between the sample segment vector and the most similar segment vector is greater than a preset threshold. The Select Segment Acquisition Submodule is used to select the sample segment corresponding to the sample segment vector if the condition is met. The segment risk score calculation submodule is used to calculate the segment risk score of the selected segment; The target structured sample generation submodule is used to generate target structured samples based on the selected segment and the segment risk score; The target training set generation submodule is used to integrate all target structured samples to obtain the target training set.

[0095] In this embodiment of the invention, the target structured sample generation submodule includes: The sorting unit is used to sort the selected segments from largest to smallest according to their segment risk scores, and obtain the sequence number of each selected segment. The target structured sample generation unit is used to replace the first target field of the preset format label of the structured sample with the selected fragments with the highest sequence number of selected fragments, and add the selected fragments with the highest fragment risk score to the second target field of the preset format label to obtain the target structured sample.

[0096] In this embodiment of the invention, the preset vector library includes a risky sample fragment vector library and a non-risky sample fragment vector library; the generation of the preset vector library includes: The historical structured sample training set acquisition submodule is used to acquire the historical structured sample training set; The Gift Sales Dataset and Non-Gift Sales Dataset Generation Submodule is used to perform human-edited annotation on the historical structured sample training set to generate the Gift Sales Dataset and Non-Gift Sales Dataset. The sharding submodule is used to shard the historical structured samples in the gift buying and selling dataset and the non-gift buying and selling dataset respectively, resulting in multiple gift shards and non-gift shards; The non-risk fragment generation submodule is used to extract fragments containing the second keyword from gift fragments as risky fragments and non-gift fragments as non-risky fragments. The risk sample fragment vector library generation submodule is used to calculate the risk score of each risk fragment and store the risk fragments and their corresponding risk scores in a preset format to obtain the risk sample fragment vector library. The non-risk sample fragment vector library generation submodule is used to calculate the non-risk score of each non-risk fragment and store the non-risk fragments and their corresponding non-risk scores in a preset format to obtain the non-risk sample fragment vector library.

[0097] In this embodiment of the invention, the risk sample fragment vector library generation submodule includes: The fragment vector generation unit is used to vectorize risk fragments to obtain fragment vectors; Clustering units are used to cluster fragment vectors to obtain multiple initial clusters; The target cluster generation unit is used to divide the overlapping data into the initial cluster with the largest amount of data when there is overlapping data in multiple initial clusters, thereby obtaining multiple target clusters; The total data volume and cluster data volume statistics unit is used to count the total data volume of all risk segments and the cluster data volume of each target cluster. The risk score calculation unit is used to calculate the risk score of each risk segment within each target cluster based on the cluster data volume and the total data volume.

[0098] In this embodiment of the invention, the segment risk score calculation submodule includes: The first segment risk score calculation unit is used to obtain the risk score of the most similar segment vector corresponding to the selected segment, which is used as the segment risk score of the selected segment.

[0099] In this embodiment of the invention, the segment risk score calculation submodule includes: The quantity statistics unit is used to count the first number of first keywords contained in the structured samples corresponding to the selected segment, and the second number of sample segments contained therein. The second similarity acquisition unit is used to acquire the second similarity between each selected sample contained in the structured sample corresponding to the selected segment and the vector of the most similar segment. The independent risk score calculation unit is used to calculate the independent risk score of each selected segment based on the second similarity corresponding to each selected sample contained in the structured sample. The average risk score calculation unit is used to calculate the average risk score of all selected segments based on the independent risk scores. The keyword violation score calculation unit is used to calculate the keyword violation score based on the first quantity and the second quantity. The segment risk score calculation unit is used to calculate the weighted average risk score and keyword violation score to obtain the segment risk score of the selected segment.

[0100] Please see Figure 9 , Figure 9 This is a structural block diagram of a gift purchase and sale identification device based on a gift purchase and sale identification model, provided in an embodiment of the present invention.

[0101] This invention provides a gift sale identification device based on a gift sale identification model. The device includes: The sample acquisition module 901 is used to acquire the sample to be identified. The classification reasoning result output module 902 is used to input the sample to be identified into the pre-trained gift buying and selling recognition model and output the classification reasoning result; The gift buying and selling behavior recognition module 903 is used to extract classification labels from the classification reasoning results and recognize gift buying and selling behavior based on the classification labels.

[0102] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0104] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0108] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0109] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0110] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0111] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a gift buying and selling recognition model, characterized in that, include: Obtain the contextual data of gift buying and selling, and extract the first keyword from the contextual data of gift buying and selling; Based on the first keyword, relevant information is matched from a pre-defined gift sales knowledge base; Extract preset format tags from the gift buying and selling context corpus and the associated information; The gift buying and selling context corpus, the associated information, and the preset format labels are used as structured samples to generate a structured sample training set; Risk scores are calculated and attribution fragments are corrected on the structured sample training set to obtain the target training set; The target training set is used to train the gift buying and selling recognition model.

2. The method according to claim 1, characterized in that, The step of extracting preset format tags from the gift buying and selling context corpus and the associated information includes: Based on a preset few-sample example, preset format labels are extracted from the gift buying and selling context corpus and the associated information through a preset large language model.

3. The method according to claim 1, characterized in that, The step of calculating risk scores and correcting attribution fragments in the structured sample training set to obtain the target training set includes: Each structured sample in the structured sample training set is split into several sample segments; Vectorize the sample fragment to obtain the sample fragment vector; The sample segment vector is matched in a preset vector library to obtain the most similar segment vector; Determine whether the first similarity between the sample segment vector and the most similar segment vector is greater than a preset threshold; If so, the sample segment corresponding to the sample segment vector is selected as the selected segment; Calculate the segment risk score of the selected segment; Based on the selected segment and the segment risk score, a target structured sample is generated; Integrate all target structured samples to obtain the target training set.

4. The method according to claim 3, characterized in that, The step of generating target structured samples based on the selected segments and the risk scores of the segments includes: The selected segments are sorted from largest to smallest according to their risk scores to obtain the sequence number of each selected segment; The first target field of the preset format label of the structured sample is replaced by the selected fragment with the highest sequence number of selected fragments, and the selected fragment with the highest risk score is added to the second target field of the preset format label to obtain the target structured sample.

5. The method according to claim 3, characterized in that, The preset vector library includes a risky sample fragment vector library and a non-risky sample fragment vector library; The steps for generating the preset vector library include: Obtain a training set of historical structured samples; Human-edited annotations are performed on the historical structured sample training set to generate a gift sales dataset and a non-gift sales dataset; The historical structured samples in the gift buying and selling dataset and the non-gift buying and selling dataset are divided into multiple gift fragments and non-gift fragments respectively. Extract the segments containing the second keyword from the gift segments as risky segments, and treat the non-gift segments as non-risky segments; Calculate the risk score for each risk segment, and store the risk segments and their corresponding risk scores in a preset format to obtain a risk sample segment vector library; Calculate the non-risk score for each non-risk segment, and store the non-risk segments and their corresponding non-risk scores in a preset format to obtain a non-risk sample segment vector library.

6. The method according to claim 5, characterized in that, The step of calculating the risk score for each of the risk segments includes: The risk segment is vectorized to obtain a segment vector; Clustering the fragment vectors yields multiple initial clusters; When there is overlapping data in multiple initial clusters, the overlapping data is assigned to the initial cluster with the largest data volume to obtain multiple target clusters; The total data volume of all risk segments and the cluster data volume of each target cluster are calculated. The risk score of each risk segment within each target cluster is calculated based on the cluster data volume and the total data volume.

7. The method according to claim 6, characterized in that, The step of calculating the segment risk score of the selected segment includes: Obtain the risk score of the most similar segment vector corresponding to the selected segment, and use it as the segment risk score of the selected segment.

8. The method according to claim 3, characterized in that, The step of calculating the segment risk score of the selected segment includes: The first number of first keywords contained in the structured samples corresponding to the selected fragment, and the second number of sample fragments contained therein are counted. Obtain the second similarity between each selected sample and the corresponding most similar segment vector contained in the structured sample corresponding to the selected segment; Calculate the independent risk score for each selected segment based on the second similarity corresponding to each selected sample contained in the structured sample; Calculate the average risk score of all selected segments based on the independent risk scores; Calculate the keyword violation score based on the first quantity and the second quantity; The selected segment's segment risk score is obtained by weighted summing the average risk score and the keyword violation score.

9. A method for identifying gift transactions based on a gift transaction identification model, characterized in that, The gift buying and selling recognition model is trained by any one of claims 1-8, and the method includes: Obtain the sample to be identified; The sample to be identified is input into a pre-trained gift buying and selling identification model, which outputs classification inference results. Extract classification labels from the classification reasoning results, and identify gift buying and selling behavior based on the classification labels.

10. A training device for a gift buying and selling recognition model, characterized in that, include: The corpus and keyword extraction module is used to acquire the contextual corpus of gift buying and selling, and extract the first keyword from the contextual corpus of gift buying and selling; The related information matching module is used to match related information from a preset gift buying and selling knowledge base based on the first keyword; A preset format tag extraction module is used to extract preset format tags from the gift buying and selling context corpus and the associated information; The structured sample training set generation module is used to generate a structured sample training set by using the gift buying and selling context corpus, the associated information, and the preset format labels as structured samples. The target training set generation module is used to calculate risk scores and correct attribution fragments in the structured sample training set to obtain the target training set. The model training module is used to train a gift buying and selling recognition model using the target training set.

11. A gift purchase and sale identification device based on a gift purchase and sale identification model, characterized in that, The gift buying and selling identification model is trained by the training device described in claim 10, the identification device comprising: The sample acquisition module is used to acquire the sample to be identified. The classification reasoning result output module is used to input the sample to be identified into the pre-trained gift buying and selling recognition model and output the classification reasoning result; The gift buying and selling behavior identification module is used to extract classification labels from the classification reasoning results and identify gift buying and selling behavior based on the classification labels.

Images