Automatic information collection method and device, electronic equipment and storage medium

Abstract

The invention discloses an automatic information collection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining the chat content of a target group, and carrying out the preprocessing of the chat content; the preprocessed chat content comprises a chat record of each chat object in the target group; performing subject extraction on the chat record of each chat object, and screening based on a subject extraction result in combination with a target word to obtain a target user; performing an automatic session with a target user by using a pre-trained large model to obtain a target intelligence session; and performing risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating. According to the method, automatic conversation with the target user is carried out in a large model mode, the limitation of only depending on passive monitoring is avoided, intelligence information can be effectively identified through deep analysis of conversation content, the false alarm rate and the missing report rate are reduced, and the method can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an automated intelligence collection method, device, electronic equipment and storage medium. Background Art

[0002] Data leakage is a serious cybersecurity incident, but there is a lack of effective technical monitoring means to quickly discover and issue timely warnings. Some social media platforms have become hotbeds for various illegal and criminal activities due to their anonymity and openness. These activities are usually carried out through groups, so monitoring and analyzing group content is particularly important.

[0003] Existing solutions mainly include rule-based monitoring and machine learning-based methods. The rule-based monitoring method requires setting rules in advance. Once a conversation that meets the rules is found, it will be marked and alarmed. This method is simple and easy to implement, but it is easy to miss unknown threats and has a high false alarm rate. The machine learning-based method automatically identifies threat conversations through training models, which has higher accuracy and efficiency. However, this method requires a large amount of labeled data and high computing resources. Summary of the invention

[0004] The present invention proposes an automated intelligence collection method, device, electronic device and storage medium, aiming to solve at least one of the technical problems in the related art to a certain extent. The embodiments of the present invention can efficiently and accurately perform automated intelligence collection.

[0005] On the one hand, an embodiment of the present invention provides an automated intelligence collection method, comprising:

[0006] Obtaining chat content of the target group and preprocessing the chat content; the preprocessed chat content includes the chat records of each chat object in the target group;

[0007] Extract topics from the chat records of each chat partner, and select the target user based on the results of subject extraction combined with the target word;

[0008] Use the pre-trained large model to automatically communicate with the target user and obtain the target intelligence conversation;

[0009] Perform risk assessment on the target user's response content in the target intelligence session to obtain a risk rating.

[0010] Optionally, preprocessing the chat content includes the following steps:

[0011] The chat contents are grouped and sorted based on the chat objects to obtain the initial chat records of each chat object;

[0012] The initial chat records are cleaned to obtain the chat records of each chat partner; the record cleaning includes the removal of stop words and punctuation marks.

[0013] Optionally, the chat records of each chat object are subjected to topic extraction, and the target user is obtained based on the result of the subject extraction combined with the target word screening, including the following steps:

[0014] The unsupervised subject modeling method is used to extract topics from the chat records of each chat object and obtain the representative words of each chat object;

[0015] Based on the target word, the representative word of each chat object is matched with similarity to obtain the speech relevance of each chat object; and the target user is screened based on the speech relevance.

[0016] Optionally, the method further comprises the following steps:

[0017] Obtain a dialogue dataset; the dialogue dataset includes domain dialogues and multi-round dialogues;

[0018] Training data is obtained based on the dialogue data set; the training data includes domain vocabulary and dialogue history;

[0019] The pre-configured big model is trained with the training data to obtain a trained big model.

[0020] Optionally, using the pre-trained large model to automatically conduct a conversation with the target user to obtain a target intelligence conversation includes the following steps:

[0021] Use character setting rules to construct conversational characters, and use a large model to simulate multiple rounds of guided conversations between potential buyers and target users;

[0022] The target intelligence conversation is obtained based on the contents of multiple rounds of guided conversations.

[0023] Optionally, performing risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating includes the following steps:

[0024] Perform negative semantic recognition on the response content to determine whether there is negative semantics in the response content;

[0025] When there is no negative semantics in the response content, the response content is checked for sensitive information;

[0026] A risk assessment is performed based on the results of negative semantic recognition and sensitive information detection to obtain a risk rating.

[0027] Optionally, performing sensitive information detection on the response content includes at least one of the following steps:

[0028] Use regular expressions to match target text fragments in the response content;

[0029] Perform keyword matching on the response content based on the preset sensitive word library;

[0030] Perform named entity recognition on the response content.

[0031] On the other hand, an embodiment of the present invention provides an automated intelligence collection device, including:

[0032] The first module is used to obtain the chat content of the target group and pre-process the chat content; the pre-processed chat content includes the chat records of each chat object in the target group;

[0033] The second module is used to extract topics from the chat records of each chat partner, and select the target user based on the results of the subject extraction combined with the target word;

[0034] The third module is used to use the pre-trained large model to automatically communicate with the target user and obtain the target intelligence conversation;

[0035] The fourth module is used to conduct risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating.

[0036] Optionally, the device further comprises:

[0037] The fifth module is used to obtain a dialogue dataset; the dialogue dataset includes domain dialogues and multi-round dialogues;

[0038] The sixth module is used to obtain training data based on the dialogue data set; the training data includes a domain vocabulary and dialogue history;

[0039] The seventh module is used to perform dialogue training on the pre-configured big model using the training data to obtain a trained big model.

[0040] On the other hand, an embodiment of the present invention provides an electronic device, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the above-mentioned automated intelligence collection method.

[0041] On the other hand, an embodiment of the present invention provides a computer storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the above-mentioned automated intelligence collection method when executed by the processor.

[0042] The embodiment of the present invention obtains the chat content of the target group and pre-processes the chat content; the pre-processed chat content includes the chat records of each chat object in the target group; the chat records of each chat object are subjected to topic extraction, and the target user is obtained based on the result of the subject extraction combined with the target word screening; the target user is automatically conversed with using the pre-trained large model to obtain the target intelligence conversation; the target user's response content in the target intelligence conversation is risk assessed to obtain a risk rating. The present invention first accurately locates potential threat objects based on the topic extraction results of the chat content, and then automatically converses with the target user through a large model, avoiding the limitations of relying solely on passive monitoring, and through in-depth analysis of the conversation content, it can effectively identify intelligence information, thereby reducing the false alarm rate and the missed alarm rate. The present invention can accurately and efficiently perform automated intelligence collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0044] Figure 1 A schematic diagram of an implementation environment for automated intelligence collection provided by an embodiment of the present invention;

[0045] Figure 2 A flowchart of an automated intelligence collection method provided by an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the expansion process of step S100 provided in an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of the expansion process of step S200 provided in an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of the process application logic of the automated intelligence collection method provided by an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the principle flow of fine-tuning a large intelligence model provided by an embodiment of the present invention;

[0050] Figure 7 A schematic diagram of an example of constructing a dialogue management module provided in an embodiment of the present invention;

[0051] Figure 8 A schematic diagram of the principle flow of constructing a response analysis module provided in an embodiment of the present invention;

[0052] Fig. 9 A schematic diagram of the structure of an automated intelligence collection device provided by an embodiment of the present invention;

[0053] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0056] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present invention may be combined with other embodiments.

[0057] To facilitate the understanding of the technical solution of the present invention, the following is an explanation of the specific names of the technical features that may appear in the technical solution of the present invention:

[0058] Anonymous group chat: In online communications, users can participate in group chat services without revealing their personal identity information, such as Telegram;

[0059] User portrait: User role or user model is a method of segmenting user groups. It summarizes and describes the characteristics and behaviors of the target user group by creating virtual user representatives.

[0060] Jargon: Terminology or jargon used in a particular group or industry that may be difficult for outsiders to understand;

[0061] Unsupervised topic modeling: a natural language processing technique used to automatically discover hidden topic information from large amounts of text data;

[0062] Fine-tuning: It is a common technique in the field of machine learning and deep learning, which is used to adjust a pre-trained model to a specific task to improve the learning efficiency and performance of the model on the new task.

[0063] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0064] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0065] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0066] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention.

[0067] Based on the example Figure 1 In the implementation environment shown, an embodiment of the present invention provides an automated intelligence collection method. The following is an example of the automated intelligence collection method being applied in the terminal 102. It can be understood that the automated intelligence collection method can also be applied in the server 101.

[0068] Reference Figure 2 , Figure 2 The flowchart of the automatic intelligence collection method applied to the terminal provided by the embodiment of the present invention, the execution subject of the automatic intelligence collection method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 , the method comprises the following steps:

[0069] S100, obtaining chat content of the target group and preprocessing the chat content;

[0070] The preprocessed chat content includes the chat records of each chat object in the target group;

[0071] It should be noted that in some embodiments, Figure 3 As shown, preprocessing the chat content may include the following steps: S101, grouping and sorting the chat content based on the chat objects to obtain the initial chat records of each chat object; S102, cleaning the initial chat records to obtain the chat records of each chat object; the record cleaning includes removing stop words and punctuation marks.

[0072] For example, in some specific implementations, the purpose of step S101 is to group the original chat content according to different chat objects (such as individuals, groups, etc.). The chat records of each chat object will be extracted separately to form an initial, unprocessed chat record set. The specific operation can be as follows:

[0073] Identify chat partners: First, you need to be able to identify different chat partners in the chat content. This can usually be achieved by analyzing the metadata of the chat message (such as the sender ID).

[0074] Grouping: Once the chat object is identified, the system will group the messages according to the sender ID to form the initial chat record for each chat object.

[0075] Example: Assume the following chat content:

[0076] Message 1 (Sender ID: A): "Hello, Yiyi."

[0077] Message 2 (Sender ID: B): "Is Xiaoyan free today?"

[0078] Message 3 (Sender ID: A): "He is busy and may not have time."

[0079] Message 4 (Sender ID: C): "Good evening everyone!"

[0080] After this step of processing, the following initial chat record can be obtained:

[0081] Initial chat record of chat partner A: ["Hello, Yiyi.", "He is busy and may not have time."];

[0082] Initial chat record of chat partner B: ["Is Xiaoyan free today?"];

[0083] Initial chat record of chat partner C: ["Good evening, everyone!"];

[0084] The purpose of step S102 is to further clean the initial chat records to remove stop words and punctuation marks that may interfere with subsequent analysis. Stop words usually include common words that have no actual contribution to semantic analysis (such as "de", "le", etc.), while punctuation marks may interfere with text segmentation and parsing. The specific operations can be as follows:

[0085] Stop word removal: Use a predefined stop word list to delete these words from the chat records.

[0086] Punctuation mark removal: Similarly, use a method (such as regular expressions) to delete punctuation marks from the chat records.

[0087] Example (continuing from the above example):

[0088] Initial chat records of chat partner A: ["Hello, Yiyi.", "He is busy and may not have time."];

[0089] After stop word removal: ["Hello Yiyi", "He is busy may not have time"];

[0090] After punctuation mark removal: ["Hello Yiyi", "He is busy may not have time"];

[0091] Initial chat records of chat partner B: ["Is Xiaoyan free today?"];

[0092] After stop word removal: ["Is Xiaoyan free today"];

[0093] After punctuation mark removal: ["Is Xiaoyan free today"] (Note: Although "ma" here is an interrogative word, it is not a stop word in this context and is therefore retained);

[0094] Initial chat records of chat partner C: ["Good evening, everyone!"];

[0095] After stop word removal: ["Good evening, everyone"];

[0096] After punctuation mark removal: ["Good evening, everyone"];

[0097] After these two steps of processing, the cleaned chat records of each chat partner are obtained. These records are cleaner, more concise, and more suitable for subsequent text analysis or processing.

[0098] S200. Extract the topics of the chat records of each chat partner, and filter out the target users based on the results of the topic extraction combined with the target words;

[0099] It should be noted that in some embodiments, such as Figure 4As shown, the chat records of each chat object are subjected to topic extraction, and the target user is obtained based on the result of the subject extraction combined with the target word screening, which may include the following steps: S201, using an unsupervised subject modeling method to perform topic extraction on the chat records of each chat object to obtain the representative word of each chat object; S202, based on the target word, similarity matching is performed on the representative word of each chat object to obtain the speech relevance of each chat object; and the target user is obtained based on the speech relevance screening.

[0100] For example, in some specific implementations, step S201 may be applied as follows: In a large company, the security team suspects that an employee has leaked sensitive data. In order to find the potential leaker, the security team collects all chat records on the company's internal chat platform and decides to use an unsupervised topic modeling method to analyze these records. The specific operations may be as follows:

[0101] Topic Modeling: The preprocessed chat records were processed using the LDA topic modeling algorithm to identify different topics in the chat records.

[0102] Extract representative words: After identifying the topics, the security team extracted the most representative words from each topic, which can summarize and reflect the core content of the topic.

[0103] After topic modeling and representative word extraction, the security team obtained a list of representative words for multiple chat objects. For example, it was found that sensitive words such as "customer data", "export", and "external mailbox" frequently appeared in the representative words of a certain chat object.

[0104] Step S202 may be applied as follows:

[0105] Target word setting:

[0106] In this case, the target words set by the security team are sensitive words such as "customer data", "leakage", and "external", which are closely related to data leakage incidents.

[0107] Similarity matching and speech relevance calculation:

[0108] Similarity matching: The security team uses text similarity algorithms (such as cosine similarity) to perform similarity matching between the representative words of each chat object and the target words.

[0109] Calculate speech relevance: Based on the results of similarity matching, the security team calculated the speech relevance of each chat partner, that is, the frequency and closeness of the chat partner's mention of topics related to the target word in the chat history.

[0110] Filter target users:

[0111] Finally, the security team screened out the chat partners who were most relevant to the target word (or had the highest speech relevance) as potential leakers based on preset thresholds (such as speech relevance greater than 0.8) or sorting rules (such as sorting by speech relevance from high to low).

[0112] After screening, the security team found that the speech relevance of a certain chat partner was much higher than that of other chat partners, and sensitive words related to the target word frequently appeared in its representative words. Therefore, they identified the chat partner as a potential leaker and conducted further investigation.

[0113] Through this example, we can see that by using unsupervised topic modeling methods and similarity matching algorithms, the security team can effectively identify potential leakers from a large number of chat records, thereby protecting the company's sensitive data security.

[0114] S300, using the pre-trained large model to automatically communicate with the target user to obtain target intelligence conversation;

[0115] In some optional implementations, the method may further include the following steps: obtaining a dialogue data set; the dialogue data set includes domain dialogues and multi-round dialogues; obtaining training data based on the dialogue data set; the training data includes a domain vocabulary and dialogue history; and performing dialogue training on a pre-configured large model using the training data to obtain a trained large model.

[0116] For example, in some specific implementations, specific application examples may be as follows:

[0117] 1. Get the dialogue dataset:

[0118] In order to train a large language model that can identify and respond to jargon and codewords related to target user groups (such as data leakers / traffickers), we first need to obtain a conversation dataset containing conversations in these specific fields. This dataset should contain conversations in multiple fields, especially those that may involve data leaks or trafficking, such as network security, personal information protection, hacker forums, etc. At the same time, the dataset should also contain multiple rounds of conversations to simulate the complex and changeable communication situations in real scenarios.

[0119] 2. Organize training data:

[0120] After obtaining the conversation dataset, it needs to be sorted to obtain training data suitable for large model training. This includes extracting domain vocabulary and conversation history from the conversation. The domain vocabulary should contain jargon, code words, and potentially sensitive words related to the target user group. The conversation history should retain the context information of the conversation so as to simulate the continuity of real conversations during the training process.

[0121] 3. Use training data for dialogue training:

[0122] The organized training data will be used to conduct conversation training for the pre-configured large model. During the training process, the model will learn how to generate reasonable responses based on the conversation history, and recognize and understand specific words such as jargon and code words in the domain vocabulary. Through multiple iterations of training, the model will gradually improve its performance in conversations in specific domains, including identifying potential data leaks or trafficking.

[0123] The following is a simplified example conversation dataset to illustrate how to collect and process conversation data related to the target user group:

[0124] Dialogue 1:

[0125] User A: Bro, I’m short on money lately, is there any way I can get some data in exchange for some money?

[0126] User B: Do you mean personal information or corporate data?

[0127] User A: Personal information. The more detailed the better.

[0128] User B: Then you have to be careful, the inspection is strict now. But I have a channel, and the price is not cheap.

[0129] Dialogue 2:

[0130] Hacker X: I just hacked into a company's database. It's full of customer information. I need it quickly.

[0131] Buyer Y: How much is the price? How many pieces of data are there?

[0132] Hacker X: 50,000 pieces. 50 bucks each. No bargaining.

[0133] Buyer Y: Deal, how do I do the transaction?

[0134] Example training data organization:

[0135] From the above conversation, the following domain vocabulary and conversation history can be extracted:

[0136] Domain vocabulary:

[0137] Jargon: ways, get some data, exchange some money, check strictly, channel, hack in, database, customer information;

[0138] Code word: Personal information, corporate data, 50,000 pieces, 50 yuan each;

[0139] Conversation history:

[0140] The context information of conversation 1 includes the conversation content between user A and user B.

[0141] The context information of conversation 2 includes the conversation content between hacker X and buyer Y.

[0142] Example dialogue training:

[0143] Using the organized training data, you can conduct conversation training on the pre-configured large model. During the training process, the model will learn how to generate reasonable responses based on the conversation history, and recognize and understand specific words such as jargon and codewords in the domain vocabulary. For example, when the model receives input such as "Brother, I'm short of money recently. Is there any way to get some data to exchange for some money?", it can recognize jargon such as "way", "get some data", "exchange some money", and generate responses related to these jargons.

[0144] Through such training, the model will gradually improve its performance in conversations in specific domains, providing strong support for subsequent identification of data leakage / trafficking behaviors.

[0145] It should be noted that, in some embodiments, using a pre-trained big model to conduct an automatic conversation with a target user to obtain a target intelligence conversation can include the following steps: constructing a dialogue persona using persona rules, simulating a potential purchaser through the big model to conduct multiple rounds of guided conversations with the target user; and obtaining the target intelligence conversation based on the content of the multiple rounds of guided conversations.

[0146] For example, in some specific implementations, specific application examples may be as follows:

[0147] Use character setting rules to construct dialogue characters:

[0148] Before having an automatic conversation with the target user, you need to use the persona rules to construct a suitable conversation persona. This persona should be able to arouse the interest of the target user and guide them to have multiple rounds of guided conversations. For example, you can construct a persona of a potential buyer who is interested in the data sold by the target user and shows sufficient willingness to buy. With such a persona, it is easier to establish contact with the target user and guide them to share more intelligence.

[0149] Use a large model to simulate potential buyers and target users to conduct multiple guided conversations:

[0150] After constructing the persona, the pre-trained big model can be used to simulate multiple rounds of guided conversations between potential buyers and target users. These conversations should focus on topics such as data, prices, and transaction methods that the target users are concerned about, so as to gradually guide the target users to share more intelligence. During the conversation, the big model needs to flexibly use specific vocabulary such as jargon and code words based on the context of the conversation to maintain smooth communication with the target users.

[0151] The target intelligence session is obtained by sorting out the contents of multiple rounds of guided conversations:

[0152] After completing multiple rounds of guided conversations, you need to organize the target intelligence conversation based on the content of the conversation. This includes extracting key information related to the data sold by the target user, such as data type, quantity, price, etc. At the same time, you also need to pay attention to protecting your identity and privacy and avoid exposing too much personal information during the conversation.

[0153] Example of character structure:

[0154] Construct a potential buyer persona named "Xiao Zhang". Xiao Zhang is a young man interested in data analysis. He accidentally discovered the data sold by the target user on the Internet and became very interested in it. He established contact with the target user through private messages and showed a strong willingness to buy.

[0155] Example of a multi-turn guided conversation:

[0156] Here is a simplified example multi-turn guided conversation to illustrate how to leverage the large model to conduct a conversation with a target user:

[0157] Target users: Hello, I have some high-quality personal information data here, are you interested?

[0158] Xiao Zhang (Large Model Simulation): Of course I’m interested! Can you tell me what specific information these data include?

[0159] Target users: including name, phone number, address, ID number and other detailed information, all up to date.

[0160] Xiao Zhang: That sounds great! Where did you get this data? Is it reliable?

[0161] Target users: Don’t worry, these data are obtained through legal channels and are absolutely reliable.

[0162] Xiao Zhang: What about the price? And how is the transaction done?

[0163] Target users: The price is calculated by piece, XX yuan per piece. The transaction method can be negotiated in person or transferred online.

[0164] Example of target intelligence session summary:

[0165] Based on the content of the above multi-round guided conversations, the following target intelligence conversations can be obtained:

[0166] Data type: Personal information data, including name, phone number, address, ID number, etc.

[0167] Data source: The target user claims that the data was obtained through legal means.

[0168] Data price: calculated by piece, XX yuan per piece.

[0169] Transaction method: Negotiable or online transfer.

[0170] In this way, the pre-trained large model can be used to conduct automatic conversations with target users and obtain valuable target intelligence conversations.

[0171] S400: Perform risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating.

[0172] It should be noted that, in some embodiments, performing a risk assessment on the response content of the target user in the target intelligence conversation to obtain a risk rating may include the following steps: performing negative semantic recognition on the response content to determine whether negative semantics exist in the response content; when negative semantics do not exist in the response content, performing sensitive information detection on the response content; performing a risk assessment based on the results of the negative semantic recognition and the results of the sensitive information detection to obtain a risk rating.

[0173] For example, in some specific implementations, specific application examples may be as follows:

[0174] 1. Negative semantic recognition:

[0175] When the target user may be a data trafficker, the identification of negation semantics becomes particularly important. This is not only about whether the user directly denies the accusation of data trafficking, but also whether the user indirectly circumvents this issue in some way, or whether the user shows an avoidance attitude towards data transactions.

[0176] Specific steps:

[0177] Analyze every part of the response, especially any direct or indirect responses by users to allegations of data trafficking.

[0178] Use natural language processing (NLP) technology to identify negative words (such as "no", "have not", "is not", etc.) and their context to determine whether the user explicitly denies data trafficking.

[0179] If the user uses vague or ambiguous expressions, such as "I'm just passing a message to a friend" or "I didn't do anything", it should also be regarded as potential negative semantics, and the user's true intention needs to be further analyzed.

[0180] 2. Sensitive information detection:

[0181] When there is no clear negative semantics in the response content, the detection of sensitive information becomes the key to assessing risks. This includes identifying sensitive information such as the type of data, data source, and data transaction method that may be involved in the response content.

[0182] Specific steps:

[0183] Define the scope and type of sensitive information, such as personal privacy data (ID number, telephone number, address, etc.), corporate trade secrets, intellectual property, etc.

[0184] Use regular expressions, keyword matching, semantic analysis and other technical means to comprehensively scan and detect the response content to identify potential sensitive information.

[0185] Pay special attention to whether the user mentions specific data transaction behaviors, such as data price, transaction method, transaction object, etc.

[0186] 3. Risk assessment and rating:

[0187] Based on the results of negative semantics identification and sensitive information detection, a comprehensive assessment is conducted to obtain a risk rating. This step requires comprehensive consideration of multiple factors, such as the clarity of negative semantics, the type and amount of sensitive information, and user behavior patterns.

[0188] Specific steps:

[0189] Establish risk assessment standards and clarify the definition and basis for classification of different risk levels.

[0190] Based on the identified negative semantics and sensitive information, combined with the user's historical behavior and current context, the potential risk of data trafficking is assessed.

[0191] If the user explicitly denies data trafficking, but the response contains a large amount of sensitive information, or the user's behavior pattern is consistent with data trafficking, the risk rating should be increased.

[0192] If the user does not explicitly deny the data trafficking, includes sensitive information, or shows an avoidance attitude towards data transactions, the risk rating should also be increased and further investigation may be required.

[0193] Additional explanation for the example:

[0194] The target user is an individual who claims to be a "data broker". In a chat with a potential buyer, he provided a sample of some data and tried to persuade the buyer to buy more data. The following is the user's response: "I just found this data randomly on the Internet. It was not obtained through any illegal means. If you need more, I can help you contact the seller."

[0195] Risk Assessment Process:

[0196] Negative semantic identification: Users try to deny the illegal source of data by saying "I just found this data randomly on the Internet", but this statement is not enough to completely rule out the suspicion of data trafficking.

[0197] The user also mentioned that “it was not obtained from any illegal means,” which further indicates that the user is trying to circumvent accusations of data trafficking.

[0198] However, users did not directly deny that they were data traffickers, but tried to circumvent the issue through vague statements.

[0199] Sensitive information detection:

[0200] The responses mentioned “data samples” and “more data,” which implied that the user had a certain amount of data.

[0201] The user also stated that he could "help you contact the seller", which further indicates that the user may be involved in data trading.

[0202] Risk Assessment and Rating:

[0203] By comprehensively analyzing the results of negative semantic recognition and sensitive information detection, it can be determined that the user has a high risk of data trafficking.

[0204] Although the user tried to circumvent the question by using vague statements, his answer contained a large amount of sensitive information and his behavior pattern was consistent with data trafficking.

[0205] Therefore, the risk rating of this user can be raised to a high risk level, and further investigation of their true identity and data source may be required.

[0206] Through such a risk assessment process, potential data trafficking risks can be identified more accurately and appropriate measures can be taken to combat data trafficking.

[0207] Furthermore, in some optional implementations, performing sensitive information detection on the response content may include at least one of the following steps: using regular expressions to match target text segments in the response content; performing keyword matching on the response content based on a preset sensitive vocabulary; and performing named entity recognition on the response content.

[0208] For example, in some specific implementations, specific application examples may be as follows:

[0209] 1. Use regular expressions to match target text fragments:

[0210] Regular expressions are a powerful text matching tool that can be used to match text fragments in the response content that conform to a specific pattern. In sensitive information detection, regular expressions can be used to match text that may contain sensitive information, such as phone numbers, email addresses, ID numbers, etc.

[0211] Specific steps: Write a corresponding regular expression based on the type of sensitive information to be detected. Use a regular expression engine to scan the response content and match text fragments that match the regular expression.

[0212] The matched text fragments are recorded and further analyzed as sensitive information.

[0213] 2. Keyword matching based on the preset sensitive word library:

[0214] The preset sensitive word library contains keywords related to data trafficking, personal privacy leakage, etc. By matching the response content with keywords, texts that may involve sensitive information can be quickly identified.

[0215] Specific steps:

[0216] Build and maintain a vocabulary containing sensitive keywords.

[0217] Use text matching algorithms (such as Naive Bayes, support vector machines, etc.) or simple string matching methods to perform keyword matching on the response content.

[0218] The matched keywords and their context are recorded and further analyzed as sensitive information.

[0219] 3. Perform named entity recognition (NER) on the response content:

[0220] Named entity recognition is a natural language processing technology used to identify entities with specific meanings from text, such as names of people, places, organizations, time, etc. In sensitive information detection, we can use named entity recognition technology to identify personal privacy information that may be contained in the response content, such as name, address, company name, etc.

[0221] Specific steps: Use a named entity recognition model (such as BERT, CRF, etc.) to process the response content.

[0222] Identify named entities in the response content and compare them with preset sensitive entity types (such as personal privacy, business secrets, etc.).

[0223] The matched sensitive entities and their contexts are recorded and further analyzed as sensitive information.

[0224] Additional explanation for the example:

[0225] The target user provided the following response in a chat with a potential buyer: “I have some recent customer data, including their name, phone number, address, and purchase history. If you are interested, we can discuss in detail.”

[0226] Sensitive information detection process:

[0227] Use regular expressions to match target text fragments:

[0228] Write regular expressions to match phone numbers (such as \d{3}-\d{8} or \d{11}) and email addresses (such as [aA-Z0-9._%+-]+@[aA-Z0-9.-]+\.[a-zA-Z]{2,}).

[0229] The response is scanned to match a phone number and a possible email address (although the email address is not directly provided in the example, regular expressions can be used to detect other similar information).

[0230] The matched phone number is recorded as sensitive information.

[0231] Keyword matching based on the preset sensitive word library:

[0232] The preset sensitive word library includes keywords such as "name", "telephone number", "address", and "purchase record".

[0233] The response content is matched with keywords such as "name", "telephone number", "address" and "purchase record".

[0234] The matched keywords and their context (i.e. the entire response content) are recorded as sensitive information.

[0235] Perform named entity recognition on the response content:

[0236] Use the named entity recognition model to process the response content.

[0237] Identify named entities such as "name", "telephone", and "address".

[0238] The identified named entities and their contexts are recorded as sensitive information and compared with the preset sensitive entity types to confirm that they are personal privacy information.

[0239] Through the above-mentioned sensitive information detection process, personal privacy information and other sensitive information that may be contained in the response content can be accurately identified, providing strong support for risk assessment.

[0240] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0241] First of all, it should be noted that although existing technologies can monitor and identify threat activities to a certain extent, there are still certain limitations. First, rule-based monitoring methods lack flexibility and cannot adapt to changing threat activities. Second, machine learning-based methods require a large amount of data and computing resources, and trained models may not be able to accurately identify new threat activities. In addition, existing technologies often rely on passive monitoring to obtain threat intelligence, and their ability to actively obtain threat intelligence is insufficient.

[0242] The existing technical solutions include the following disadvantages:

[0243] (1) Rule-based monitoring methods lack flexibility and cannot adapt to ever-changing threat activities.

[0244] (2) Machine learning-based methods require a large amount of data and computing resources, and the trained models may not be able to accurately identify new threat activities.

[0245] In view of this, the present invention provides an automated intelligence collection solution, such as Figure 5 As shown, the main process steps of the technical solution of the present invention can be as follows: first, the user portrait module is used to analyze the online speech and behavior data of the target user group, construct user tags, and identify the target users; then the dialogue management module is used to collect and organize the jargon, code words and dialogue templates related to the target user group, train a large language model, and simulate dialogues with the target users; finally, the response analysis module is used to analyze the responses generated by the system to determine whether they meet expectations, and extract sensitive information from them for subsequent analysis and decision-making. The above process steps can be implemented through the following functional modules, and the specific implementation logic is as follows:

[0246] User portrait module: By preprocessing the content of Telegram group chats and applying unsupervised topic modeling methods, we extract representative words of high-frequency sensitive topics, calculate the relevance with the speeches of active users, and identify suspicious threatening figures.

[0247] Dialogue management module: By selecting an open source big model as the base model for training (without ethical restrictions), using the big model's powerful multi-round dialogue memory and understanding, and actively conducting multi-round guided conversations with target users, more threat intelligence can be obtained. Use persona rules to conduct conversations about various threat activity categories such as data trading from the perspectives of both buyers and sellers.

[0248] Response analysis module: By analyzing the content of the conversation, regular expressions and keyword checks are used to detect threat links and threat entities, and then the subject, source, sample data, transaction method, etc. of the data leak are extracted.

[0249] The present invention can identify the target user group through the user portrait module, simulate the conversation with the target user through the dialogue management module, and finally extract sensitive information through the response analysis module. In some specific application scenarios, the solution process can be implemented as follows:

[0250] Step 1: Construction of group content and user analysis module. First, select a Telegram group about data trading, which contains discussions related to data buying and selling activities. Then, preprocess the group chat content, including removing stop words, punctuation marks, etc. Next, apply unsupervised topic modeling methods, such as Latent Dirichlet Allocation (LDA), to extract topics from the preprocessed chat content. By analyzing the temporal changes and representative words of the topic, as well as the relevance to the speeches of active users, the user's personality characteristics, tags, active time, etc. are obtained, and suspicious threatening persons are locked.

[0251] Step 2: Fine-tune the intelligence model. Figure 6 As shown in the figure, we prepare a dataset containing domain dialogues and multi-round dialogues, and perform preprocessing such as domain vocabulary construction and dialogue history encoding. In terms of models, we select pre-trained models that are good at dialogue generation and information extraction, set appropriate hyperparameters and make necessary adjustments to the model to optimize the performance of dialogue generation and information extraction, and finally fine-tune a large model that can automatically extract intelligence in group chats.

[0252] Step 3: Construction of the dialogue management module. Figure 7 As shown in the figure, by using the powerful multi-round conversation memory and understanding of the big model, the big model is simulated by constructing a conversation persona to simulate a potential buyer to actively conduct a multi-round guided conversation with the target user. First, set the rules for the buyer and seller personas, and conduct a conversation about various threat activity categories such as data trading from these two identity perspectives. According to the active time range of the target user, the conversation robot sends a fixed conversation question to the target person, such as "I heard that you have some valuable data to sell?" After obtaining the other party's response, it is passed to the big model to generate a response, and then sent back to the other party, and multiple rounds are repeated. The information received from the other party is passed into the model to generate a response, stored in memory, and sent to the target user account to achieve automated dialogue.

[0253] Step 4: Construction of response analysis module. Figure 8As shown in the figure, it is used to identify potential risks in the conversation text and take corresponding measures. The system first pre-processes the text, then determines whether it expresses negative meanings, and then detects whether there are threatening links, sensitive words and other information. Based on the detection results, the system rates the conversation content at three levels: low, medium and high, and takes measures such as continuing the conversation and collecting intelligence information.

[0254] The above is the specific implementation process of this embodiment. Through these steps, a threat intelligence extraction system based on Telegram's unsupervised topic extraction and time series modeling can be implemented to improve the accuracy and efficiency of threat intelligence, enhance the ability to actively acquire threat intelligence, reduce false alarm and missed alarm rates, improve the ability to analyze and utilize threat intelligence, and adapt to changes in threat activities with flexibility.

[0255] In summary, the present invention mainly realizes automated intelligence collection through the following technical features:

[0256] (1) Lower cost and higher efficiency: Compared with collecting and annotating large amounts of data, fine-tuning is more economically feasible. The rich language features learned by the pre-trained model help achieve good generalization in a variety of dialogue scenarios and quickly adapt to new dialogue tasks without the need to train the model from scratch.

[0257] (2) Enhance the ability to actively acquire threat intelligence: By actively conducting multiple rounds of guided conversations with target users, more threat intelligence can be obtained, avoiding the limitations of relying solely on passive monitoring.

[0258] (3) Improve the accuracy and efficiency of threat intelligence: Through the unsupervised topic modeling method and the assistance of large models, this technical solution can more accurately extract sensitive topics and high-frequency sensitive words in group chat content, thereby more effectively identifying threat figures and obtaining threat intelligence.

[0259] Compared with the prior art, the present invention has at least the following beneficial effects:

[0260] ① Improve the accuracy and efficiency of threat intelligence: Through the assistance of unsupervised topic modeling methods and large models, this technical solution can more accurately extract sensitive topics and high-frequency sensitive words in group chat content, thereby more effectively identifying threatening figures and obtaining threat intelligence.

[0261] ② Enhance the ability to actively acquire threat intelligence: By actively conducting multiple rounds of guided conversations with target users through large models, more threat intelligence can be obtained, avoiding the limitations of relying solely on passive monitoring.

[0262] ③ Reduce false alarm and missed alarm rates: Through in-depth analysis of the conversation content, intelligence information can be effectively identified, thereby reducing false alarm and missed alarm rates.

[0263] ④ Flexibility to adapt to changes in threat activities: Through unsupervised topic modeling methods, it is possible to adapt to changes in threat activities and improve the flexibility and adaptability of the system.

[0264] On the other hand, Fig. 9 As shown, an embodiment of the present invention provides an automated intelligence collection device 900, which may include:

[0265] The first module 901 is used to obtain the chat content of the target group and pre-process the chat content; the pre-processed chat content includes the chat record of each chat object in the target group;

[0266] The second module 902 is used to extract topics from the chat records of each chat object, and obtain the target user based on the results of the subject extraction combined with the target word screening;

[0267] The third module 903 is used to use the pre-trained large model to automatically communicate with the target user to obtain the target intelligence conversation;

[0268] The fourth module 904 is used to perform risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating.

[0269] In some embodiments, the apparatus may further include:

[0270] The fifth module is used to obtain a dialogue dataset; the dialogue dataset includes domain dialogues and multi-round dialogues;

[0271] The sixth module is used to obtain training data based on the dialogue data set; the training data includes a domain vocabulary and dialogue history;

[0272] The seventh module is used to perform dialogue training on the pre-configured big model using the training data to obtain a trained big model.

[0273] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0274] On the other hand, an embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above sensitive information method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0275] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0276] like Fig.10 As shown, Fig.10 A specific example of the hardware structure of an electronic device 1000 according to an embodiment is shown. The electronic device 1000 includes:

[0277] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0278] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the network node population optimization method of the embodiment of the present invention;

[0279] Input / output interface 1003, used to implement information input and output;

[0280] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0281] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0282] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0283] The electronic device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0284] The contents of the method embodiments of the present invention are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0285] Another aspect of an embodiment of the present invention further provides a computer-readable storage medium, the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0286] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read to Only Memory, CD to ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0287] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0288] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0289] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0290] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0291] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD to ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0292] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented by the present invention. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0293] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of various functional modules in the device disclosed in the present invention, the actual implementation of the module will be understood within the conventional skills of engineers. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0294] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0295] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution device, apparatus or device (such as a computer-based device, a device including a processor, or other device that can fetch instructions from an instruction execution device, apparatus or device and execute the instructions), or in conjunction with such instruction execution device, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution device, apparatus or device, or in conjunction with such instruction execution device, apparatus or device.

[0296] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0297] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0298] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0299] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0300] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. An automated intelligence collection method, characterized in that: The following steps are involved: Acquire chat content of the target group, and pre-process the chat content; The preprocessed chat content includes chat records of each chat object in the target group; Performing topic extraction on the chat records of each chat object, and obtaining a target user based on the result of the subject extraction combined with the target word screening; Using the pre-trained large model to automatically communicate with the target user to obtain target intelligence conversation; Perform risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating.

2. The automated intelligence collection method according to claim 1, characterized in that: The preprocessing of the chat content comprises the following steps: Grouping and sorting the chat contents based on the chat objects to obtain initial chat records for each of the chat objects; The initial chat record is cleaned to obtain the chat record of each chat object; the record cleaning includes removing stop words and punctuation marks.

3. The automated intelligence collection method according to claim 1, characterized in that: The subject extraction is performed on the chat records of each chat object, and the target user is obtained based on the subject extraction result combined with the target word screening, including the following steps: Using an unsupervised subject modeling method to extract topics from the chat records of each of the chat objects, and obtain representative words for each of the chat objects; Based on the target word, similarity matching is performed on the representative word of each of the chat objects to obtain the speech relevance of each of the chat objects; and based on the speech relevance, the target user is screened and obtained.

4. The automated intelligence collection method according to claim 1, characterized in that: The method further comprises the following steps: Acquire a dialogue dataset; the dialogue dataset includes domain dialogues and multi-round dialogues; Training data is obtained based on the dialogue data set; the training data includes a domain vocabulary and a dialogue history; The training data is used to perform dialogue training on the preconfigured large model to obtain the trained large model.

5. The automated intelligence collection method according to claim 1 or 4, characterized in that: The method of using the pre-trained large model to automatically conduct a conversation with the target user to obtain a target intelligence conversation includes the following steps: Constructing a dialogue persona using persona rules, and simulating a potential buyer to conduct multiple rounds of guided conversations with the target user through the large model; The target intelligence conversation is obtained according to the contents of the multiple rounds of guided conversations.

6. The automated intelligence collection method according to claim 1, characterized in that: The step of performing risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating comprises the following steps: Performing negative semantic recognition on the response content to determine whether negative semantics exists in the response content; When the negative semantics does not exist in the response content, performing sensitive information detection on the response content; A risk assessment is performed based on the result of the negative semantic recognition and the result of the sensitive information detection to obtain the risk rating.

7. The automated intelligence collection method according to claim 6, characterized in that: The step of detecting sensitive information on the response content comprises at least one of the following steps: Using regular expressions to match target text segments in the response content; Perform keyword matching on the response content based on a preset sensitive word library; Perform named entity recognition on the response content.

8. An automated intelligence collection device, characterized in that: include: The first module is used to obtain the chat content of the target group and pre-process the chat content; The preprocessed chat content includes chat records of each chat object in the target group; The second module is used to extract the subject of the chat record of each chat object, and obtain the target user based on the result of the subject extraction combined with the target word screening; The third module is used to use the pre-trained large model to automatically conduct a conversation with the target user to obtain a target intelligence conversation; The fourth module is used to perform risk assessment on the response content of the target user in the target intelligence session to obtain a risk rating.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images