Chat record identifying and grouping method based on visual large model

Abstract

The invention belongs to the cross technical field of computer vision and natural language processing, and discloses a chat record identifying and grouping method based on a visual large model, which has extremely strong generalization ability and robustness: due to the adoption of the visual large model, the scheme does not depend on priori knowledge of specific application UI layout any more. The method can adapt to various different chat application interfaces, and even can maintain high recognition accuracy after the interface is updated. The method is realized through a technical means of replacing a hard rule with a visual large model. The visual large model can understand the global context of the image, so that head portraits and bubbles are accurately associated, truncated text lines are combined, complex message types are identified, and the problems of text disorder and affiliation errors in a traditional OCR scheme are effectively avoided. The method is realized through a technical means of utilizing the comprehensive reasoning capability of a multi-modal large model.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision and natural language processing, and particularly relates to a method for chat record recognition and grouping based on a large visual model. Background Technology

[0002] Chat log management and grouping technologies are currently primarily based on in-application text metadata. Mainstream instant messaging applications (such as WeChat, WhatsApp, and Telegram) generally offer search functionality based on contacts, group names, or keywords. Their technological foundation lies in querying and manipulating structured text databases. Furthermore, Optical Character Recognition (OCR) technologies, such as Tesseract, Baidu OCR, and Alibaba Cloud OCR, are mature and well-known technologies for converting text regions in images into machine-encoded text.

[0003] Existing technical solutions involve processing chat screenshots using traditional OCR technology combined with rule-based or simple machine learning algorithms. These solutions typically follow these steps: Step 1: Image Preprocessing. The input chat screenshot undergoes operations such as grayscale conversion, binarization, noise reduction, and perspective correction to improve the clarity of the text areas.

[0004] Step 2: Text Detection and Recognition. A general-purpose OCR engine is used to detect text lines or blocks in the preprocessed image (Text Detection) and recognize the text content (Text Recognition).

[0005] Step 3: Rule-based analysis. Based on the UI design rules of the specific application (such as bubble position, avatar size, timestamp format, and the color or position of the sender's name), write hard rules to distinguish different speakers and message content. For example, if a bubble is detected on the right side of the image, it is determined to be "your own message," and on the left side, it is "the other party's message."

[0006] Step 4: Simple Grouping. Based on the keywords extracted from the OCR results (such as "Project A" or "Meeting Notification"), the chat logs are categorized into preset categories using string matching or a simple classification model.

[0007] Based on the above analysis, the problems and shortcomings of the existing technology are as follows: (1) Poor generalization ability and reliance on prior rules: Existing solutions are highly dependent on rules designed for specific application UIs. Once the application is updated and the interface layout (such as bubble style and avatar position) changes, or when dealing with another completely redesigned chat application, all rules need to be rewritten and debugged manually, resulting in high adaptation costs and difficulty in scaling up applications.

[0008] (2) Weak contextual understanding and low recognition accuracy: Traditional OCR engines treat images as isolated collections of text lines, making it difficult to effectively understand the visual context of chat logs. For example, it may not be able to accurately associate text scattered across multiple lines with the same chat bubble, and it may not be able to distinguish which text belongs to which sender. In particular, when the interface elements are complex, there are multiple sessions or forwarded messages, text confusion and misjudgment are likely to occur.

[0009] (3) Crude grouping ability: The grouping method based on keyword matching is too mechanical and cannot understand the deep semantics of the dialogue. For content with the same meaning but different expressions (such as "meeting" and "conference"), or complex topics that require multiple sentences to understand, this method is not effective and leads to inaccurate grouping.

[0010] The technical problem this application aims to solve is: how to overcome the dependence of existing technologies on specific UI rules and improve the depth of contextual understanding and semantic understanding accuracy of chat screenshot content, so as to achieve more accurate and generalized chat record recognition and intelligent grouping. Summary of the Invention

[0011] To address the problems existing in the prior art, this invention provides a method for chat history recognition and grouping based on a large visual model.

[0012] This invention is implemented as follows: A method for chat history recognition and grouping based on a large visual model includes: S101: Obtain chat history image; The input can be a screenshot of a single chat window from a mobile device or desktop application, or a long screenshot containing multiple chat sessions. S102: Image preprocessing and standardization; S103: Visual-semantic joint parsing based on a large visual model; S104: Parse the model output and construct structured session data; S105: Intelligent grouping based on semantic embedding and clustering; S106: Output the grouping results.

[0013] Furthermore, the image preprocessing and standardization include scaling the image and segmenting long-screen screenshots to better suit the input requirements of large visual models.

[0014] Furthermore, the method involves performing joint visual-semantic parsing based on a large visual model. The preprocessed image is directly input into the pre-trained large visual model, and carefully designed prompts are used to guide the model to output the parsing results in a structured manner.

[0015] Furthermore, the parsing model outputs and constructs structured session data; it receives the text response returned by the visual big data model, parses, cleans, and verifies it, and constructs a structured list containing multiple messages.

[0016] Furthermore, the intelligent grouping based on semantic embedding and clustering is another core improvement. a. Semantic embedding: Using a natural language processing embedding model, the content field of all messages extracted in step S104 is converted into a high-dimensional vector; the position of these vectors in the vector space can represent their semantic information, and texts with similar semantics are closer in vector distance; b. Clustering: An unsupervised clustering algorithm is used to analyze the semantic vector set generated above; the algorithm will automatically aggregate messages that discuss the same or similar topics into several clusters. c. Topic Summary (Optional): For each generated cluster, the Large Language Model (LLM) can be used again to summarize all message content within the cluster and automatically generate a label representing the topic of the group.

[0017] Furthermore, the output grouping results; the final output not only includes the original message content and sender information, but also adds semantic-based grouping tags to each message, thereby realizing intelligent chat record archiving and management.

[0018] Another objective of this invention is to provide a chat log recognition and grouping system based on a large visual model, comprising: The image acquisition module is used to acquire chat history images; the input can be a screenshot of a single chat window of a mobile device or desktop application, or a long screenshot containing multiple chat sessions. The preprocessing module is used for image preprocessing and normalization; The parsing module is used for joint visual-semantic parsing based on a large visual model; Build modules are used to parse model output and construct structured session data; The grouping module is used to achieve intelligent grouping based on semantic embedding and clustering; The output module is used to output the grouping results.

[0019] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the chat history recognition and grouping method based on a large visual model.

[0020] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the chat log recognition and grouping method based on a large visual model.

[0021] Another objective of this invention is to provide an information data processing terminal, which includes the chat history recognition and grouping system based on a large visual model.

[0022] Based on the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solution to be protected by this invention from the following aspects: First, this invention possesses extremely strong generalization ability and robustness: Due to the adoption of a large visual model, this solution no longer relies on prior knowledge of specific application UI layouts. It can adapt to various chat application interfaces (such as WeChat, QQ, iMessage, Slack, etc.), and even maintain a high recognition accuracy after interface updates. This is achieved through the technical means of "replacing hard rules with a large visual model."

[0023] This invention boasts higher recognition accuracy and contextual integrity: the visual large model can understand the global context of an image, thereby accurately associating avatars with speech bubbles, merging truncated text lines, and recognizing complex message types, effectively avoiding text distortion and attribution errors in traditional OCR solutions. This is achieved through the technical means of "utilizing the comprehensive reasoning capabilities of a multimodal large model."

[0024] This invention features smarter and more accurate semantic grouping: by converting text content into semantic vectors and clustering them, this solution can group text based on the true meaning of the conversation, rather than mechanical keyword matching. This makes the grouping results more reflective of the user's true intent and the chat topic, improving the efficiency of information management. This is achieved through the technical means of "introducing semantic embedding and unsupervised clustering algorithms".

[0025] Secondly, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects: (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows: The technical solution of this invention successfully integrates cutting-edge Visual Large Model (VLLM) with Natural Language Processing (NLP) technology, creating a completely new solution with huge commercialization potential and expected benefits: A groundbreaking product form: It will give rise to a completely new software category—a cross-platform, intelligent unstructured dialogue data management tool. Whether it's compliance auditing and electronic evidence SaaS services for enterprises, or efficiency improvement tools for individual users, there is a clear willingness to pay and market potential.

[0026] Disruptive cost advantages: Compared with traditional solutions that require the development and maintenance of a separate set of parsing rules for each new app, this invention has strong generalization capabilities, which can significantly reduce the cost of technology adaptation and maintenance, and achieve economies of scale.

[0027] High-value data entry point: This technology is a key entry point for processing "dark data" scattered throughout images. By intelligently identifying and grouping these massive amounts of unstructured chat logs, it can provide high-quality structured data sources for subsequent big data analysis, business insights, and artificial intelligence training, thereby generating high-value-added business models.

[0028] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally: In existing technologies, processing chat log images mainly relies on "OCR + manual rules" or specific parsers for a single application. There is currently no universal solution in the industry, either domestically or internationally, that can deeply understand the visual context and deep semantics of chat logs without relying on predefined UI rules.

[0029] This invention is the first to propose and implement a technical path based on a large visual model as the core engine, which directly and jointly understands visual layout and text semantics from pixel-level images, and further achieves intelligent grouping through semantic embedding and clustering. This solution fills the gap in the complete technology chain from "machine vision recognition" to "dialogue semantic understanding" and then to "intelligent information management", establishing a brand-new technological paradigm.

[0030] (3) The technical solution of the present invention solves a technical problem that people have long wanted to solve but have never been able to solve successfully: This invention successfully solves a long-standing technical problem: how to efficiently, accurately, and automatically manage and understand chat log information stored in image formats across multiple application platforms.

[0031] People have long desired to be able to easily retrieve, categorize, and analyze information in chat screenshots, just like searching local text. However, this problem has remained unsolved for the following reasons: The industry has long been limited to improving OCR accuracy and writing more complex rules, which is a quantitative improvement that cannot fundamentally solve the problems of generalization and contextual understanding.

[0032] Seamlessly integrating visual understanding and semantic understanding requires the emerging and highly technical capability of multimodal large models, a technology that was previously neither mature nor effectively applied.

[0033] This invention, by adopting a disruptive technical approach and utilizing the inherent capabilities of large visual models, breaks through the bottleneck of traditional methods, achieving a qualitative leap from "cannot" to "can," and solving this long-standing technical problem.

[0034] (4) The technical solution of the present invention overcomes technical bias: In this technical field, the following technical biases have long existed: Bias 1: "Processing chat screenshots must first involve Optical Character Recognition (OCR) followed by semantic analysis." This is a common industry mindset, treating "visual" and "language" processing as two independent, sequential modules. This invention breaks this bias, creatively employing a large visual model for end-to-end joint visual-semantic parsing, combining the two steps into one. This approach transcends the limitations of traditional pipeline thinking, directly outputting structured semantic information and achieving unexpected technical results.

[0035] Bias 2: "To achieve cross-application compatibility, a custom parser must be developed for each application." This bias stems from reliance on traditional template matching and rule engines. This invention demonstrates that by leveraging a large visual model with powerful generalization capabilities, it is possible to break free from dependence on specific application UIs and use a unified model to handle ever-changing interfaces. This in itself is a fundamental subversion of traditional technical approaches.

[0036] Bias 3: "Complex end-to-end models are costly and unsuitable for such application scenarios." This invention, through ingenious prompt word engineering, delegates the complex tasks of visual understanding and structured output to a single model. Although the cost per call may be higher than traditional OCR, it significantly reduces the overall development, maintenance, and adaptation costs, and proves its feasibility and superiority from an overall business logic perspective, overcoming the technical bias regarding "cost-benefit." Attached Figure Description

[0037] Figure 1 This is a flowchart of a chat history recognition and grouping method based on a large visual model provided in an embodiment of the present invention.

[0038] Figure 2 This invention provides an intelligent grouping graph based on semantic embedding and clustering, as provided in the embodiments of the present invention.

[0039] Figure 3 This is a block diagram of a chat history recognition and grouping system based on a large visual model provided in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0041] like Figure 1 As shown in the figure, the chat history recognition and grouping method based on a large visual model provided by this invention includes the following steps: S101: Obtain chat history image; The input can be a screenshot of a single chat window from a mobile device or desktop application, or a long screenshot containing multiple chat sessions. S102: Image preprocessing and standardization; S103: Visual-semantic joint parsing based on a large visual model; S104: Parse the model output and construct structured session data; S105: Intelligent grouping based on semantic embedding and clustering; S106: Output the grouping results.

[0042] The image preprocessing and standardization provided in this embodiment of the invention includes operations such as image scaling and long-screen screenshot segmentation to make the image more suitable for the input requirements of the visual large model; this step is not necessary because the visual large model itself has powerful image understanding capabilities.

[0043] The embodiment of this invention provides a visual-semantic joint parsing based on a large visual model; this is the core improvement step of this invention; the preprocessed image is directly input into a pre-trained large visual model, and with the help of carefully designed prompts, the model is guided to output the parsing results in a structured manner; Example of prompt words: "You are a professional chat history analysis assistant; please analyze this chat screenshot in detail." 1. Identify each individual message bubble in the image; 2. For each bubble, please output it in JSON format: sender: [sender name or 'me'] timestamp: [Message sent time] content: [The complete text of the message] type: [Message type, such as 'text', 'image', 'link', 'file'] 3. Ensure all content is extracted completely; do not omit any information. Visual large models, with their powerful visual reasoning and natural language generation capabilities, can overcome the limitations of traditional OCR: Contextual understanding: It can understand visual layout, accurately associate text blocks with corresponding avatars and speech bubbles, and correctly attribute them to the sender; Multimodal information processing: It can not only recognize text, but also non-text elements such as images, emoticons, and file icons, and accurately describe their meaning or type; Powerful generalization capabilities: It can adapt to various chat applications with different UI designs without the need to write specific rules for each application.

[0044] The parsing model provided in this embodiment of the invention outputs and constructs structured session data; it receives text responses (usually JSON or similar structured data) returned by the visual large model, parses, cleans, and validates them, and constructs a structured list (ListofDictionaries) containing multiple messages.

[0045] like Figure 2 As shown, the embodiment of the present invention provides intelligent grouping based on semantic embedding and clustering; this is another core improvement. S201, Semantic Embedding: Using natural language processing embedding models (such as Sentence-BERT, OpenAI's text-embedding model), the content fields of all messages extracted in step S104 are converted into high-dimensional vectors (semantic vectors); the position of these vectors in the vector space can represent their semantic information, and texts with similar semantics are closer in vector distance. S202, Clustering: Unsupervised clustering algorithms (such as K-Means, DBSCAN, or hierarchical clustering) are used to analyze the semantic vector set generated above; the algorithm will automatically aggregate messages discussing the same or similar topics into several clusters. S203, Topic Summary (Optional): For each generated cluster, the Large Language Model (LLM) can be used again to summarize all the message content within the cluster and automatically generate a tag representing the topic of the group (e.g., "Project A progress discussion", "Weekend dinner arrangements").

[0046] The output grouping results provided by the embodiments of the present invention not only include the original message content, sender and other information, but also add semantic grouping tags to each message, thereby realizing intelligent chat record archiving and management.

[0047] like Figure 3 As shown in the figure, an embodiment of the present invention provides a chat history recognition and grouping system based on a large visual model, comprising: The image acquisition module is used to acquire chat history images; the input can be a screenshot of a single chat window of a mobile device or desktop application, or a long screenshot containing multiple chat sessions. The preprocessing module is used for image preprocessing and normalization; The parsing module is used for joint visual-semantic parsing based on a large visual model; Build modules are used to parse model output and construct structured session data; The grouping module is used to achieve intelligent grouping based on semantic embedding and clustering; The output module is used to output the grouping results.

[0048] This invention provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the chat history recognition and grouping method based on a large visual model.

[0049] This invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the chat history recognition and grouping method based on a large visual model.

[0050] This invention provides an information data processing terminal, which includes the chat history recognition and grouping system based on a large visual model.

[0051] I. Specific application areas or related products of this invention.

[0052] Example 1: Enterprise Compliance and Audit File Management System Application Background: Financial institutions, listed companies, and other regulated enterprises need to regularly audit the external work communication records of their employees and clients / partners (often conducted through instant messaging tools such as WeChat and DingTalk) to ensure compliance. These records are mostly archived as screenshots, are numerous, and manual review is extremely inefficient.

[0053] Problems with traditional methods: Auditors need to manually sift through massive amounts of screenshots, visually identify and categorize them, which is time-consuming, labor-intensive, and prone to omissions.

[0054] It cannot perform unified searches across platforms; for example, it cannot search for all chat logs involving "insider information" at once.

[0055] Relying on subjective human judgment for classification results in inconsistent standards and makes it difficult to guarantee audit quality.

[0056] Specific application of this invention in this product: Product form: A SaaS-based "intelligent chat log auditing platform" deployed on an enterprise intranet.

[0057] Implementation process: Employees or the audit department upload chat screenshots from different apps to the system in batches.

[0058] The system background calls the technical solution of the present invention: First, it uses a large visual model (such as GPT-4V) to parse all images and accurately extract the sender, time, and plain text content of each message (and identify non-text message types such as images and files).

[0059] Subsequently, the system uses an embedding model to convert all extracted text content into vectors and uses a clustering algorithm (such as DBSCAN) to automatically form multiple topic clusters.

[0060] The system automatically generates topic tags for each cluster (such as "contract approval", "expense reimbursement", "customer complaints") and allows auditors to manually define risk keywords (such as "kickbacks" and "leakage"). The system will automatically highlight and aggregate all relevant messages.

[0061] Beneficial effects: Efficiency Improvement: Reduces the manual review process from months to hours, increasing review efficiency by hundreds of times.

[0062] High audit quality: Based on semantic understanding and grouping, it avoids omissions in keyword matching and can discover more hidden compliance risks.

[0063] Standardization and Traceability: The audit process is automated and standardized, and all operations and grouping results are traceable, greatly enhancing the authority and credibility of the audit report.

[0064] Example 2: Personal Smart Photo Album and Knowledge Management App Application Background: Personal users' mobile phone photo albums contain a large number of chat screenshots, such as travel plans, product discussions, study notes, etc., which are difficult to retrieve and the information cannot be effectively utilized.

[0065] Problems with traditional methods: The phone's photo album can only search by the image itself or limited text tags, and cannot retrieve the specific conversation content within the screenshot.

[0066] Users need to manually create albums and categorize pictures, a tedious process.

[0067] Specific application of this invention in this product: Product form: A mobile app, serving as an enhancement plugin for the system's photo album or a standalone application.

[0068] Implementation process: After the user authorizes the app, it scans chat screenshots in the user's photo album.

[0069] The present invention can be implemented on the device or in the cloud using a lightweight version of the large visual model and semantic model.

[0070] After parsing, users can directly use natural language search within the app. For example, searching for "the Japanese restaurant that Xiao Zhang recommended last Wednesday" will allow the system to directly locate the message from the screenshot.

[0071] The system will automatically create smart albums such as "Project A", "Baby's Growth", and "Home Decor Inspiration", and automatically categorize related conversations in different screenshots into the same theme.

[0072] Beneficial effects: A revolution in information retrieval: It enables "content-level" second-level retrieval of chat screenshots, completely changing the way users manage fragmented information.

[0073] Automated organization: Eliminating the hassle of manual organization for users, the intelligent grouping function allows knowledge to be automatically accumulated and structured, greatly improving personal efficiency and digital life experience.

[0074] Example 3: Judicial and Electronic Evidence Analysis Tools Application Background: Chat screenshots are crucial electronic evidence in case investigations and judicial evidence collection. Evidence collectors need to quickly sift through the massive amounts of screenshots provided by the parties involved to extract relevant and valid information and establish a chain of evidence.

[0075] Problems with traditional methods: Evidence collection relies on manual browsing and screening, which is inefficient and may delay crucial cases.

[0076] Traditional OCR and human visual inspection are insufficient to distinguish maliciously forged screenshots (such as spliced ​​or tampered ones).

[0077] It is difficult to quickly understand the logical relationships and interpersonal relationships in complex dialogues.

[0078] Specific application of this invention in this product: Product form: A core analysis module in professional electronic forensics software.

[0079] Implementation process: The evidence collectors imported all the chat screenshots related to the case.

[0080] After extracting structured information using the technology of this invention, the system further utilizes the reasoning capabilities of a large model to perform in-depth analysis: Relationship graph construction: Automatically identify different senders and generate a relationship graph based on dialogue frequency and topic relevance.

[0081] Timeline reconstruction: Sort all cross-platform and cross-group messages by timestamp to reconstruct a complete event timeline.

[0082] Contradiction and Anomaly Detection: Through logical reasoning, identify potential time or content contradictions between different messages to help identify false evidence.

[0083] Beneficial effects: Improved efficiency and depth of evidence collection: It not only enables rapid extraction and classification of evidence, but also allows for in-depth data mining to uncover hidden connections and contradictions.

[0084] Enhancing the reliability of evidence: By using technical means to assist in identifying false evidence, the probative value of electronic evidence is improved.

[0085] Visual presentation: The generated relationship graphs and timelines allow complex case details to be presented to judges and juries in an intuitive way.

[0086] Conclusion: The three distinct application examples demonstrate that the technical solution of this invention possesses strong versatility and universality, capable of being concretized into a variety of competitive products, addressing long-standing technical pain points in different fields. This fully proves that this invention is not limited to theoretical concepts, but rather a highly practical and promising creative technical solution with significant and multifaceted technical value.

[0087] II. Evidence related to the technical effects obtained by the embodiments of the present invention.

[0088] To verify the effectiveness and superiority of the technical solution described in this invention, a comparative experiment was designed and conducted, and the method of this invention was quantitatively compared with the traditional "OCR + rule" method on several key indicators.

[0089] 1. Experimental Design Dataset: A test dataset containing 500 chat screenshots was constructed. These screenshots came from five different mainstream instant messaging applications (such as WeChat, DingTalk, QQ, Telegram, and WhatsApp), covering various scenarios including one-on-one chats, group chats, complex messages containing images / emojis / links, and long screenshots.

[0090] Each image in the dataset has been precisely labeled by humans, including the sender of each message, the content text, the ground truth of the timestamp, and the topic group label to which it belongs.

[0091] Comparison method: Baseline approach (comparative): The parsing pipeline is constructed using the industry-leading Tesseract OCR engine combined with position-based heuristics (such as determining the sender by the left or right position of the bubble on the screen).

[0092] The method of this invention (example): adopts a joint parsing scheme based on a large visual model (specifically using LLaVA-1.5), followed by the Sentence-BERT semantic embedding model and the K-Means clustering algorithm for grouping.

[0093] Evaluation indicators: Message-level text recognition accuracy: Number of correctly recognized message texts / Total number of messages.

[0094] Sender attribution accuracy: The number of messages whose senders (yourself / the other party / specific contact person) were correctly identified / the total number of messages.

[0095] Topic grouping normalized mutual information: evaluates the consistency between automatic grouping results and manually labeled groupings, with a value range of [0,1]. A higher value indicates a better grouping effect.

[0096] 2. Experimental Results and Data Analysis Table 1 Comparison of Message Recognition and Parsing Accuracy

[0097] Data Analysis: Text recognition: Traditional OCR methods have high error rates when the background is complex, the font is special, or the text is slanted. However, the large visual model used in this invention can understand the context and can complete the text based on semantics even if some text is blurred, thus significantly improving the accuracy by ~17.7%.

[0098] Sender attribution: Traditional methods are prone to misjudgment when faced with non-standard layouts or group chats. The method of this invention makes a judgment by comprehensively understanding the overall visual relationship between the avatar, nickname, and speech bubble, resulting in higher accuracy and greater stability.

[0099] Generalization ability: Traditional methods exhibit huge performance fluctuations across different applications (variance as high as 25.7%), while the method of this invention has stable performance (variance of only 5.3%), which strongly demonstrates its excellent generalization ability and no longer depends on the UI rules of specific applications.

[0100] Table 2. Subject Group Quality Assessment (NMI Score)

[0101] Data Analysis: Normalized mutual information (NMI) score directly reflects the consistency between the grouping results and the actual situation. The method of this invention achieves significantly higher NMI scores than traditional methods in both simple and complex scenarios, with an average improvement of up to 57.5%.

[0102] This proves that the method of the present invention can effectively understand the deep semantics of dialogues, and can clearly distinguish messages of different topics even in complex dialogues where topics are intertwined, thus solving the core pain points of traditional methods such as coarse grouping and low accuracy.

[0103] Conclusion: Through the above-mentioned carefully designed comparative experiments, quantitative data analysis, and simulation results, it can be clearly demonstrated that the technical solution of the present invention is significantly superior to the closest existing technology in several core technologies such as recognition accuracy, generalization ability, and semantic grouping accuracy, achieving unexpected technical effects and fully demonstrating the creativity and technical value of the present invention.

[0104] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for chat history recognition and grouping based on a large visual model, characterized in that, The chat history recognition and grouping method based on a large visual model includes the following steps: S101: Obtain chat history image; The input is a screenshot of a single chat window from a mobile device or desktop application, or a long screenshot containing multiple chat sessions. S102: Image preprocessing and standardization; S103: Visual-semantic joint parsing based on a large visual model; S104: Parse the model output and construct structured session data; S105: Intelligent grouping based on semantic embedding and clustering; S106: Output the grouping results.

2. The chat history recognition and grouping method based on a large visual model as described in claim 1, characterized in that, The image preprocessing and standardization include scaling and segmenting the image into long-screen screenshots to better suit the input requirements of large visual models.

3. The chat history recognition and grouping method based on a large visual model as described in claim 1, characterized in that, The method involves performing joint visual-semantic parsing based on a large visual model. The preprocessed image is directly input into the pre-trained large visual model, and carefully designed prompts guide the model to output the parsing results in a structured manner.

4. The chat history recognition and grouping method based on a large visual model as described in claim 1, characterized in that, The parsing model outputs and constructs structured session data; it receives text responses returned by the large visual model, parses, cleans, and validates them, and constructs a structured list containing multiple messages.

5. The chat history recognition and grouping method based on a large visual model as described in claim 1, characterized in that, The intelligent grouping is achieved based on semantic embedding and clustering; a. Semantic embedding: Using a natural language processing embedding model, the content field of all messages extracted in step S104 is converted into a high-dimensional vector; the position of these vectors in the vector space can represent their semantic information, and texts with similar semantics are closer in vector distance; b. Clustering: An unsupervised clustering algorithm is used to analyze the semantic vector set generated above; the algorithm will automatically aggregate messages discussing the same or similar topics into several clusters. c. Topic Summary Optional: For each generated cluster, the Large Language Model (LLM) can be used again to summarize all message content within the cluster and automatically generate a tag representing the topic of the group.

6. The chat history recognition and grouping method based on a large visual model as described in claim 1, characterized in that, The output grouping results; the final output not only includes the original message content and sender information, but also adds semantic-based grouping tags to each message, thereby realizing intelligent chat record archiving and management.

7. A chat log recognition and grouping system based on a large visual model, implementing the chat log recognition and grouping method based on any one of claims 1-6, characterized in that, include: The image acquisition module is used to acquire images from chat logs. The input is a screenshot of a single chat window from a mobile device or desktop application, or a long screenshot containing multiple chat sessions. The preprocessing module is used for image preprocessing and normalization; The parsing module is used for joint visual-semantic parsing based on a large visual model; Build modules are used to parse model output and construct structured session data; The grouping module is used to achieve intelligent grouping based on semantic embedding and clustering; The output module is used to output the grouping results.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the chat log recognition and grouping method based on a large visual model as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the chat log recognition and grouping method based on a large visual model as described in any one of claims 1-6.

10. An information data processing terminal, characterized in that, The information data processing terminal includes the chat history recognition and grouping system based on a large visual model as described in claim 7.

Images