Universal chat record backup storage method and system

Abstract

The invention relates to the technical field of data backup and storage, and discloses a universal chatting record backup and storage method and system.The universal chatting record backup and storage system comprises a multi-source data acquisition unit, an information storage unit, an operation recording unit and a supplementary acquisition unit. One-stop efficient acquisition is realized, and the data processing efficiency is improved; classified storage and structured processing are carried out according to data characteristics, original data storage and efficient query analysis are considered, and data structure changes are flexibly coped with; the operation recording unit completely records the whole operation process by using a screen recording and Hash algorithm, ensures that the operation record is tamper-proof, and provides authenticity guarantee for the chat record as evidence in scenes such as commercial disputes and lawsuit; the supplementary acquisition unit breaks through the limitation of a data interface by means of technologies such as screenshot and optical character recognition, realizes complete acquisition of chat records of a complex communication platform, and enhances the universality and adaptability of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of data backup and storage, and in particular to a universal chat record backup and storage method and system. Background Art

[0002] With the rapid development of instant messaging technology, various instant messaging platforms such as WeChat, QQ, DingTalk, and WhatsApp have been widely used in scenarios such as personal social interaction, business communication, and team collaboration. Users have accumulated a large amount of valuable chat records on different platforms. However, existing chat record storage methods have many problems:

[0003] On the one hand, the data formats and storage structures of various instant messaging platforms are independent and significantly different. For example, WeChat chat logs may be stored in a specific format, while QQ chat logs are stored in a different format. This makes it difficult to uniformly manage and back up chat logs across different platforms. Traditional single backup methods are designed only for a specific platform and cannot meet the needs of integrated chat log backup across multiple platforms. To back up chat logs across multiple platforms, users must manually perform this operation on each platform separately, which is extremely inefficient and prone to missing critical information.

[0004] On the other hand, existing chat history storage systems lack flexibility and efficiency in data storage and management. Some systems use a single database to store all chat history, failing to fully leverage the strengths of different database types. For example, relational databases struggle to efficiently store and process raw, semi-structured chat history; while unstructured databases struggle to meet the requirements for fast query and statistics for structured data analysis. Furthermore, for frequently changing data, such as friend lists, traditional storage methods suffer from inefficiencies during data updates and maintenance, and difficulties ensuring data consistency.

[0005] Furthermore, existing technologies also have shortcomings in data security and evidence retention. Chat logs can serve as crucial evidence in commercial disputes and legal proceedings, but existing chat log storage systems lack effective operational record-keeping and evidence retention mechanisms, making it difficult to guarantee the integrity and authenticity of chat logs and meet legal compliance verification requirements. Furthermore, some instant messaging platforms lack standard data collection interfaces, making it impossible to access chat logs through conventional means, further limiting the comprehensiveness and versatility of chat log backup and storage. Summary of the Invention

[0006] The purpose of the present invention is to provide a universal chat record backup storage method and system, which solves the technical problems raised in the background technology.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A universal chat record backup and storage method and system, comprising:

[0009] Multi-source data acquisition unit: used to collect chat record data from multiple heterogeneous instant messaging platforms and perform data standardization processing to convert chat records from different sources into a unified format specification; chat record extraction method: using independent plug-ins for different communication platforms to extract chat records, extracting messages corresponding to text, images, and voice; unified data format: converting chat records from different sources into a unified format, and then transmitting them to the information storage unit through a data transmission protocol;

[0010] Information storage unit: adopts a multi-database hybrid storage architecture, combining the characteristics of different databases to store chat records;

[0011] The storage method is a hybrid storage solution of MongoDB+ClickHouse+MySQL;

[0012] Among them, MongoDB refers to a document-based database used to store raw, semi-structured, and unstructured data; ClickHouse refers to a column-based storage database used to store structured data; MySQL refers to a relational database used to store data with easily changeable structures;

[0013] Operation recording unit: used to record program operations in detail and use screen recording to preserve key operation evidence; recorded program operations include: recording the operation type, time, chat record identifiers involved, and the person who performed the operation in the relevant operations of obtaining and saving chat records;

[0014] Supplementary collection unit: used to use screenshots and image analysis technology to realize data collection from communication platforms that are not yet supported by the multi-source data acquisition unit; specifically, when the data interface of the target communication platform is incompatible with the standard collection protocol, this unit will start the auxiliary collection process to ensure the complete acquisition of chat records.

[0015] As a further solution of the present invention: the specific method of extracting chat records is as follows:

[0016] Step 1.1, data format identification:

[0017] First check the extension of the chat history file;

[0018] If the file extension is ".json", the chat history file is in JSON format;

[0019] If the file extension is ".xml", the chat history file is in XML format;

[0020] If the file extension is ".csv", the chat log file is in CSV format;

[0021] Step 1.2, different format analysis:

[0022] Step 1.2.1, JSON format parsing:

[0023] Loading files: Using the JSON parser, load the JSON file content into memory to form an operable data structure;

[0024] Traversing the structure: If the loaded data structure is an array, loop through each element in the array, where each element represents a chat record;

[0025] If the loaded data structure is an object, directly operate on each attribute in the object;

[0026] During the traversal process, the corresponding attribute values are extracted based on the known chat record structure;

[0027] Among them, the chat record structure contains fields corresponding to the sender, receiver, message content, and time;

[0028] Step 1.2.2, XML format parsing:

[0029] Construct a parse tree: Use XML parsing tools to construct a parse tree from the XML file. The parse tree uses each tag in the XML file as a node to show the hierarchical structure of the data.

[0030] Node location and information extraction: Using XPath expressions or other node search methods, locate the node containing chat history information on the parse tree;

[0031] Step 1.2.3, CSV format parsing:

[0032] Read line by line: read the CSV file content line by line;

[0033] Since CSV files are stored in text format, each line represents a record or part of a record. A text reading tool is used to obtain data line by line in sequence.

[0034] Field splitting: according to the delimiter of the CSV file;

[0035] The delimiter is determined through preliminary observation or configuration of the communication platform and is used to split each row of data into different fields.

[0036] As a further solution of the present invention: the data format is unified in this way: by pre-setting a standard data structure, including necessary fields corresponding to "sender", "receiver", "message content", "timestamp" and "message type", and then extracting corresponding information from records of different formats, and mapping and filling them according to this standard data structure.

[0037] As a further solution of the present invention: the specific storage method of the information storage unit is as follows:

[0038] Step 2.1: Store the original chat records received from the multi-source data acquisition unit in MongoDB;

[0039] In MongoDB, each chat record is considered a document;

[0040] Step 2.2: Read the original chat history from MongoDB; then extract the data structure and obtain the structured information in the original chat history, and then store the structured information in ClickHouse;

[0041] Among them, structured information includes sender, receiver, time, and keywords;

[0042] The data structure is extracted as follows:

[0043] Step 2.2.1, word segmentation operation:

[0044] For the text message content in the original chat history, use word segmentation technology to split it into single words or phrases;

[0045] Step 2.2.2, part-of-speech tagging:

[0046] Based on the word segmentation, each word is tagged with the part of speech to determine whether it is a noun, verb, or adjective;

[0047] Step 2.2.3, key information extraction:

[0048] Extract key information from the results of word segmentation and part-of-speech tagging based on the characteristics of the chat records and analysis requirements;

[0049] Step 2.3: For data with easily changeable structure, store it in a table pre-created in MySQL.

[0050] As a further solution of the present invention: the evidence fixing method is as follows:

[0051] For chat records that need to be preserved as evidence, start the screen recording program to record the chat record operation process and save it in the log file. The screen recording area is the designated chat window;

[0052] At the same time, the SHA-256 hash algorithm is used to calculate the hash value of the log file generated by each operation, and the hash value is stored together with the log file;

[0053] When verifying whether a log file has been tampered with, recalculate the log file's hash value and compare it with the original hash value:

[0054] When the comparison results are consistent, it is determined that the log file has not been tampered with;

[0055] If the comparison results are inconsistent, it is determined that the log file has been tampered with.

[0056] As a further solution of the present invention, the data collection method of the supplementary collection unit is as follows:

[0057] Step 3.1, screenshot acquisition:

[0058] For a specified chat window, a screenshot is obtained using the screenshot function of the communication platform, and the screenshot is then saved as an image file;

[0059] Step 3.2, Image Analysis:

[0060] Use optical character recognition technology to identify the text content in the image, and then analyze the recognized text based on a pre-trained machine learning model to determine the sender, recipient, and chat content;

[0061] Step 3.3, information forwarding:

[0062] The extracted information corresponding to the sender, receiver, and chat content is converted according to the unified format specification in the multi-source data acquisition unit, and forwarded to the information storage unit for storage according to the data transmission protocol.

[0063] As a further solution of the present invention: the parsing method of identifying text is as follows:

[0064] Collect a large number of chat record samples and label each sample with the sender, recipient, and chat content. Then use this labeled data to train a machine learning model.

[0065] The machine learning model uses either a recurrent neural network or a long short-term memory network;

[0066] The recognized text content of the image is input into a trained machine learning model for prediction, and the machine learning model outputs the prediction results of the sender, recipient and chat content.

[0067] A universal chat record backup and storage method is implemented by a universal chat record backup and storage system, and the method comprises the following steps:

[0068] Multi-source data acquisition: Collect chat history data from multiple heterogeneous instant messaging platforms and perform data standardization to convert chat history from different sources into a unified data format;

[0069] Supplementary collection: Using screenshots and image analysis technology to achieve data collection for communication platforms that are not yet supported by the multi-source data acquisition unit;

[0070] Information preservation: Using a multi-database hybrid storage architecture, combined with the characteristics of different databases, to store chat records;

[0071] Operation records: Record program operations in detail and use screen recording to capture key operation evidence.

[0072] Beneficial effects of the present invention:

[0073] Efficient Multi-Source Data Processing: The multi-source data acquisition unit uses independent plug-ins for different communication platforms to extract chat logs, covering text, images, voice, and other messages. Through the system's data format recognition, parsing, and unified processes, it can convert data in different formats such as JSON, XML, and CSV into a standard structure. This makes the system compatible with multiple heterogeneous instant messaging platforms, eliminating the previous inefficient method of manually backing up chat logs on each platform. It achieves one-stop, efficient collection and standardized processing of chat logs from multiple platforms, greatly improving data processing efficiency and convenience.

[0074] Intelligent Storage Architecture Optimization: The information storage unit innovatively utilizes a hybrid storage solution of MongoDB, ClickHouse, and MySQL, categorizing and storing data based on its characteristics. MongoDB stores raw, semi-structured, and unstructured data, ClickHouse processes structured data, and MySQL handles data with easily volatile structures. Through word segmentation, part-of-speech tagging, and key information extraction, raw records are converted into structured information storage. This architecture leverages the strengths of each database, fully preserving raw data while facilitating efficient query, statistics, and analysis. It also flexibly responds to changes in data structure, ensuring efficient and flexible data storage management.

[0075] Reliable Operation Records and Evidence Preservation: The operation record unit records the entire process of chat record acquisition and storage, combining screen recording with the SHA-256 hash algorithm to record key information such as operation type and time, ensuring that the operation record is complete and tamper-proof. In scenarios such as commercial disputes and legal proceedings, it can effectively safeguard the authenticity and integrity of chat records as evidence, meet legal compliance verification requirements, and provide solid protection for the legal validity of chat records.

[0076] Comprehensive Data Collection Capabilities: A supplementary collection unit utilizes screenshots, optical character recognition, and machine learning technologies to capture complete chat logs through screenshot acquisition, image analysis, and information forwarding, even when the target communication platform's data interface is incompatible. This breaks through the traditional collection method's reliance on specific interfaces and significantly expands the scope of data collection, ensuring comprehensive chat log acquisition even across complex and diverse communication platforms, enhancing the system's versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The present invention will be further described below with reference to the accompanying drawings.

[0078] Figure 1 This is a system block diagram of a universal chat record backup storage method and system of the present invention.

[0079] Figure 2 It is a flow chart of a universal chat record backup and storage method and system of the present invention. DETAILED DESCRIPTION

[0080] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0081] Example 1

[0082] See also Figure 1 and Figure 2 As shown, the present invention is a universal chat record backup storage method and system, including:

[0083] Multi-source data acquisition unit: used to collect chat record data from multiple heterogeneous instant messaging platforms and perform data standardization processing to convert chat records from different sources into a unified format specification;

[0084] The chat history extraction method is: for different communication platforms, independent plug-ins are used to extract chat history and extract the corresponding messages of text, pictures and voice;

[0085] The unified data format is to convert the chat records from different sources into a unified format, and then transmit them to the information storage unit through a data transmission protocol; in this embodiment, the data transmission protocol adopts the TCP protocol;

[0086] The specific method is as follows:

[0087] Step 1.1, data format identification:

[0088] First, check the extension of the chat record file;

[0089] If the file extension is ".json", then the chat record file is in JSON format;

[0090] If the file extension is ".xml", then the chat record file is in XML format;

[0091] If the file extension is ".csv", then the chat record file is in CSV format;

[0092] Step 1.2, Parsing of different formats:

[0093] Step 1.2.1, JSON format parsing:

[0094] Load the file: Use a JSON parser to load the content of the JSON file into memory to form an operable data structure;

[0095] For example, load the content of a JSON file containing chat records in the form of an object or an array, which is convenient for subsequent information extraction;

[0096] Traverse the structure: If the loaded data structure is an array, loop through each element in the array, where each element represents a chat record;

[0097] If the loaded data structure is an object, directly operate on each property in the object;

[0098] During the traversal process, extract the corresponding property values according to the known chat record structure;

[0099] Among them, the chat record structure includes fields corresponding to the sender, receiver, message content, and time;

[0100] For example, from a record like {"sender":"Zhang San","receiver":"Li Si","content":"Hello","time":"2025-05-05 14:30:00"}, extract "Zhang San" as the sender, "Li Si" as the receiver, etc. information;

[0101] Among them, sender represents the message sender, receiver represents the message receiver, content represents the message content, and time represents the time;

[0102] Step 1.2.2, XML format parsing:

[0103] Build a parse tree: Use an XML parsing tool to build an XML file into a parse tree, where the parse tree uses each tag in the XML file as a node to show the hierarchical structure of the data;

[0104] For example, for an XML file containing chat records, after parsing, a tree structure is formed with chat as the root node, representing the entire chat session, record as the child node, representing a single chat record, and sender, receiver, content, etc. as the next level nodes;

[0105] Node location and information extraction: Using XPath expressions or other node search methods, locate the node containing chat history information on the parse tree;

[0106] For example, find all chat record nodes through record, and then extract specific information from each record node through paths such as record / sender and record / receiver;

[0107] Step 1.2.3, CSV format parsing:

[0108] Read line by line: read the CSV file content line by line;

[0109] Since CSV files are stored in text format, each line represents a record or part of a record. A text reading tool is used to obtain data line by line in sequence.

[0110] Field splitting: according to the delimiter of the CSV file;

[0111] The delimiter is determined through preliminary observation or configuration of the communication platform and is used to separate each row of data into different fields;

[0112] In this embodiment, the separator is a comma;

[0113] For example, for a line of data like "Zhang San, Li Si, Hello, 2025-05-05 14:30:00", it is split into four fields, "Zhang San", "Li Si", "Hello", and "2025-05-05 14:30:00", using commas as delimiters. These fields correspond to the sender, receiver, message content, and time, respectively.

[0114] The data format is unified by pre-setting a standard data structure that includes the necessary fields for "sender," "receiver," "message content," "timestamp," and "message type." The corresponding information is then extracted from records of different formats and mapped and filled in according to this standard data structure.

[0115] Information storage unit: adopts a multi-database hybrid storage strategy to store chat records to achieve efficient data management;

[0116] The storage method of the information storage unit is a hybrid storage solution of MongoDB+ClickHouse+MySQL;

[0117] Among them, MongoDB refers to a document-based database used to store raw, semi-structured, and unstructured data; ClickHouse refers to a column-based storage database used to store structured data; MySQL refers to a relational database used to store data with easily changeable structures;

[0118] The specific storage methods are as follows:

[0119] Step 2.1: Store the original chat records received from the multi-source data acquisition unit in MongoDB;

[0120] In MongoDB, each chat record is considered a document;

[0121] For example, the original data of a WeChat chat record contains information such as the message's unique identifier, message type, source, sender, receiver, content, and time, and is then inserted into a pre-established WeChat chat record collection in the form of a document.

[0122] Step 2.2: Read the original chat history from MongoDB; then extract the data structure and obtain the structured information in the original chat history, and then store the structured information in ClickHouse;

[0123] Among them, structured information includes sender, receiver, time, and keywords;

[0124] The data structure is extracted as follows:

[0125] Step 2.2.1, word segmentation operation:

[0126] For the text message content in the original chat history, use word segmentation technology to split it into single words or phrases;

[0127] For example, for the message "The weather is really nice today, perfect for going out for fun", we can segment it into "today", "weather", "really nice", "suitable", "go out", "play", etc. based on the dictionary's word segmentation method;

[0128] This word segmentation process facilitates subsequent more detailed analysis of text content, such as keyword search and semantic understanding;

[0129] Step 2.2.2, part-of-speech tagging:

[0130] Based on the word segmentation, each word is tagged with the part of speech to determine whether it is a noun, verb, or adjective;

[0131] For example, “today” is marked as a noun and “suitable” is marked as a verb;

[0132] Part-of-speech tagging can help us further understand the grammatical structure and semantic focus of the text. When analyzing chat records, we can perform statistics and analysis in different dimensions based on part-of-speech. For example, we can count the frequency of noun occurrences to understand the main issues involved in the chat.

[0133] Step 2.2.3, key information extraction:

[0134] Extract key information from the results of word segmentation and part-of-speech tagging based on the characteristics of the chat records and analysis requirements;

[0135] For example, extract words that represent time, place, people, etc.

[0136] In the message "Let's meet at the park tomorrow", "tomorrow" is time information, "park" is location information, and "we" is person-related information;

[0137] This key information is very important for subsequent structured storage and data analysis. For example, statistical analysis of the time dimension can be performed based on the extracted time information.

[0138] Step 2.3: For data with easily changeable structure, store it in a table created in MySQL.

[0139] For example, if the structure is easy to change the data into a friend list, then create a corresponding friend list form, which contains fields corresponding to user ID, friend ID, and friend name;

[0140] When a user adds or deletes a friend, the friend list form is modified through the MySQL update or delete operation;

[0141] The universal chat record backup storage method and system proposed in this embodiment can efficiently process chat records of multi-source heterogeneous instant messaging platforms. The multi-source data acquisition unit extracts chat records from different platforms through independent plug-ins, and then performs data standardization processing, which can unify data in different formats and facilitate subsequent storage and analysis. The TCP protocol is used for data transmission to ensure the reliability of data transmission. In terms of data parsing, detailed parsing methods are formulated for the three common formats of JSON, XML, and CSV, which can accurately extract key information from chat records. The information storage unit adopts a MongoDB+ClickHouse+MySQL hybrid storage strategy, and classifies and stores the data according to its characteristics, which not only meets the storage needs of the original data, but also can efficiently process structured data and data with easily changeable structures, thereby realizing efficient data management.

[0142] Example 2

[0143] As the second embodiment of the present invention, when the present application is implemented, compared with the first embodiment, the technical solution of this embodiment differs from that of the first embodiment only in that this embodiment further includes:

[0144] Operation recording unit: used to record program operations in detail and use screen recording to fix evidence, prevent data tampering, record all operation logs, and support legal compliance verification;

[0145] The recorded program operations include: in the operations related to obtaining and saving chat records, the operation type, time, the chat record identifier involved, and the person who performed the operation;

[0146] The evidence is fixed in the following ways:

[0147] For chat records that need to be preserved as evidence, start the screen recording program to record the chat record operation process and save it in the log file. The screen recording area is the designated chat window;

[0148] At the same time, the SHA-256 hash algorithm is used to calculate the hash value of the log file generated by each operation, and the hash value is stored together with the log file;

[0149] When verifying whether a log file has been tampered with, recalculate the log file's hash value and compare it with the original hash value:

[0150] When the comparison results are consistent, it is determined that the log file has not been tampered with;

[0151] If the comparison results are inconsistent, it is determined that the log file has been tampered with;

[0152] Compared to Example 1, this embodiment adds an operation recording unit. This unit can record program operations in detail, including information such as operation type, time, chat record identifiers involved, and the operator of the operation. At the same time, it uses screen recording to fix evidence and prevent data tampering. By recording the chat record operation process through screen recording and saving it as a log file, and verifying the log file with the SHA-256 hash algorithm, the integrity and authenticity of the operation record are ensured, providing strong support for legal compliance verification and enhancing the security and credibility of the system.

[0153] Example 3

[0154] As the third embodiment of the present invention, when the present application is specifically implemented, compared with the first and second embodiments, the technical solution of this embodiment is to combine the solutions of the first and second embodiments. The technical solution of this embodiment differs from the first and second embodiments only in that this embodiment also includes:

[0155] Supplementary Collection Unit: This unit utilizes screenshot and image analysis technology to enable data collection from communication platforms not yet supported by the Multi-Source Data Acquisition Unit. Specifically, when the target communication platform's data interface is incompatible with the standard acquisition protocol, this unit will initiate a supplementary collection process to ensure complete acquisition of chat records.

[0156] The data collection method of the supplementary collection unit is as follows:

[0157] Step 3.1, screenshot acquisition:

[0158] For a specified chat window, a screenshot is obtained using the screenshot function of the communication platform, and the screenshot is then saved as an image file;

[0159] Step 3.2, Image Analysis:

[0160] Use optical character recognition technology to identify the text content in the image, and then analyze the recognized text based on a pre-trained machine learning model to determine the sender, recipient, and chat content;

[0161] The parsing method for recognized text is as follows:

[0162] First, a large number of chat record samples are collected and the sender, recipient, and chat content are labeled for each sample. The labeled data is then used to train the machine learning model.

[0163] The machine learning model uses either a recurrent neural network or a long short-term memory network. The training methods of the recurrent neural network and the long short-term memory network are existing technologies and are not described in detail here.

[0164] The recognized text content in the image is input into a trained machine learning model for prediction, and the machine learning model outputs the prediction results of the sender, recipient, and chat content;

[0165] Step 3.3, information forwarding:

[0166] The extracted information corresponding to the sender, receiver, and chat content is converted according to the unified format specification in the multi-source data acquisition unit, and forwarded to the information storage unit for storage according to the data transmission protocol.

[0167] This embodiment adds a supplementary collection unit to the first and second embodiments. When the target communication platform's data interface isn't compatible with the standard collection protocol, the supplementary collection unit can utilize screenshot and image analysis techniques to collect data from communication platforms not yet supported by the multi-source data acquisition unit. Through steps such as screenshot acquisition, image analysis, and information forwarding, the system ensures complete capture of chat logs, further expanding its data collection scope and improving its versatility and applicability.

[0168] Example 4

[0169] As the fourth embodiment of the present invention, when this application is specifically implemented, compared with the first, second and third embodiments, the technical solution of this embodiment is to combine and implement the solutions of the above-mentioned first, second and third embodiments.

[0170] This embodiment combines the solutions of Embodiments 1, 2, and 3. It integrates functions such as multi-source data acquisition, data standardization, multi-database hybrid storage, operation logging and evidence fixation, and supplementary data collection. This not only enables efficient backup, storage, and management of multi-source heterogeneous chat records, but also ensures the security of operation records and the integrity of data collection. This comprehensively enhances the system's functionality and performance, and can meet more complex and stringent chat record backup and storage requirements.

[0171] The present invention also provides a universal chat record backup storage method, which is implemented by a universal chat record backup storage system, and the method includes the following steps:

[0172] Multi-source data acquisition: Collect chat history data from multiple heterogeneous instant messaging platforms and perform data standardization to convert chat history from different sources into a unified data format;

[0173] Supplementary collection: Using screenshots and image analysis technology to achieve data collection for communication platforms that are not yet supported by the multi-source data acquisition unit;

[0174] Information preservation: Using a multi-database hybrid storage architecture, combined with the characteristics of different databases, to store chat records;

[0175] Operation records: Record program operations in detail and use screen recording to capture key operation evidence.

[0176] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters and thresholds in the formulas are set by technicians in this field according to actual conditions.

[0177] It should be stated that all data collected in this application is collected with the user's consent and authorization, and the use of all data is legal and compliant, and the use and processing of data complies with the relevant laws, regulations and standards of the relevant regions.

[0178] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A universal chat record backup storage system, characterized in that: include: Multi-source data acquisition unit: used to collect chat record data from multiple heterogeneous instant messaging platforms and perform data standardization processing to convert chat records from different sources into a unified data format; Information storage unit: adopts a multi-database hybrid storage architecture, combining the characteristics of different databases to store chat records; Operation recording unit: used to record program operations in detail and use screen recording to capture key operation evidence; Supplementary acquisition unit: used to use screenshot and image analysis technology to realize data acquisition on communication platforms that are not yet supported by the multi-source data acquisition unit.

2. A universal chat record backup storage system according to claim 1, characterized in that: The chat history extraction method is as follows: Step 1.1, data format identification: First check the extension of the chat history file; If the file extension is ".json", the chat history file is in JSON format; If the file extension is ".xml", the chat log file is in XML format; If the file extension is ".csv", the chat log file is in CSV format; Step 1.2, different format analysis: Step 1.2.1, JSON format parsing: Loading files: Using the JSON parser, load the JSON file content into memory to form an operable data structure; Traversing the structure: If the loaded data structure is an array, loop through each element in the array, where each element represents a chat record; If the loaded data structure is an object, directly operate on each attribute in the object; During the traversal process, the corresponding attribute values are extracted based on the known chat record structure; Among them, the chat record structure contains fields corresponding to the sender, receiver, message content, and time; Step 1.2.2, XML format parsing: Construct a parse tree: Use XML parsing tools to construct a parse tree from the XML file. The parse tree uses each tag in the XML file as a node to show the hierarchical structure of the data. Node location and information extraction: Using XPath expressions or other node search methods, locate the node containing chat history information on the parse tree; Step 1.2.3, CSV format parsing: Read line by line: read the CSV file content line by line; Since CSV files are stored in text format, each line represents a record or part of a record. A text reading tool is used to obtain data line by line in sequence. Field splitting: according to the delimiter of the CSV file; The delimiter is determined through preliminary observation or configuration of the communication platform and is used to split each row of data into different fields.

3. A universal chat record backup storage system according to claim 1, characterized in that: The data format is unified in this way: by pre-setting a standard data structure, including the necessary fields corresponding to "sender", "receiver", "message content", "timestamp", and "message type", the corresponding information is then extracted from records of different formats and mapped and filled according to this standard data structure.

4. A universal chat record backup storage system according to claim 1, characterized in that: The multi-database hybrid storage architecture adopts MongoDB+ClickHouse+MySQL hybrid storage solution; Among them, MongoDB refers to a document-based database used to store raw, semi-structured, and unstructured data; ClickHouse refers to a column-based storage database used to store structured data; MySQL refers to a relational database used to store data with easily changeable structures.

5. A universal chat record backup storage system according to claim 4, characterized in that: The specific storage method of the information storage unit is as follows: Step 2.1: Store the original chat records received from the multi-source data acquisition unit in MongoDB. In MongoDB, each chat record is considered a document. Step 2.2: Read the original chat history from MongoDB; then extract the data structure and obtain the structured information in the original chat history, and then store the structured information in ClickHouse; Among them, structured information includes sender, receiver, time, and keywords; Step 2.3: For data with easily changeable structure, store it in a table pre-created in MySQL.

6. A universal chat record backup storage system according to claim 5, characterized in that: The data structure is extracted as follows: Step 2.2.1: Split the text message content in the original chat log into individual words or phrases using word segmentation technology; Step 2.2.2: Based on the word segmentation, perform part-of-speech tagging on each word to determine whether it is a noun, verb, or adjective; Step 2.2.3: Extract key information from the results of word segmentation and part-of-speech tagging based on the characteristics of the chat records and analysis requirements.

7. A universal chat record backup storage system according to claim 1, characterized in that: The recorded procedural operations include: in the operations related to obtaining and saving chat records, the operation type, time, the chat record identifier involved, and the person who performed the operation; the evidence is fixed in the following ways: For chat records that need to be preserved as evidence, start the screen recording program to record the chat record operation process and save it in the log file. The screen recording area is the designated chat window; At the same time, the SHA-256 hash algorithm is used to calculate the hash value of the log file generated by each operation, and the hash value is stored together with the log file; When verifying whether a log file has been tampered with, recalculate the log file's hash value and compare it with the original hash value: When the comparison results are consistent, it is determined that the log file has not been tampered with; If the comparison results are inconsistent, it is determined that the log file has been tampered with.

8. A universal chat record backup storage system according to claim 1, characterized in that: The data collection method of the supplementary collection unit is as follows: Step 3.1: For a specific chat window, obtain a chat screenshot using the screenshot function of the communication platform, and then save the chat screenshot as an image file; Step 3.2: Use optical character recognition technology to identify the text content in the image, and then parse the recognized text based on a pre-trained machine learning model to determine the sender, recipient, and chat content; Step 3.3: The extracted information of the sender, receiver, and chat content is converted according to the unified data format in the multi-source data acquisition unit, and forwarded to the information storage unit for storage according to the data transmission protocol.

9. A universal chat record backup storage system according to claim 8, characterized in that: The parsing method for recognized text is as follows: Collect a large number of chat record samples and label each sample with the sender, recipient, and chat content. Then use this labeled data to train a machine learning model, where the machine learning model uses either a recurrent neural network or a long short-term memory network. The recognized text content of the image is input into a trained machine learning model for prediction, and the machine learning model outputs the prediction results of the sender, recipient and chat content.

10. A universal chat record backup and storage method, the method being implemented by a universal chat record backup and storage system according to any one of claims 1 to 9, characterized in that: The method comprises the following steps: Multi-source data acquisition: Collect chat history data from multiple heterogeneous instant messaging platforms and perform data standardization to convert chat history from different sources into a unified data format; Supplementary collection: Using screenshot and image analysis technology, data collection can be achieved for communication platforms that are not yet supported by the multi-source data acquisition unit; Information preservation: Using a multi-database hybrid storage architecture, combined with the characteristics of different databases, to store chat records; Operation records: Record program operations in detail and use screen recording to capture key operation evidence.

Images