A method and system for judging WeChat chat screenshots

Abstract

The present application discloses a method and system for judging WeChat chat screenshots. By inputting a picture to be detected into a WeChat chat screenshot classification model, the WeChat chat screenshot classification model judges whether the picture to be detected is a chat screenshot; if so, the text features in the picture to be detected are matched and compared with the text features in the precise matching rule library. If the match fails, the picture to be detected is input into a written language and spoken language classification model, and the written language and spoken language classification model uses text classification technology to analyze and judge the text in the picture to be detected; if the judgment result is that the text of the picture to be detected is spoken language, the picture is confirmed to be a WeChat chat screenshot. This solution uses image classification, target detection and text classification technology, and through data collection, model training and rule establishment and other steps, it can effectively solve the problems existing in the prior art and improve the accuracy and judgment ability of WeChat chat screenshots.

Description

Technical Field

[0001] The present application relates to the technical field of judging WeChat chat screenshots, and in particular to a method and system for judging WeChat chat screenshots. Background Art

[0002] The existing methods for judging WeChat chat screenshots include the following:

[0003] Image classification method: Collect WeChat chat screenshots and other pictures, organize and annotate them. Use classification network (CNN) architecture, such as VGG, ResNet, Inception, etc. to train classification models. Finally, use the classified model to predict whether a picture is a WeChat chat screenshot.

[0004] Object detection method: Collect WeChat chat screenshots and annotate key elements in the pictures, such as WeChat video, voice, emoticon icon button, mobile phone signal, and battery icon. Train the object detection model through CNN, YOLO and other networks and algorithms. Finally, use the model to detect the input image. If it contains the above elements of WeChat, it can be judged that the image is a WeChat chat screenshot.

[0005] Template matching method: Use the template matching function provided by OpenCV (such as matchTemplate) to match the image to be classified with the pre-prepared WeChat chat record template, and then classify it according to the matching results to determine whether the image is a WeChat chat screenshot.

[0006] The existing method for judging WeChat chat screenshots has the following defects:

[0007] In actual use, the image classification method may encounter an image type that it has not been trained for, resulting in misclassification as a WeChat chat screenshot. In addition, for images with partial occlusion or complex backgrounds, the model may have difficulty correctly identifying the target, resulting in classification errors.

[0008] Some chat images may not have the WeChat sending icon, battery level and other information, or the target may be too small, resulting in target detection failure, making it impossible to accurately determine whether it is a WeChat chat screenshot.

[0009] Images may contain complex scenes, diverse lighting conditions, occlusions, deformations and other factors, which increase the difficulty of image processing and make it difficult for traditional template matching algorithms to process accurately, resulting in matching failures.

[0010] In summary, the existing methods have certain limitations and shortcomings in judging WeChat chat screenshots. Summary of the invention

[0011] The present application provides a method, system, computer device and storage medium for judging WeChat chat screenshots, aiming to solve the problem that existing methods have certain limitations and shortcomings in judging WeChat chat screenshots.

[0012] In a first aspect, a method for determining a WeChat chat screenshot comprises:

[0013] Input the image to be detected into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the image to be detected is a chat screenshot;

[0014] If yes, the text features in the image to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, it is determined that the image to be detected is a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0015] If the match fails, the image to be detected is input into a written and spoken language classification model, and the written and spoken language classification model uses text classification technology to analyze and judge the text in the image to be detected;

[0016] If the judgment result is that the text of the image to be detected is spoken language, the image is confirmed to be a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0017] If the written language and spoken language classification model determines that it is written language, then the image to be detected is not a screenshot of a WeChat chat.

[0018] In the above solution, optionally, the WeChat chat screenshot classification model is established before the following steps are performed:

[0019] Collect no less than 10,000 WeChat chat screenshots covering a variety of scenes and situations;

[0020] Collect no less than 10,000 other types of pictures, including documents and images;

[0021] Collect spoken texts and written texts, the spoken texts are from social media, forum posts, and transcripts of audio recordings, and the written texts are from news and literature;

[0022] The WeChat chat screenshots, other types of pictures, spoken texts and written texts are cleaned, organized and labeled to ensure that the data format meets the requirements of model training.

[0023] In the above solution, the optional training of the WeChat chat screenshot classification model includes:

[0024] The collected WeChat chat screenshots and other types of pictures are divided into training sets and validation sets, using 80% of the data as the training set and 20% of the data as the validation set;

[0025] Select a target deep learning network model, which includes ResNet and YOLO, and set a target learning rate and optimizer parameters;

[0026] Input data and perform model training, continuously adjust parameters to improve the performance and accuracy of the model, and obtain the WeChat chat screenshot classification model.

[0027] In the above solution, optionally, the training text classification model includes:

[0028] Use the collected spoken and written text data to train the model using the FastText text classification tool;

[0029] Ensure that the format of the training data meets the requirements of FastText, where the format of the training data includes text content and labels.

[0030] In the above solution, optionally, the establishment of the WeChat chat screenshot accurate matching rule library includes:

[0031] Analyze various scenes and features in WeChat chat screenshots, and extract key information and features of the WeChat chat screenshots;

[0032] A dictionary library is established, which includes iconic texts of WeChat chats, including: "withdrawing a message" and "not being friends with other people in the group".

[0033] In a second aspect, a system for determining WeChat chat screenshots is provided, the system comprising:

[0034] The first judgment module is used to input the image to be detected into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the image to be detected is a chat screenshot;

[0035] Matching module: if the judgment is yes, then the text features in the image to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, it is determined that the image to be detected is a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0036] The second judgment module is used to judge that if the match fails, the image to be detected is input into the written and spoken language classification model, and the written and spoken language classification model uses text classification technology to analyze and judge the text in the image to be detected;

[0037] The first output module is used to determine that if the judgment result is that the text of the image to be detected is spoken language, then the image is confirmed to be a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0038] The second output module is used to determine that if the written language and spoken language classification model determines that it is written language, then the image to be detected is not a screenshot of a WeChat chat.

[0039] In the above solution, optionally, the WeChat chat screenshot classification model is established before the following steps are performed:

[0040] Collect no less than 10,000 WeChat chat screenshots covering a variety of scenes and situations;

[0041] Collect no less than 10,000 other types of pictures, including documents and images;

[0042] Collect spoken texts and written texts, the spoken texts are from social media, forum posts, and transcripts of audio recordings, and the written texts are from news and literature;

[0043] The WeChat chat screenshots, other types of pictures, spoken texts and written texts are cleaned, organized and labeled to ensure that the data format meets the requirements of model training.

[0044] In the above solution, the optional training of the WeChat chat screenshot classification model includes:

[0045] The collected WeChat chat screenshots and other types of pictures are divided into training sets and validation sets, using 80% of the data as the training set and 20% of the data as the validation set;

[0046] Select a target deep learning network model, which includes ResNet and YOLO, and set a target learning rate and optimizer parameters;

[0047] Input data and perform model training, continuously adjust parameters to improve the performance and accuracy of the model, and obtain the WeChat chat screenshot classification model.

[0048] In the above solution, optionally, the training text classification model includes:

[0049] Use the collected spoken and written text data to train the model using the FastText text classification tool;

[0050] Ensure that the format of the training data meets the requirements of FastText, where the format of the training data includes text content and labels.

[0051] In the above solution, optionally, the establishment of the WeChat chat screenshot accurate matching rule library includes:

[0052] Analyze various scenes and features in WeChat chat screenshots, and extract key information and features of the WeChat chat screenshots;

[0053] A dictionary library is established, which includes iconic texts of WeChat chats, including: "withdrawing a message" and "not being friends with other people in the group".

[0054] Compared with the prior art, this application has at least the following beneficial effects:

[0055] Based on further analysis and research on the existing technical problems, this application recognizes that the existing methods have certain limitations and shortcomings in judging WeChat chat screenshots. This solution inputs the image to be detected into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the image to be detected is a chat screenshot; if so, the text features in the image to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, it is determined that the image to be detected is a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot; if the match fails, the image to be detected is input into the book A facial language and spoken language classification model, wherein the written language and spoken language classification model uses text classification technology to analyze and judge the text in the image to be detected; if the judgment result is that the text of the image to be detected is spoken language, the image is confirmed to be a WeChat chat screenshot, and the output judgment result is a WeChat chat screenshot; if the written language and spoken language classification model determines that it is written language, the image to be detected is not a WeChat chat screenshot. By using image classification, target detection and text classification technology, through the steps of data collection, model training and rule establishment, the problems existing in the prior art can be effectively solved, and the accuracy and judgment ability of WeChat chat screenshots can be improved.

[0056] In the traditional image classification, detection and judgment of WeChat chat screenshots, only the characteristics of the image itself are focused on, and the text information on the image is ignored. The key point of this application is to add rule verification, text recognition and classification algorithms to the traditional image classification technology. WeChat chat screenshots contain text, which are all spoken or WeChat-specific words. By establishing a "spoken and written text classification model", the accuracy of WeChat chat screenshot image judgment is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a flow chart of a method for determining a WeChat chat screenshot provided by an embodiment of the present application;

[0058] Figure 2 A schematic diagram of the judgment flow of a method for judging WeChat chat screenshots provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] In one embodiment, Figure 1-2 As shown, a method for determining a WeChat chat screenshot is provided, comprising the following steps:

[0061] Input the image to be detected into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the image to be detected is a chat screenshot;

[0062] If yes, the text features in the image to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, it is determined that the image to be detected is a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0063] If the match fails, the image to be detected is input into a written and spoken language classification model, and the written and spoken language classification model uses text classification technology to analyze and judge the text in the image to be detected;

[0064] If the judgment result is that the text of the image to be detected is spoken language, the image is confirmed to be a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0065] If the written language and spoken language classification model determines that it is written language, then the image to be detected is not a screenshot of a WeChat chat.

[0066] In this embodiment, the WeChat chat screenshot classification model is established before the following steps are performed:

[0067] Collect no less than 10,000 WeChat chat screenshots covering a variety of scenes and situations;

[0068] Collect no less than 10,000 other types of pictures, including documents and images;

[0069] Collect spoken texts and written texts, the spoken texts are from social media, forum posts, and transcripts of audio recordings, and the written texts are from news and literature;

[0070] The WeChat chat screenshots, other types of pictures, spoken texts and written texts are cleaned, organized and labeled to ensure that the data format meets the requirements of model training.

[0071] In this embodiment, training the WeChat chat screenshot classification model includes:

[0072] The collected WeChat chat screenshots and other types of pictures are divided into training sets and validation sets, using 80% of the data as the training set and 20% of the data as the validation set;

[0073] Select a target deep learning network model, which includes ResNet and YOLO, and set a target learning rate and optimizer parameters;

[0074] Input data and perform model training, continuously adjust parameters to improve the performance and accuracy of the model, and obtain the WeChat chat screenshot classification model.

[0075] In this embodiment, the training text classification model includes:

[0076] Use the collected spoken and written text data to train the model using the FastText text classification tool;

[0077] Ensure that the format of the training data meets the requirements of FastText, where the format of the training data includes text content and labels.

[0078] In this embodiment, the establishment of the WeChat chat screenshot accurate matching rule base includes:

[0079] Analyze various scenes and features in WeChat chat screenshots, and extract key information and features of the WeChat chat screenshots;

[0080] A dictionary library is established, which includes iconic texts of WeChat chats, including: "withdrawing a message" and "not being friends with other people in the group".

[0081] In one embodiment, rule verification, text recognition and classification algorithms are added to the traditional image classification technology. The WeChat chat screenshots contain text, which are all spoken language or WeChat-specific words. By establishing a "spoken and written text classification model", the accuracy of WeChat chat screenshot image judgment is improved.

[0082] In this embodiment, training the WeChat chat screenshot classification model includes:

[0083] Collect >= 10,000 WeChat chat screenshots. If that is not enough, use data augmentation to expand the image data.

[0084] Collect other pictures >= 10,000, including documents, images, etc.

[0085] The data is divided into a training set and a validation set, 80% for training and 20% for validation.

[0086] Select a classification network such as ResNet, Yolo, etc. Set the learning rate, optimizer and other parameters before training, and input the data in step 3 into the network for training.

[0087] In this embodiment, training a text classification model for written and spoken Chinese includes: collecting spoken texts, collecting a large amount of spoken Chinese texts from the Internet, open source data sets, etc., such as social media, forum posts, and recordings (which need to be converted into text).

[0088] Collect written texts, and collect large amounts of Chinese written texts, such as news and documents, from the Internet, open source data sets, etc.

[0089] Clean, organize, and annotate the data to meet the format required by the FastText text classification network, and use the command line tools or APIs provided by FastText to train the model. Through the command line tools, you can specify the path of the training data, the parameters of the model (such as vector dimension, learning rate, number of iterations, etc.), and the output path.

[0090] In this embodiment, establishing a WeChat chat screenshot accurate matching rule library includes: In order to improve the efficiency of WeChat chat screenshot judgment, some text only exists in WeChat chat screenshots.

[0091] Analyze various scenarios in WeChat chat screenshots;

[0092] Create a dictionary library that contains WeChat chat marker texts such as: "Withdraw a message", "Not friends with other people in the group", and "The above is a greeting."

[0093] In this embodiment, the picture to be detected is input into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the picture to be detected is a chat screenshot; if so, the text features in the picture to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, the picture to be detected is determined to be a WeChat chat screenshot, and the output judgment result is a WeChat chat screenshot; if the match fails, the picture to be detected is input into the written language and spoken language classification model, and the written language and spoken language classification model uses text classification technology to analyze and judge the text in the picture to be detected; if the judgment result is that the text of the picture to be detected belongs to spoken language, the picture is confirmed to be a WeChat chat screenshot, and the output judgment result is a WeChat chat screenshot; if the written language and spoken language classification model determines that it is written language, the picture to be detected is not a WeChat chat screenshot. By using image classification, target detection and text classification technology, through data collection, model training and rule establishment and other steps, the problems existing in the prior art can be effectively solved, and the accuracy and judgment ability of WeChat chat screenshots can be improved.

[0094] In the traditional image classification, detection and judgment of WeChat chat screenshots, only the characteristics of the image itself are focused on, and the text information on the image is ignored. The key point of this application is to add rule verification, text recognition and classification algorithms to the traditional image classification technology. WeChat chat screenshots contain text, which are all spoken or WeChat-specific words. By establishing a "spoken and written text classification model", the accuracy of WeChat chat screenshot image judgment is improved.

[0095] In one embodiment, a system for determining WeChat chat screenshots is provided, including the following program modules:

[0096] The first judgment module is used to input the image to be detected into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the image to be detected is a chat screenshot;

[0097] Matching module: if the judgment is yes, then the text features in the image to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, it is determined that the image to be detected is a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0098] The second judgment module is used to judge that if the match fails, the image to be detected is input into the written and spoken language classification model, and the written and spoken language classification model uses text classification technology to analyze and judge the text in the image to be detected;

[0099] The first output module is used to determine that if the judgment result is that the text of the image to be detected is spoken language, then the image is confirmed to be a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot;

[0100] The second output module is used to determine that if the written language and spoken language classification model determines that it is written language, then the image to be detected is not a screenshot of a WeChat chat.

[0101] In this embodiment, the WeChat chat screenshot classification model is established before the following steps are performed:

[0102] Collect no less than 10,000 WeChat chat screenshots covering a variety of scenes and situations;

[0103] Collect no less than 10,000 other types of pictures, including documents and images;

[0104] Collect spoken texts and written texts, the spoken texts are from social media, forum posts, and transcripts of audio recordings, and the written texts are from news and literature;

[0105] The WeChat chat screenshots, other types of pictures, spoken texts and written texts are cleaned, organized and labeled to ensure that the data format meets the requirements of model training.

[0106] In this embodiment, training the WeChat chat screenshot classification model includes:

[0107] The collected WeChat chat screenshots and other types of pictures are divided into training sets and validation sets, using 80% of the data as the training set and 20% of the data as the validation set;

[0108] Select a target deep learning network model, which includes ResNet and YOLO, and set a target learning rate and optimizer parameters;

[0109] Input data and perform model training, continuously adjust parameters to improve the performance and accuracy of the model, and obtain the WeChat chat screenshot classification model.

[0110] In this embodiment, the training text classification model includes:

[0111] Use the collected spoken and written text data to train the model using the FastText text classification tool;

[0112] Ensure that the format of the training data meets the requirements of FastText, where the format of the training data includes text content and labels.

[0113] In this embodiment, the establishment of the WeChat chat screenshot accurate matching rule base includes:

[0114] Analyze various scenes and features in WeChat chat screenshots, and extract key information and features of the WeChat chat screenshots;

[0115] A dictionary library is established, which includes iconic texts of WeChat chats, including: "withdrawing a message" and "not being friends with other people in the group".

[0116] The specific implementation content of each module can be found in the above-mentioned limitations on the method for determining WeChat chat screenshots, which will not be repeated here.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for determining a WeChat chat screenshot, characterized in that: The method comprises: Input the image to be detected into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the image to be detected is a chat screenshot; If yes, the text features in the image to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, it is determined that the image to be detected is a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot; If the match fails, the image to be detected is input into a written and spoken language classification model, and the written and spoken language classification model uses text classification technology to analyze and judge the text in the image to be detected; If the judgment result is that the text of the image to be detected is spoken language, the image is confirmed to be a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot; If the written language and spoken language classification model determines that it is written language, then the image to be detected is not a screenshot of a WeChat chat; The WeChat chat screenshot classification model is established before: Collect no less than 10,000 WeChat chat screenshots covering a variety of scenes and situations; Collect no less than 10,000 other types of pictures, including documents and images; Collect spoken texts and written texts, the spoken texts are from social media, forum posts, and transcripts of audio recordings, and the written texts are from news and literature; Clean, organize and annotate the WeChat chat screenshots, other types of pictures, spoken texts and written texts to ensure that the data format meets the requirements of model training; The establishment of the WeChat chat screenshot accurate matching rule base includes: Analyze various scenes and features in WeChat chat screenshots, and extract key information and features of the WeChat chat screenshots; A dictionary library is established, which includes iconic texts of WeChat chats, wherein the iconic texts of WeChat chats include: "withdraw a message" and "are not friends with other people in the group".

2. The method according to claim 1, characterized in that Training the WeChat chat screenshot classification model includes: The collected WeChat chat screenshots and other types of pictures are divided into training sets and validation sets, using 80% of the data as the training set and 20% of the data as the validation set; Select a target deep learning network model, which includes ResNet and YOLO, and set a target learning rate and optimizer parameters; Input data and perform model training, continuously adjust parameters to improve the performance and accuracy of the model, and obtain the WeChat chat screenshot classification model.

3. The method according to claim 1, characterized in that The written and spoken language classification model training includes: Use the collected spoken and written text data to train the model using the FastText text classification tool; Ensure that the format of the training data meets the requirements of FastText, where the format of the training data includes text content and labels.

4. A system for judging WeChat chat screenshots, characterized in that: The system comprises: The first judgment module is used to input the image to be detected into the WeChat chat screenshot classification model, and the WeChat chat screenshot classification model determines whether the image to be detected is a chat screenshot; Matching module: if the judgment is yes, then the text features in the image to be detected are matched and compared with the text features in the precise matching rule library. If the match is successful, it is determined that the image to be detected is a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot; The second judgment module is used to judge that if the match fails, the image to be detected is input into the written and spoken language classification model, and the written and spoken language classification model uses text classification technology to analyze and judge the text in the image to be detected; The first output module is used to determine that if the judgment result is that the text of the image to be detected is spoken language, then the image is confirmed to be a WeChat chat screenshot, and the judgment result is output as a WeChat chat screenshot; The second output module is used to determine that if the written language and spoken language classification model determines that it is written language, then the image to be detected is not a WeChat chat screenshot; The WeChat chat screenshot classification model is established before: Collect no less than 10,000 WeChat chat screenshots covering a variety of scenes and situations; Collect no less than 10,000 other types of pictures, including documents and images; Collect spoken texts and written texts, the spoken texts are from social media, forum posts, and transcripts of audio recordings, and the written texts are from news and literature; Clean, organize and annotate the WeChat chat screenshots, other types of pictures, spoken texts and written texts to ensure that the data format meets the requirements of model training; The establishment of the WeChat chat screenshot accurate matching rule base includes: Analyze various scenes and features in WeChat chat screenshots, and extract key information and features of the WeChat chat screenshots; A dictionary library is established, which includes iconic texts of WeChat chats, wherein the iconic texts of WeChat chats include: "withdraw a message" and "are not friends with other people in the group".

5. The system according to claim 4, characterized in that Training the WeChat chat screenshot classification model includes: The collected WeChat chat screenshots and other types of pictures are divided into training sets and validation sets, using 80% of the data as the training set and 20% of the data as the validation set; Select a target deep learning network model, which includes ResNet and YOLO, and set a target learning rate and optimizer parameters; Input data and perform model training, continuously adjust parameters to improve the performance and accuracy of the model, and obtain the WeChat chat screenshot classification model.

6. The system according to claim 5, characterized in that The written and spoken language classification model training includes: Use the collected spoken and written text data to train the model using the FastText text classification tool; Ensure that the format of the training data meets the requirements of FastText, where the format of the training data includes text content and labels.

Images