Method and system for determining target chat information

Abstract

The invention relates to the technical field of communication supervision, and discloses a method and system for determining target chat information, and the method comprises the steps: carrying out the screenshot of a chat interface on a target device in response to a screenshot trigger event, and obtaining a chat interface image, the chat interface image comprising a plurality of chat information arranged according to a preset format; cutting the chat interface image according to a preset format to obtain a plurality of sub-images; determining image coding information of each sub-image in the plurality of sub-images; obtaining a coding information base, wherein the coding information base at least comprises multiple pieces of image coding information determined by performing screenshot on the chat interface last time; determining the sub-image of which the image coding information does not exist in the coding information base as a target sub-image; and determining target chat information according to the target sub-graph. In this way, the situation that a supervision program wastes a large amount of memory of the target equipment is avoided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication supervision technology, and specifically to a method and system for determining target chat information. Background Art

[0002] With the popularity of social applications, users have an increasing need to monitor and manage chat content during communication, for example, to collect forensic evidence from chat messages, or to monitor children's chat content to prevent them from being exposed to harmful information on the Internet.

[0003] After the supervision program is started on the supervised target device, for example, a child opens the SMS interface or WeChat interface on a smart watch, existing communication supervision applications usually take a full-screen screenshot of the current chat interface at a preset screenshot frequency, and then perform OCR (Optical Character Recognition) on the content of the full-screen screenshot to obtain a text result, and then display the text result to the main device, for example, sending the chat content recognized by OCR to the parent's mobile phone.

[0004] In the prior art, it is necessary to take a full-screen screenshot of the current chat interface at a preset screenshot frequency, which will result in a large number of images with almost exactly the same content among the multiple images obtained by the screenshot, thereby wasting a lot of memory of the monitored target device. If the screen is not scrolled, a large number of exactly the same images will be obtained. If the screen is scrolled, it is very likely that there will be only one chat message that is different between the first screenshot and the second screenshot.

[0005] Therefore, how to solve the problem of regulatory programs wasting a large amount of memory on target devices has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In view of the above problems, an embodiment of the present application provides a method and system for determining target chat information, which is used to solve the problem in the prior art that the supervision program wastes a large amount of memory of the target device.

[0007] According to one aspect of an embodiment of the present application, a method for determining target chat information is provided, which is applied to a target device, and the method for determining the target chat information includes: in response to a screenshot trigger event, taking a screenshot of a chat interface on the target device to obtain a chat interface image, wherein the chat interface image includes multiple chat information arranged in a preset format; cropping the chat interface image according to the preset format to obtain multiple sub-images; determining image coding information of each of the multiple sub-images; obtaining a coding information library, the coding information library at least including multiple image coding information determined by the last screenshot of the chat interface; determining a sub-image whose image coding information does not exist in the coding information library as a target sub-image; and determining the target chat information based on the target sub-image.

[0008] Preferably, the method for determining the target chat information also includes: determining the arrangement order between multiple target sub-images and generating sequence information corresponding to the target sub-images; sending the target chat information corresponding to the target sub-image and the sequence information corresponding to the target sub-image to the server, so that the server splices the target chat information according to the sequence information.

[0009] Preferably, determining the target chat information according to the target sub-graph also includes: determining the target sub-graph as the target chat information; sending the target chat information corresponding to the target sub-graph and the sequence information corresponding to the target sub-graph to the server, and also includes: sending the target chat information and the sequence information corresponding to the target chat information to the server, so that the server splices the target chat information according to the sequence information to obtain a display image.

[0010] Preferably, after determining the sub-graph whose image coding information does not exist in the coding information library as the target sub-graph, the method for determining the target chat information further includes: saving the image coding information of the target sub-graph to the coding information library.

[0011] Preferably, the chat interface image includes interface identification information corresponding to the chat interface, and multiple sub-images all correspond to the interface identification information of the chat interface image. The multiple image coding information in the coding information library is image coding information grouped according to different interface identification information; before determining the sub-image whose image coding information does not exist in the coding information library as the target sub-image, the method for determining the target chat information also includes: determining the interface identification information of the chat interface image; determining the sub-image whose image coding information does not exist in the coding information library as the target sub-image, including: judging whether the coding information library includes image coding information grouped according to the interface identification information; if so, determining the target coding information group according to the image coding information grouped according to the interface identification information included in the coding information library, and determining the sub-image whose image coding information does not exist in the target coding information group as the target sub-image.

[0012] Preferably, before taking a screenshot of the chat interface on the target device to obtain a chat interface image in response to a screenshot trigger event, the method for determining the target chat information also includes: monitoring the input variables of the input box of the chat interface; if the input variable changes from a non-null value to a null value, a screenshot trigger event is generated.

[0013] Preferably, the chat interface image also includes multiple user information corresponding to each chat message; the chat interface image is cropped according to a preset format to obtain multiple sub-images, including: scanning the chat interface image, determining the boundary information of multiple user information displayed on the chat interface image and the boundary information of multiple chat information; determining multiple first cropping boundaries on the chat interface image according to the boundary information of the multiple user information; determining multiple second cropping boundaries on the chat interface image according to the boundary information of the multiple chat information; cropping the chat interface image according to the multiple first cropping boundaries and the multiple second cropping boundaries to obtain multiple target sub-images.

[0014] According to another aspect of an embodiment of the present application, a method for determining target chat information is provided, which is applied to a server, and the method for determining target chat information includes: receiving target chat information sent by a target device according to any one of claims 1 to 7; generating display information based on the target chat information; and sending the display information to a main device for display.

[0015] Preferably, the target chat information is a target sub-graph. Before generating display information according to the target chat information, the method for determining the target chat information also includes: receiving sequence information corresponding to the target chat information sent by the target device; generating display information according to the target chat information, including: splicing the target chat information according to the sequence information to obtain a display image; sending the display information to the main device for display, including: sending the display image to the main device for display.

[0016] According to another aspect of an embodiment of the present application, a system for determining target chat information is provided, including: a target device and a server, wherein a communication connection is established between the target device and the server; the target device is used to, in response to a screenshot trigger event, take a screenshot of a chat interface on the target device to obtain a chat interface image, wherein the chat interface image includes multiple chat information arranged in a preset format; crop the chat interface image according to the preset format to obtain multiple sub-images; determine image coding information of each of the multiple sub-images; obtain a coding information library, wherein the coding information library at least includes multiple image coding information determined by the last screenshot of the chat interface; determine a sub-image whose image coding information does not exist in the coding information library as a target sub-image; determine the target chat information based on the target sub-image; send the target chat information to the server; the server is used to receive the target chat information sent by the target device; generate display information based on the target chat information; and send the display information to the main device for display.

[0017] In an embodiment of the present application, a screenshot of the chat interface on the target device is taken in response to a screenshot trigger event to obtain a chat interface image. The screenshot timing can be set as an event trigger so that the chat interface image obtained by each screenshot contains at least one new chat message that does not exist in the chat interface image obtained by the previous screenshot, thereby avoiding the presence of a large number of images with exactly the same content in the multiple images obtained by the screenshot; multiple sub-images are obtained by cropping the chat interface image according to a preset format, and the sub-image whose image coding information does not exist in the coding information library is determined as the target sub-image, and the target chat information is determined based on the target sub-image. The sub-image with repeated chat content can be removed more accurately based on the image content presented when each chat message is presented on the image, thereby ensuring the integrity of the chat content while fully utilizing the target device memory.

[0018] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings: Figure 1 A schematic diagram showing an application scenario of the method for determining target chat information provided by an embodiment of the present application; Figure 2 A schematic diagram showing a flow chart of a method for determining target chat information provided by an embodiment of the present application; Figure 3 A schematic diagram showing a flow chart of a method for determining target chat information provided by another embodiment of the present application; Figure 4 A schematic diagram showing the structure of a device for determining target chat information provided by an embodiment of the present application is shown; Figure 5 A schematic diagram showing the structure of a device for determining target chat information provided by another embodiment of the present application is shown; Figure 6 A schematic diagram showing the structure of a device for determining target chat information provided by an embodiment of the present application is shown; Figure 7 A schematic diagram of the structure of a system for determining target chat information provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0021] In solving the technical problem that the regulatory program wastes a large amount of memory of the target device, the inventor of the present application first discovered that the text results obtained by OCR processing can be deduplicated. That is, after performing OCR processing on each screenshot image to obtain the text result, the repeated text results in multiple text results can be found and the repeated parts can be removed.

[0022] However, the inventors of the present application have found in practice that the accuracy of the duplicate checking process for finding duplicate text results in multiple text results is not ideal. Specifically, OCR text recognition does not support multiple languages ​​well, and does not support non-text messages such as emoticons. This causes OCR to output the same recognition results when processing similar emoticons or foreign languages ​​with connected writing. As a result, when deduplicating, the supervision program will determine different chat messages as duplicate chat messages and delete them, resulting in incoherent content in the subsequently generated chat messages.

[0023] For example, when a child is in a chat group, a smiling emoticon with upturned corners of the mouth appears in the chat group, and then a frustrated emoticon with downturned corners of the mouth appears. OCR can easily recognize these two chat messages as the same character, thereby removing the frustrated emoticon with downturned corners of the mouth and retaining only the smiling emoticon with upturned corners of the mouth.

[0024] In order to improve the accuracy of deduplication, the inventor of the present application further discovered that deduplication can be performed not based on the OCR recognition results, but based on the image content presented when each chat message is presented on the image. That is, the full-screen screenshot image can be divided according to the arrangement of the chat messages first, and sub-images corresponding to each chat message can be segmented from the full-screen screenshot image. Then, based on the image encoding information of the sub-image, it is determined whether the sub-image is repeated with the saved sub-image, and the sub-image that does not repeat with the saved sub-image is saved, so that the target device does not repeatedly store duplicate chat information, thereby making full use of the device memory. In addition, deduplication based on image content can make different chat messages less likely to be misjudged as duplicate content, thereby improving the accuracy of deduplication to ensure the integrity of the chat content.

[0025] This application is applicable to supervising the communication process of the target device, wherein the target device can be a portable mobile device such as a mobile phone, smart watch, tablet, notebook, computer, etc., or a device with display and communication functions; the communication process can be implemented based on a mobile operating system (such as Android and iOS), or based on a desktop operating system, or based on a web page.

[0026] Figure 1 A schematic diagram showing an application scenario of the method for determining target chat information provided by an embodiment of the present application is shown. Figure 1 The target user is a user who needs to be supervised, such as a child or an elderly person. The target user uses a communication program on the target device for instant messaging. A supervision program is also running on the target device. The supervision program captures the chat information of the instant messaging and sends the captured chat information to the server. The server processes the chat information to obtain a display result and sends the display result to the main device. The main user views the display result on the main device. The main user is a user with viewing authority, such as a parent, so as to realize the communication supervision of the target user by the main user.

[0027] Figure 2 The flowchart of the method for determining the target chat information provided by the embodiment of the present application is shown, and the method is applied to the above-mentioned supervision procedure. Figure 2 As shown, the method comprises the following steps: S110, in response to a screenshot triggering event, taking a screenshot of the chat interface on the target device to obtain a chat interface image, wherein the chat interface image includes a plurality of chat messages arranged in a preset format.

[0028] When applied on a mobile phone, the screenshot is preferably a full-screen screenshot to improve the operating efficiency of the target device. When applied on a web page or desktop, the screenshot is preferably a screenshot of the target area on the screen, wherein the target area is the display area on the web page or desktop that includes the chat interface.

[0029] Text chat messages are generally filled in chat bubbles, and the chat bubbles are arranged up and down according to the sending time of the chat messages. Therefore, different chat messages can be distinguished more clearly. In an optional manner, the screenshot trigger event can be set to the user's chat message sending event. For example, when the user clicks the send button next to the input box on the mobile phone screen, the screenshot is triggered, or when the user presses the enter key on the keyboard, the screenshot is triggered. However, different target devices have different authorization standards for the permissions involved in monitoring the user's operation events. Therefore, it is possible to consider monitoring the events of the input box. Specifically, before S110, the method for determining the target chat information also includes S108~S109, specifically: S108, monitoring the input variables of the input box of the chat interface.

[0030] S109: If the input variable changes from a non-null value to a null value, a screenshot trigger event is generated.

[0031] The user's operation events are simulated by changing the value of the input variable. When the input variable changes from a non-null value to a null value, it can be inferred that the user has completed sending a new chat message.

[0032] The embodiment of the present application sets the screenshot timing on the target device as a screenshot trigger event through S110, so that the chat interface image obtained by each screenshot can contain at least one new chat message that does not exist in the chat interface image obtained by the previous screenshot, thereby avoiding the existence of a large number of images with exactly the same content in the multiple images obtained by the screenshots.

[0033] S120, cropping the chat interface image according to a preset format to obtain multiple sub-images.

[0034] After executing S110 to obtain the chat interface image, image processing technology can be used to perform denoising, binarization, and other processing on the chat interface image to obtain an image with higher quality and clearer edge features.

[0035] Next, the edge detection algorithm can be used to identify the message box outline of each chat message in the chat interface image, that is, to identify the boundary of the chat bubble.

[0036] The cropped sub-image may include only the chat bubble carrying the chat information, and may also include user information, for example, the nickname of the chat user displayed above the chat bubble and the avatar of the chat user displayed on the side of the chat bubble.

[0037] When the chat interface image only contains chat bubbles carrying chat information, an edge detection algorithm can be used to identify the message box outline of each chat message in the chat interface image, that is, to identify the boundary of the chat bubble.

[0038] When the chat interface image also includes multiple user information corresponding to each chat message, the edge detection algorithm can be used to first identify the starting height position of the chat user's avatar or nickname, and then identify the ending height position of the chat bubble, so as to define the distribution area of ​​the sub-image on the chat interface image by the starting height position of the chat user's avatar or nickname and the ending height position of the chat bubble. Specifically, S120 further includes sub-steps S121 to S124: S121, scanning the chat interface image, and determining boundary information of multiple user information and boundary information of multiple chat information displayed on the chat interface image.

[0039] The chat interface image is scanned using an edge detection algorithm, and the starting height position of the chat user's avatar or nickname on the chat interface image is identified, and then the ending height position of the chat bubble on the chat interface image is identified.

[0040] S122, determining multiple first cropping boundaries on the chat interface image based on the boundary information of multiple user information.

[0041] The starting height position of the chat user's avatar or nickname on the chat interface image is determined as the starting height position of the sub-image.

[0042] S123, determining multiple second cropping boundaries on the chat interface image according to the boundary information of multiple chat messages.

[0043] The termination height position of the chat bubble on the chat interface image is determined as the termination height position of the sub-image.

[0044] S124, cropping the chat interface image according to multiple first cropping boundaries and multiple second cropping boundaries to obtain multiple target sub-images.

[0045] The sub-image is cropped according to the starting height position and the ending height position of the sub-image in the chat interface image.

[0046] Specifically, for a chat interface image containing user information, the upper edge of the sub-image can be determined by the features of the avatar (such as color, shape, specific logos, etc.), the lower edge of the sub-image can be determined by the boundary between the bubble and the background, and the left and right edges can use appropriate fixed margins to determine the screenshot area range of a sub-image.

[0047] Under the premise of a fixed algorithm, the edge detection of the same sub-image is also fixed, which can ensure that when the same chat information is cropped on different chat interface images, the image encoding information of the cropped sub-image containing the same chat information is also the same, thereby ensuring the accuracy of deduplication of repeated sub-images and ensuring the integrity of subsequent chat content.

[0048] Furthermore, for the time information or system notification information on the chat interface image, the upper and lower edge coordinates of the information area can be determined by analyzing the pixel distribution on the chat interface image. For example, the font color of WeChat's system notifications is generally light gray. At this time, the line-by-line scanning technology can be used to identify and distinguish whether it has entered the upper and lower edges of such information. When an area with large continuous pixel changes (representing text lines) is detected, and the spacing between text lines is relatively stable, it can be considered to be an independent content area, that is, the display area of ​​the time information or system notification information on the chat interface image.

[0049] S130, determining image coding information of each sub-image among the multiple sub-images.

[0050] The image coding information may be original coding information, predictive coding information, or transform domain coding information. Furthermore, the image coding information may be binary coding information, hexadecimal coding information, etc. Furthermore, the image coding information may be encrypted coding information, for example, the image coding information is obtained by encrypting the original coding information using the MD5 Message-Digest Algorithm (MD5 Message-Digest Algorithm).

[0051] In essence, image coding information is the pixel distribution information on the image. Therefore, the coding information of images with different pixel distributions is also different. Therefore, the image coding information of the sub-image can be used as the unique identifier of the sub-image to distinguish multiple sub-images. Moreover, when the chat interface image also includes multiple user information corresponding to each chat message, the cropped sub-image also includes user information. The pixel distribution of the sub-image is more complex, which further avoids the situation where sub-images containing different chat information are confirmed as duplicate sub-images, so as to further ensure the accuracy of deduplication of duplicate sub-images.

[0052] S140, obtaining a coding information library, where the coding information library at least includes multiple image coding information determined when taking a screenshot of the chat interface last time.

[0053] When the target user continuously triggers the screenshot trigger event, the current chat interface on the target device is repeatedly screenshoted to obtain multiple chat interface images. At this time, the same chat information is likely to appear between the multiple chat interface images. For example, the first chat interface image has the chat information "What are you eating for lunch today?", "Stir-fried beef" and "Do you want to drive? It's raining outside" arranged in sequence from top to bottom. After the new chat information is sent and the screen scrolls, the second chat interface image has the chat information "Stir-fried beef", "Do you want to drive? It's raining outside" and "Tairan 6th Road is a bit congested" arranged in sequence from top to bottom. After the new chat information is sent and the screen scrolls, the third chat interface image has the chat information "Tairan 6th Road is a bit congested", "Did you bring an umbrella" and "Forget it, I choose takeout" arranged in sequence from top to bottom.

[0054] When taking a screenshot of the first chat interface image, a first sub-image containing the information “What are you eating for lunch today?”, a second sub-image containing the information “Stir-fried yellow beef”, and a third sub-image containing the information “Do you want to drive? It’s raining outside?” are obtained; when taking a screenshot of the second chat interface image, a first sub-image containing the information “Stir-fried yellow beef”, a second sub-image containing the information “Do you want to drive? It’s raining outside”, and a third sub-image containing the information “Tairan 6th Road is a bit congested” are obtained; when taking a screenshot of the third chat interface image, a first sub-image containing the information “Tairan 6th Road is a bit congested”, a second sub-image containing the information “Did you bring an umbrella?” and a third sub-image containing the information “Forget it, I’ll order takeout” are obtained.

[0055] It can be found that the chat information "Stir-fried Yellow Beef" and "Do you want to drive? It's raining outside" are repeated in the first chat interface image and the second chat interface image, and "Tairan 6th Road is a bit congested" is repeated in the second chat interface image and the third chat interface image. Therefore, the third sub-image of the information "Tairan 6th Road is a bit congested" obtained by cropping the second chat interface image is repeated with the first sub-image of the information "Tairan 6th Road is a bit congested" obtained by cropping the third chat interface image.

[0056] The coding information library may be obtained from a server or stored locally on the target device.

[0057] S150: Determine a sub-picture whose image coding information does not exist in the coding information library as a target sub-picture.

[0058] After determining the target sub-image, the coding information library can also be updated to facilitate determining whether the sub-image is a repeated sub-image after the next screenshot is taken to obtain the chat interface image and divided into multiple sub-images. That is, after S150, the method for determining the target chat information also includes: S190, saving the image coding information of the target sub-image into a coding information library.

[0059] Optionally, the coding information library stores image coding information of all sub-images cropped on the target device.

[0060] In order to improve the operation efficiency of the supervision program, preferably, the chat interface image includes interface identification information corresponding to the chat interface, the multiple sub-images correspond to the interface identification information of the chat interface image, and the multiple image encoding information in the encoding information library is image encoding information grouped according to different interface identification information; before S150, the method for determining the target chat information also includes: S141, determining interface identification information of the chat interface image.

[0061] The interface identification information may be chat session name information, such as a group name in a group chat, or may be encrypted information of the session name.

[0062] S150, including sub-steps S151 to S153: S151, judging whether the coding information library includes image coding information grouped according to the interface identification information; if yes, jumping to S152; if no, jumping to S153.

[0063] S152: If yes, determine the target coding information group according to the image coding information grouped according to the interface identification information included in the coding information library, and determine the sub-image whose image coding information does not exist in the target coding information group as the target sub-image.

[0064] S153: If it does not exist, a new target coding information group is created in the coding information library according to the interface identification information, and the image information of the target sub-image is reserved in the newly created target coding information group.

[0065] Through S141~S153, when determining whether a sub-image is a repeated sub-image, the search range of the image coding information of the current sub-image in the coding information library can be limited to a group of image coding information, thereby greatly reducing the search range and improving the operating efficiency of the supervision program.

[0066] S160, determining target chat information according to the target subgraph.

[0067] Optionally, the target chat information is an OCR recognition result of the target sub-image; preferably, the target chat information is a target sub-image, so that the server can accept the target sub-image and display the target sub-image, thereby improving the browsing experience of the main user.

[0068] After receiving the target chat information, the server needs to splice the target chat information to generate display information. For the target chat information generated by different chat interface images through S110~S160, the target chat information generated by the chat interface image obtained by the later screenshot can be spliced ​​after the chat interface image obtained by the earlier screenshot; for the target chat information generated by different target sub-images based on the same chat interface image through S110~S160, it is necessary to determine the arrangement order between the multiple target sub-images corresponding to the same chat interface image, and the server splices the multiple target sub-images according to the arrangement order. Specifically, the method for determining the target chat information also includes S170~S180: S170, determining an arrangement order among the plurality of target sub-graphs and generating sequence information corresponding to the target sub-graphs.

[0069] S180, sending the target chat information corresponding to the target subgraph and the sequence information corresponding to the target subgraph to the server, so that the server splices the target chat information according to the sequence information.

[0070] Furthermore, when the target chat information is a target subgraph, S160 further includes: S161, determining the target subgraph as the target chat information; S180 also includes: S181, sending the target chat information and the sequence information corresponding to the target chat information to the server, so that the server splices the target chat information according to the sequence information to obtain a display image.

[0071] Furthermore, taking the target device as a mobile phone developed based on the Android system as an example, the operation process of the supervision program is explained: The supervision process performs the following steps in sequence: S101, applying for the screenshot permission of the application.

[0072] Preferably, the screenshot permission can be obtained by applying for the permission of the auxiliary function service.

[0073] The basic steps to apply for Android accessibility permissions are usually: <manifest>In the tag <application>Under the tag, add <service>Tag to define the accessibility service and declare the android.permission.bind_accessibility_service permission. At the same time, you need to create a configuration file (such as accessibility_service_config.xml) in the res / xml folder to configure the accessibility service in detail, such as setting the service description, label and other information.

[0074] When the application starts, use the ContextCompat.checkSelfPermission() method to check whether the application has obtained accessibility permissions. If the permission is not granted, call the ActivityCompat.requestPermissions() method to request permission from the user. When the user responds to the permission request, the system calls the onRequestPermissionsResult() method to determine the subsequent operation based on the user's authorization result. If the user grants permission, the related accessibility operations can be performed. If the user denies permission, prompt the user or guide the user to apply for permission again based on the specific situation.

[0075] By applying for accessibility service permissions to obtain screenshot permissions, even if the target device is restarted or the application is actively or abnormally exited, as long as the application is started again, the screenshot permission applied for last time will still be valid without having to apply again. This solves the problem of having to apply for screenshot permissions again to take screenshots in the background after the device is restarted or the application is exited or crashed unexpectedly.

[0076] S102, check whether the target device application has screenshot permission. If yes, jump to S103; if no, jump to S104.

[0077] S103, monitoring application switching events, and determining whether to start the screenshot service program to execute S110-S160.

[0078] After obtaining authorization for the accessibility service, create an accessibility service class and implement related methods to monitor application switching events.

[0079] The basic steps are: create a class that inherits AccessibilityService as the core implementation class of the accessibility service. Override the onAccessibilityEvent method. This method will be called whenever an accessibility event that meets the configuration occurs, including application switching. Usually when the event type is judged to be TYPE_WINDOW_STATE_CHANGED, the system API (such as the event.getPackageName method) is called to obtain the package name and other information of the current foreground application. Compare the package name with the package name of the preset monitoring target application list to determine whether the current foreground application is the application that needs to be monitored. If so, start the screenshot service program. If not, continue to listen for application switching events.

[0080] After starting the screenshot service program, the supervisory program responds to the screenshot trigger event, takes a full-screen screenshot of the chat interface on the target device to obtain a chat interface image, crops the chat interface image according to a preset format to obtain multiple sub-images, generates a hash value based on the binary data of the sub-image, and compares the hash value with multiple hash values ​​stored on the server. If the hash value already exists on the server, it means that the sub-image is a duplicate sub-image. If not, it means that the sub-image is a new sub-image, and the sub-image and the hash value of the sub-image can be stored on the server. The format of the sub-image may be a picture format such as bmp, jpg, png, etc. The binary data of the sub-image is the storage data of the sub-image stored on the target device in the above picture format.

[0081] When the hash value generated according to the binary data of the image is used as the image encoding information, the md5 algorithm may be used to generate the hash value, or other hash algorithms may be used to generate the hash value.

[0082] When comparing the hash value with the stored multiple hash values, the server may use a hash comparison method to compare the hash value of the image data, or may use a pixel-by-pixel comparison method to compare the pixel values ​​of the image data.

[0083] The basic principle of the hash value comparison method is that the hash function can convert image data into a hash value of fixed length. If the content of two images is exactly the same, then their hash values ​​calculated by the hash function should be the same. Commonly used hash algorithms include MD5, SHA-1, etc. MD5 and SHA-1 algorithms are usually selected. Perceptual hash algorithms (such as pHash, aHash, dHash) can be used when security requirements are strict.

[0084] The basic principle of the pixel-by-pixel comparison method is to traverse each pixel of the two images and compare whether their pixel values ​​are exactly the same. For color images, it is necessary to compare the values ​​of the RGB or RGBA channels of each pixel; for grayscale images, it is only necessary to compare the grayscale values ​​of each pixel. This method is relatively direct, especially for high-resolution images, and can make the image encoding information have a good identification effect.

[0085] S104, prompting the target user to authorize the screenshot permission.

[0086] In an embodiment of the present application, in response to a screenshot trigger event, a screenshot of the chat interface on the target device is taken to obtain a chat interface image. The screenshot timing can be set as an event trigger so that each chat interface image obtained by taking a screenshot contains at least one new chat message that does not exist in the chat interface image obtained by the previous screenshot, thereby avoiding the presence of a large number of images with exactly the same content in the multiple images obtained by taking screenshots. The chat interface image is cropped according to a preset format to obtain multiple sub-images, and the sub-image whose image coding information does not exist in the coding information library is determined as a target sub-image, and the target chat information is determined based on the target sub-image. The sub-image with repeated chat content can be more accurately removed based on the image content presented when each chat message is presented on the image, thereby ensuring the integrity of the chat content while fully utilizing the target device memory.

[0087] According to another aspect of the embodiment of the present application, a method for determining target chat information is also provided. Figure 3 , Figure 3 A flow chart of a method for determining target chat information provided by another embodiment of the present application is shown. The method for determining target chat information is applied to a server. The method for determining target chat information includes: S210, receiving target chat information sent by a target device; S230, generating display information according to the target chat information; S250: Send the display information to the main device for display.

[0088] Preferably, the target chat information is a target subgraph. Before S230, the method for determining the target chat information further includes: S220, receiving sequence information corresponding to the target chat information sent by the target device; S230, generating display information according to the target chat information, including: S231, splicing the target chat information according to the sequence information to obtain a display image; S250, including: S251, sending the display image to the main device for display.

[0089] OCR text recognition does not support multiple languages ​​well and does not support non-text messages such as emoticons. By sending the display image to the main device for display, the problem of poor display effect caused by sending the OCR recognition result to the main device for display can be avoided.

[0090] Figure 4 The schematic diagram of the structure of the device for determining target chat information provided by the embodiment of the present application is shown. Figure 4 As shown, the device 310 includes: a screenshot module 311 , a cropping module 312 , a first determination module 313 , an acquisition module 314 , a second determination module 315 and a third determination module 316 .

[0091] The screenshot module 311 is used to take a screenshot of the chat interface on the target device in response to a screenshot trigger event to obtain a chat interface image, where the chat interface image includes a plurality of chat messages arranged in a preset format; The cropping module 312 is used to crop the chat interface image according to a preset format to obtain multiple sub-images; The first determination module 313 is used to determine the image coding information of each sub-image in the plurality of sub-images; The acquisition module 314 is used to acquire a coding information library, where the coding information library at least includes a plurality of image coding information determined by taking a screenshot of the chat interface last time; The second determination module 315 is used to determine the sub-image whose image coding information does not exist in the coding information library as the target sub-image; The third determination module 316 is used to determine the target chat information according to the target subgraph.

[0092] The target chat information determination device 310 of the embodiment of the present application also includes other modules for executing the various steps of the above-mentioned target chat information determination method embodiment on the target device, which will not be described one by one here.

[0093] Figure 5 FIG. 2 is a schematic diagram showing a structure of a device for determining target chat information provided by another embodiment of the present application. Figure 5 As shown, the device 320 includes: a receiving module 321, a generating module 322 and a sending module 323.

[0094] The receiving module 321 is used to receive the target chat information sent by the target device; The generating module 322 is used to generate display information according to the target chat information; The sending module 323 is used to send the display information to the main device for display.

[0095] The target chat information determination device 320 of the embodiment of the present application also includes other modules for executing the various steps of the above-mentioned target chat information determination method embodiment on the server, which will not be described one by one here.

[0096] Figure 5 A schematic diagram of the structure of a target chat information determination device provided by an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the target chat information determination device 400.

[0097] like Figure 5 As shown, the target chat information determination device 400 may include: a processor (processor) 402 and a memory (memory) 404.

[0098] The memory 404 is used to store a computer program 406. The memory 404 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory. The computer program 406 may include computer executable instructions.

[0099] The processor 402 is used to execute the computer program 406 to implement the steps in the embodiment of the method for determining the target chat information running on the above-mentioned target device, or the steps in the embodiment of the method for determining the target chat information running on the server.

[0100] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the target chat information determination device 400 may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0101] According to another aspect of the embodiment of the present application, a system for determining target chat information is also provided. Figure 7 The target chat information determination system 500 includes a target device 510 and a server 520 , and a communication connection is established between the target device 510 and the server 520 .

[0102] The target device 510 is used to respond to a screenshot trigger event, take a screenshot of the chat interface on the target device to obtain a chat interface image, wherein the chat interface image includes multiple chat messages arranged in a preset format; crop the chat interface image according to the preset format to obtain multiple sub-images; determine image coding information of each sub-image in the multiple sub-images; obtain a coding information library, the coding information library at least includes multiple image coding information determined when the chat interface was screenshoted last time; determine the sub-image whose image coding information does not exist in the coding information library as a target sub-image; determine the target chat information based on the target sub-image; and send the target chat information to the server.

[0103] The server 520 is used to receive the target chat information sent by the target device; generate display information according to the target chat information; and send the display information to the main device for display.

[0104] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the embodiment of the method for determining the target chat information are implemented.

[0105] An embodiment of the present application provides a computer program, which can be executed by a processor to implement the steps in the above-mentioned method for determining target chat information.

[0106] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in the embodiment of the method for determining the target chat information are implemented.

[0107] The algorithm or display provided here are not inherently related to any specific computer, virtual system or other equipment. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application.

[0108] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0109] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed application requires more features than those explicitly recited in each claim.

[0110] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0111] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware including a number of different elements and by means of a suitably programmed computer. In a unit claim that lists a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.< / service> < / application> < / manifest>

Claims

1. A method for determining target chat information, applied to a target device, characterized in that: The method for determining the target chat information includes: In response to a screenshot triggering event, taking a screenshot of the chat interface on the target device to obtain a chat interface image, wherein the chat interface image includes a plurality of chat messages arranged in a preset format; Cropping the chat interface image according to the preset format to obtain a plurality of sub-images; Determining image coding information of each sub-image in the plurality of sub-images; Acquire a coding information library, wherein the coding information library at least includes a plurality of image coding information determined by taking a screenshot of the chat interface last time; Determine a sub-picture whose image coding information does not exist in the coding information library as a target sub-picture; The target chat information is determined according to the target subgraph.

2. The method for determining target chat information according to claim 1, characterized in that: The method for determining the target chat information further includes: Determining an arrangement order among the plurality of target subgraphs and generating sequence information corresponding to the target subgraphs; The target chat information corresponding to the target subgraph and the sequence information corresponding to the target subgraph are sent to the server, so that the server splices the target chat information according to the sequence information.

3. The method for determining target chat information according to claim 2, characterized in that: The determining the target chat information according to the target subgraph further includes: Determine the target subgraph as target chat information; The sending of the target chat information corresponding to the target subgraph and the sequence information corresponding to the target subgraph to the server also includes: The target chat information and the sequence information corresponding to the target chat information are sent to the server, so that the server splices the target chat information according to the sequence information to obtain a display image.

4. The method for determining target chat information according to claim 1, characterized in that: After determining the subgraph whose image coding information does not exist in the coding information library as the target subgraph, the method for determining the target chat information further includes: The image coding information of the target sub-image is saved in the coding information library.

5. The method for determining target chat information according to claim 1, characterized in that: The chat interface image includes interface identification information corresponding to the chat interface, the multiple sub-images all correspond to the interface identification information of the chat interface image, and the multiple image encoding information in the encoding information library is image encoding information grouped according to different interface identification information; Before determining the subgraph whose image coding information does not exist in the coding information library as the target subgraph, the method for determining the target chat information further includes: Determining interface identification information of the chat interface image; The step of determining the sub-picture whose image coding information does not exist in the coding information library as the target sub-picture comprises: Determining whether the coding information library includes image coding information grouped according to the interface identification information; If so, a target coding information group is determined according to the image coding information grouped according to the interface identification information included in the coding information library, and a sub-graph in which the image coding information does not exist in the target coding information group is determined as a target sub-graph.

6. The method for determining target chat information according to claim 1, characterized in that: Before taking a screenshot of the chat interface on the target device to obtain a chat interface image in response to the screenshot triggering event, the method for determining the target chat information further includes: Monitor the input variables of the input box of the chat interface; If the input variable changes from a non-null value to a null value, the screenshot trigger event is generated.

7. The method for determining target chat information according to claim 1, characterized in that: The chat interface image also includes a plurality of user information corresponding to each of the chat messages; The step of cropping the chat interface image according to the preset format to obtain a plurality of sub-images includes: Scan the chat interface image to determine boundary information of multiple user information and boundary information of multiple chat information displayed on the chat interface image; Determine a plurality of first cropping boundaries on the chat interface image according to the boundary information of the plurality of user information; Determine a plurality of second cropping boundaries on the chat interface image according to the boundary information of the plurality of chat messages; The chat interface image is cropped according to the multiple first cropping boundaries and the multiple second cropping boundaries to obtain the multiple target sub-images.

8. A method for determining target chat information, applied to a server, characterized in that: The method for determining the target chat information includes: Receiving target chat information sent by a target device according to any one of claims 1 to 7; Generate display information according to the target chat information; The display information is sent to the main device for display.

9. The method for determining target chat information according to claim 8, characterized in that: The target chat information is the target subgraph. Before generating the display information according to the target chat information, the method for determining the target chat information further includes: Receiving sequence information corresponding to the target chat information sent by the target device; The generating display information according to the target chat information includes: splicing the target chat information according to the sequence information to obtain a display image; The sending the display information to the main device for display includes: The display image is sent to the main device for display.

10. A system for determining target chat information, characterized in that: It includes a target device and a server, wherein the target device and the server have a communication connection; the target device, In response to a screenshot triggering event, taking a screenshot of the chat interface on the target device to obtain a chat interface image, wherein the chat interface image includes a plurality of chat messages arranged in a preset format; Cropping the chat interface image according to the preset format to obtain a plurality of sub-images; Determining image coding information of each sub-image in the plurality of sub-images; Acquire a coding information library, wherein the coding information library at least includes a plurality of image coding information determined by taking a screenshot of the chat interface last time; Determine a sub-picture whose image coding information does not exist in the coding information library as a target sub-picture; Determine target chat information according to the target subgraph; Sending the target chat information to the server; the server, Used to receive target chat information sent by the target device; Generate display information according to the target chat information; The display information is sent to the main device for display.

Images