Display content analysis method, device, server and storage medium
By receiving and analyzing screenshots of screenshots of screenshots of electronic devices and using convolutional neural networks to distinguish target windows, the problem of inability to accurately and efficiently analyze users' office behavior in the existing technology is solved, efficient and accurate monitoring and analysis are achieved, and work efficiency is improved .
Patent Information
- Application Number
- CN202110875239.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-30
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-07-30
AI Technical Summary
Existing monitoring and analysis technologies cannot accurately and efficiently analyze whether users work normally. Users can bypass monitoring by modifying the process name, and the existing technology consumes human resources and computer resources, making the analysis efficiency inexpensive.
By receiving screenshots of the screen display content of the electronic device, input them into the pre-trained analysis model, use the convolutional neural network to distinguish between target windows and non-target windows, obtain user information and generation time, store them as analysis data, and perform corresponding results based on the analysis results. Operation to improve analysis efficiency.
It realizes accurate and efficient analysis of user office behavior, reduces manual operations by administrators, improves monitoring and analysis efficiency, and avoids users' bypassing monitoring.
Smart Images

Figure CN113608984B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communication technology, and in particular to a display content analysis method, device, server and storage medium. Background Art
[0002] At present, many corporate employees use computers for work. However, during working hours, some users browse software / websites that are not related to work, such as game software, chat software, shopping platforms, and securities trading. This behavior affects the work efficiency of employees and causes economic losses to the company. In view of this, in order to improve work efficiency, administrators hope to know whether computer users are in a normal working state through monitoring and analysis technology.
[0003] Existing monitoring and analysis technologies mainly include the following two types of technologies:
[0004] Prior art 1 determines whether the current operation is non-office software / website by process name. However, users can circumvent the monitoring of prior art 1 by modifying the process name, for example, by modifying the process name of the game software to the process name of the office software to bypass the monitoring. In addition, it is also impossible to accurately determine the browser operation behavior. For example, the process names of the online office system and the online shopping are the same, resulting in an inability to accurately analyze whether the user is working normally.
[0005] Prior art 2 uses a computer to send the computer screen to the administrator in real time, and the administrator watches and analyzes it manually in real time. However, the monitoring and analysis scheme of prior art 2 consumes human resources and computer resources, and has low analysis efficiency. If many computers are used at the same time, the administrator will not be able to watch at the same time, resulting in inadequate monitoring and analysis. Summary of the invention
[0006] The present invention provides a display content analysis method, device, server and storage medium, which are used to solve the defect in the prior art that it is impossible to accurately and efficiently analyze whether a user is working normally, and to achieve accurate and efficient analysis of the user's office behavior.
[0007] In a first aspect, the present invention provides a display content analysis method, comprising:
[0008] Receiving a screenshot to be analyzed of display content of a screen of the electronic device sent by the electronic device;
[0009] The screenshot to be analyzed is input into the analysis model to obtain the analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes a target window; the analysis model is trained based on sample screenshots and sample analysis results corresponding to the sample screenshots.
[0010] According to a display content analysis method provided by the present invention, after inputting the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model, the method further includes:
[0011] In a case where the target window is included in the screenshot to be analyzed, obtaining user information of the electronic device and a generation time of the screenshot to be analyzed;
[0012] The user information, the screenshot to be analyzed and the generation time are stored as a set of analysis data.
[0013] According to a display content analysis method provided by the present invention, after obtaining the user information of the electronic device and the generation time of the screenshot to be analyzed, the method further includes:
[0014] Execute a first operation corresponding to the type of the target window; wherein the first operation includes at least any one of the following:
[0015] Sending a first message to the electronic device; wherein the first message is used to instruct the electronic device to lock the screen of the electronic device;
[0016] Sending a second message to the electronic device; wherein the second message is used to instruct the electronic device to restart the operating system of the electronic device;
[0017] Send a third message to the target device; wherein the third message carries the user information and the generation time.
[0018] According to a display content analysis method provided by the present invention, after storing the user information, the screenshot to be analyzed and the generation time as a set of analysis data, the method further includes:
[0019] Retrieving at least one set of stored analysis data;
[0020] Classify and count the analysis data based on the target dimension to obtain statistical results.
[0021] In a second aspect, the present invention further provides a display content analysis method, comprising:
[0022] Taking a screenshot of the displayed content on the screen to obtain a screenshot of the displayed content to be analyzed;
[0023] Sending the screenshot of the displayed content to be analyzed to the server.
[0024] According to a display content analysis method provided by the present invention, after sending the screenshot of the display content to be analyzed to the server, the method further includes:
[0025] receiving a first message sent by the server, and locking the screen of the electronic device based on the first message; or,
[0026] A second message sent by the server is received, and an operating system of the electronic device is restarted based on the second message.
[0027] In a third aspect, the present invention further provides a display content analysis device, comprising:
[0028] A first receiving module, configured to receive a screenshot to be analyzed of display content on a screen of the electronic device sent by the electronic device;
[0029] The analysis module is used to input the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes a target window; the analysis model is trained based on the sample screenshot and the sample analysis result corresponding to the sample screenshot.
[0030] In a fourth aspect, the present invention further provides a display content analysis device, comprising:
[0031] A screenshot module, used to take a screenshot of the displayed content on the screen and obtain a screenshot of the displayed content to be analyzed;
[0032] The sending module is used to send the screenshot of the displayed content to be analyzed to the server.
[0033] In a fifth aspect, the present invention further provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the display content analysis method as described in the first aspect above are implemented.
[0034] In a sixth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the display content analysis method as described in the first aspect above.
[0035] The display content analysis method, device, server and storage medium provided by the present invention obtain a screenshot of the screen display content of the electronic device to be analyzed, input the screenshot to be analyzed into a pre-trained analysis model for analysis, and obtain an analysis result indicating whether the screenshot to be analyzed includes a target window; since the analysis model is a convolutional neural network obtained by deep training and learning based on a large number of sample screenshots and sample analysis results corresponding to the sample screenshots, it can accurately distinguish between target windows and non-target windows, and the analysis process does not require manual operation by the administrator, thereby improving the analysis efficiency, thereby achieving accurate and efficient analysis of users' office behavior. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0037] Figure 1 It is one of the flow charts of the display content analysis method provided by the present invention;
[0038] Figure 2 This is the second flow chart of the display content analysis method provided by the present invention;
[0039] Figure 3 This is the third flow chart of the display content analysis method provided by the present invention;
[0040] Figure 4 This is the fourth flow chart of the display content analysis method provided by the present invention;
[0041] Figure 5 It is one of the structural schematic diagrams of the display content analysis device provided by the present invention;
[0042] Figure 6 This is the second structural schematic diagram of the display content analysis device provided by the present invention;
[0043] Figure 7 It is a structural schematic diagram of the server provided by the present invention. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0045] The present invention provides a display content analysis method. Figure 1-Figure 4 The display content analysis method provided by the present invention is described.
[0046] Figure 1 is one of the flow charts of the display content analysis method provided by the present invention; Figure 1 As shown, the display content analysis method provided by the present invention includes:
[0047] Step 101: Receive a screenshot of the display content of the screen of the electronic device to be analyzed, which is sent by the electronic device.
[0048] Optionally, the display content analysis method provided by the present invention can be executed by a display content analysis device, or a control module in the display content analysis device for executing the display content analysis method. The display content analysis device can be, for example, a server, which is not specifically limited in the present invention. The screenshot to be analyzed can be a color image or a grayscale image.
[0049] Optionally, the electronic device takes a screenshot of the display content on the screen of the electronic device, obtains the screenshot of the display content to be analyzed, and then sends the screenshot to be analyzed to the server.
[0050] Step 102: input the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes the target window; the analysis model is trained based on the sample screenshots and the sample analysis results corresponding to the sample screenshots.
[0051] Optionally, the analysis model is, for example, a convolutional neural network, which is obtained through deep training and learning based on a large number of sample screenshots and sample analysis results corresponding to the sample screenshots. It can accurately distinguish between target windows and non-target windows. Non-target windows are office windows such as office software windows, while target windows are non-office windows such as game application windows, shopping websites, chat software windows, securities trading software windows, video playback software windows, etc.
[0052] Optionally, the target window may be a full-screen window or a non-full-screen window.
[0053] The method provided by the present invention obtains a screenshot to be analyzed of the screen display content of the electronic device, inputs the screenshot to be analyzed into a pre-trained analysis model for analysis, and obtains an analysis result indicating whether the screenshot to be analyzed includes a target window; since the analysis model is a convolutional neural network obtained by deep training and learning based on a large number of sample screenshots and sample analysis results corresponding to the sample screenshots, it can accurately distinguish between target windows and non-target windows, and the analysis process does not require manual operation by the administrator, thereby improving the analysis efficiency, thereby achieving accurate and efficient analysis of the user's office behavior.
[0054] Optionally, Figure 2 FIG. 2 is a flow chart of the display content analysis method provided by the present invention; the execution subject of the display content analysis method may be a server, such as Figure 2 As shown, the display content analysis method provided by the present invention includes:
[0055] Step 201: Receive a screenshot of the display content of the screen of the electronic device to be analyzed, which is sent by the electronic device.
[0056] Optionally, when the electronic device sends the screenshot to be analyzed to the server, it may also send the generation time of the screenshot to be analyzed to the server.
[0057] Step 202: input the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes the target window; the analysis model is trained based on the sample screenshots and the sample analysis results corresponding to the sample screenshots.
[0058] Optionally, the analysis model is a convolutional neural network model, which includes an input layer, a hidden layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; when constructing the analysis model, back propagation supervised learning is used to obtain a sample window image and a sample analysis result corresponding to the sample window image, grayscale the sample window image (i.e., the sample screenshot), and then extract the feature value through Fourier transform, and input the feature data into the input layer, the input layer is a two-dimensional pixel point input, the input layer is connected to the convolutional layer, the convolutional layer includes multiple convolution kernels to extract features from the input data, after the convolutional layer extracts features, the output data is passed to the pooling layer, the pooling layer performs feature selection and information filtering on the data, after the data is filtered, the fully connected layer performs nonlinear combination on the extracted features to obtain the label of the sample window image, the output layer compares the label of the sample window image with the sample analysis result corresponding to the sample window image, and obtains the accuracy; when the accuracy is lower than the preset value, the parameters of the convolutional neural network model are adjusted, and iterated again. Until the accuracy is greater than or equal to the preset value, the parameters and feature values of the convolutional neural network model are recorded at this time. All algorithm programs can optionally use graphics processing units (GPUs) for heterogeneous parallel computing to improve computing power.
[0059] Step 203: When the screenshot to be analyzed includes the target window, obtain user information of the electronic device and the generation time of the screenshot to be analyzed.
[0060] Optionally, after obtaining the analysis result output by the analysis model, the server obtains user information of the electronic device and generation time of the screenshot to be analyzed when it is known based on the analysis result that the target window is included in the screenshot to be analyzed.
[0061] Optionally, the user enters user name and other information on the electronic device in advance, and the electronic device sends the user information to the server; the user information may include, for example, user name, Internet Protocol (IP) address of the electronic device, Media Access Control (MAC) address and other information.
[0062] Optionally, after acquiring the user information of the electronic device and the generation time of the screenshot to be analyzed, the method further includes: performing a first operation corresponding to the type of the target window; wherein the first operation includes at least any one of the following operation modes:
[0063] Operation 1: The server sends a first message to the electronic device; wherein the first message is used to instruct the electronic device to lock the screen of the electronic device, so as to remind the user to terminate non-office activities and improve work efficiency.
[0064] Operation 2: The server sends a second message to the electronic device; wherein the second message is used to instruct the electronic device to restart the operating system of the electronic device, so as to remind the user to terminate non-office activities and improve work efficiency.
[0065] Operation 3: The server sends a third message to the target device, wherein the third message carries the user information and the generation time. Optionally, the target device is, for example, a user device of an administrator. The server notifies the administrator of the user's non-office behavior by sending the third message carrying the user information and the generation time to the target device, so that the administrator can manage and record it.
[0066] Optionally, different target window types correspond to different operation modes of the first operation. For example, when the target window type is a game application window, the operation mode of the first operation is operation 2; the server instructs the electronic device to restart the operating system of the electronic device through the second message, thereby reminding the user to terminate the non-office behavior of playing games. For another example, when the target window type is a shopping website, the operation mode of the first operation is operation 1; the server instructs the electronic device to lock the screen of the electronic device through the first message, thereby reminding the user to terminate the non-office behavior of browsing the shopping website.
[0067] Step 204: store the user information, the screenshot to be analyzed, and the generation time as a set of analysis data.
[0068] Optionally, if the target window is included in the screenshot to be analyzed, it means that the user is performing non-office activities, such as using non-office software or non-office websites. At this time, the user information, the screenshot to be analyzed and the generation time are stored as a set of analysis data to facilitate management and query of the analysis results.
[0069] Optionally, the server may perform statistical analysis on the stored multiple sets of analysis data, specifically by: obtaining at least one set of stored analysis data; classifying and counting each set of analysis data based on the target dimension to obtain statistical results. The target dimension may include at least one of the following: individual, department, non-office behavior, behavior duration, pie chart, or bar chart display. By performing statistical analysis on the stored analysis data, information such as the duration of the non-office behavior of the user or department can be calculated. For example, the duration of a certain employee opening a game application window during work hours can be calculated.
[0070] The method provided by the present invention can determine whether the screenshot to be analyzed includes the target window through the analysis model. If the screenshot to be analyzed includes the target window, the user information, the screenshot to be analyzed and the generation time are stored as a set of analysis data, which is convenient for managing and querying the analysis results and improving the analysis efficiency.
[0071] Figure 3 FIG. 3 is a flow chart of the display content analysis method provided by the present invention; the execution subject of the display content analysis method may be an electronic device, such as Figure 3 As shown, the display content analysis method provided by the present invention includes:
[0072] Step 301: Take a screenshot of the displayed content on the screen to obtain a screenshot of the displayed content to be analyzed.
[0073] Optionally, a screenshot is taken of the displayed content of the screen, that is, the displayed content of the screen is captured. The electronic device can periodically take a screenshot of the displayed content of the screen. The electronic device can automatically adjust the frequency of the screenshot operation according to information such as processor utilization and wireless network throughput.
[0074] Optionally, when the electronic device obtains the screenshot of the display content to be analyzed, it performs color conversion on the screenshot of the display content to be analyzed to obtain a grayscale image of the screenshot of the display content to be analyzed. Since the grayscale image occupies less bandwidth than the screenshot to be analyzed during network transmission, the electronic device can save bandwidth by sending the grayscale image to the server.
[0075] Step 302: Send the screenshot of the displayed content to be analyzed to the server.
[0076] Optionally, the electronic device sends a grayscale image corresponding to the screenshot to be analyzed to the server.
[0077] Optionally, the implementation of step 301 - step 302 does not rely on manual operation of the user, and the user cannot intervene.
[0078] For example, after the electronic device sends the screenshot of the display content to be analyzed to the server, when it receives the first message sent by the server, it locks the screen of the electronic device based on the first message, so as to remind the user to terminate non-office activities and improve work efficiency; wherein the first message is used to instruct the electronic device to lock the screen of the electronic device.
[0079] After the electronic device sends the screenshot of the display content to be analyzed to the server, when it receives the second message sent by the server, it restarts the operating system of the electronic device based on the second message, so as to remind the user to terminate non-office activities and improve work efficiency; wherein the second message is used to instruct the electronic device to restart the operating system of the electronic device.
[0080] The method provided by the present invention takes a screenshot of the displayed content on the screen, obtains the screenshot of the displayed content to be analyzed and sends it to the server, so that the server analyzes whether the screenshot to be analyzed includes the analysis result of the target window based on the convolutional network model. The analysis process does not require manual operation by the administrator, thereby improving the analysis efficiency.
[0081] Figure 4 4 is a flow chart of the display content analysis method provided by the present invention; the display content analysis method is executed by the electronic device and the server in cooperation, such as Figure 4 As shown, the display content analysis method provided by the present invention includes:
[0082] Step 401: The electronic device takes a screenshot of the display content on the screen to obtain a screenshot of the display content to be analyzed.
[0083] Step 402: the electronic device sends a screenshot of the display content to be analyzed to the server; the server receives the screenshot of the display content to be analyzed on the screen of the electronic device sent by the electronic device.
[0084] Step 403: The server inputs the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes the target window; the analysis model is trained based on the sample screenshots and the sample analysis results corresponding to the sample screenshots.
[0085] Step 404: When the screenshot to be analyzed includes the target window, the server obtains the user information of the electronic device and the generation time of the screenshot to be analyzed.
[0086] Step 405: The server stores the user information, the screenshot to be analyzed, and the generation time as a set of analysis data.
[0087] Step 406: The server sends a first message to the electronic device; wherein the first message is used to instruct the electronic device to lock the screen of the electronic device.
[0088] Step 407: When the electronic device receives the first message sent by the server, the electronic device locks the screen of the electronic device based on the first message.
[0089] The method provided by the present invention cooperates with an electronic device and a server. The electronic device takes a screenshot of the displayed content on the screen. The server inputs the screenshot to be analyzed into an analysis model to analyze whether the screenshot to be analyzed includes a target window. If the screenshot to be analyzed includes the target window, the server sends a first message to the electronic device. The electronic device locks the screen of the electronic device based on the first message, thereby reminding the user to terminate non-office activities and improve work efficiency.
[0090] The display content analysis device provided by the present invention is described below. The display content analysis device described below and the display content analysis method described above can be referenced to each other.
[0091] The present application embodiment provides a display content analysis device, Figure 5 is one of the structural diagrams of the display content analysis device provided in the embodiment of the present application, such as Figure 5 As shown, the display content analysis device includes: a first receiving module 501 and an analysis module 502; wherein:
[0092] A first receiving module 501 is used to receive a screenshot of the display content of the screen of the electronic device to be analyzed sent by the electronic device;
[0093] Analysis module 502 is used to input the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes the target window; the analysis model is trained based on the sample screenshot and the sample analysis result corresponding to the sample screenshot.
[0094] Optionally, the display content analysis device further includes:
[0095] A first acquisition module, configured to acquire user information of the electronic device and a generation time of the screenshot to be analyzed when the target window is included in the screenshot to be analyzed;
[0096] The storage module is used to store the user information, the screenshot to be analyzed and the generation time as a set of analysis data.
[0097] Optionally, the display content analysis device further includes:
[0098] An execution module, configured to execute a first operation corresponding to the type of the target window; wherein the first operation includes at least any one of the following:
[0099] Sending a first message to the electronic device; wherein the first message is used to instruct the electronic device to lock the screen of the electronic device;
[0100] Sending a second message to the electronic device; wherein the second message is used to instruct the electronic device to restart the operating system of the electronic device;
[0101] Send a third message to the target device; wherein the third message carries the user information and the generation time.
[0102] Optionally, the display content analysis device further includes:
[0103] A second acquisition module, used to acquire at least one set of stored analysis data;
[0104] The statistical module is used to classify and count the analysis data based on the target dimension to obtain statistical results.
[0105] The present application embodiment provides a display content analysis device, Figure 6 This is a second structural diagram of the display content analysis device provided in an embodiment of the present application, such as Figure 6 As shown, the display content analysis device includes: a screenshot module 601 and a sending module 602; wherein:
[0106] The screenshot module 601 is used to take a screenshot of the displayed content on the screen and obtain a screenshot of the displayed content to be analyzed;
[0107] The sending module 602 is used to send the screenshot of the displayed content to be analyzed to the server.
[0108] Optionally, the display content analysis device further includes: a second receiving module, or a third receiving module, wherein:
[0109] A second receiving module, configured to receive a first message sent by the server, and lock the screen of the electronic device based on the first message;
[0110] The third receiving module is used to receive a second message sent by the server, and restart the operating system of the electronic device based on the second message.
[0111] Figure 7 An example of a server physical structure diagram is shown below: Figure 7As shown, the server may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the display content analysis method, which includes: receiving a screenshot of the display content of the screen of the electronic device to be analyzed sent by an electronic device; inputting the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes a target window; the analysis model is trained based on the sample screenshot and the sample analysis result corresponding to the sample screenshot.
[0112] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0113] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the display content analysis method provided by the above-mentioned methods, and the method includes: receiving a screenshot of the display content of the screen of the electronic device to be analyzed sent by an electronic device; inputting the screenshot to be analyzed into an analysis model to obtain an analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes a target window; the analysis model is trained based on a sample screenshot and a sample analysis result corresponding to the sample screenshot.
[0114] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-mentioned display content analysis methods, the method comprising: receiving a screenshot to be analyzed of the display content of the screen of the electronic device sent by an electronic device; inputting the screenshot to be analyzed into an analysis model to obtain an analysis result output by the analysis model; the analysis result is used to indicate whether the screenshot to be analyzed includes a target window; the analysis model is trained based on a sample screenshot and a sample analysis result corresponding to the sample screenshot.
[0115] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A display content analysis method, characterized in that: include: Receiving a screenshot to be analyzed of display content of a screen of the electronic device sent by the electronic device; Inputting the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model; The analysis result is used to indicate whether the screenshot to be analyzed includes a target window; The analysis model is trained based on the sample screenshots and the sample analysis results corresponding to the sample screenshots; After inputting the screenshot to be analyzed into the analysis model to obtain the analysis result output by the analysis model, the display content analysis method further includes: In a case where the target window is included in the screenshot to be analyzed, obtaining user information of the electronic device and a generation time of the screenshot to be analyzed; storing the user information, the screenshot to be analyzed and the generation time as a set of analysis data; After storing the user information, the screenshot to be analyzed and the generation time as a set of analysis data, the method further includes: Retrieving at least one set of stored analysis data; Classify and count the analysis data based on the target dimension to obtain statistical results; The target dimensions include at least one of the following: individual, department, non-office behavior, behavior duration, pie chart and bar chart display; After acquiring the user information of the electronic device and the generation time of the screenshot to be analyzed, the display content analysis method further includes: Execute a first operation corresponding to the type of the target window; wherein the first operation includes at least any one of the following: Sending a first message to the electronic device; wherein the first message is used to instruct the electronic device to lock the screen of the electronic device; Sending a second message to the electronic device; wherein the second message is used to instruct the electronic device to restart the operating system of the electronic device; Send a third message to the target device; wherein the third message carries the user information and the generation time.
2. A display content analysis device, characterized in that: include: A first receiving module, configured to receive a screenshot to be analyzed of display content on a screen of the electronic device sent by the electronic device; An analysis module, used for inputting the screenshot to be analyzed into an analysis model to obtain an analysis result output by the analysis model; The analysis result is used to indicate whether the screenshot to be analyzed includes a target window; The analysis model is trained based on the sample screenshots and the sample analysis results corresponding to the sample screenshots; A first acquisition module, configured to acquire user information of the electronic device and a generation time of the screenshot to be analyzed when the target window is included in the screenshot to be analyzed; A storage module, used to store the user information, the screenshot to be analyzed and the generation time as a set of analysis data; A second acquisition module, used to acquire at least one set of stored analysis data; A statistical module is used to classify and count the analysis data based on the target dimension to obtain statistical results; An execution module, configured to execute a first operation corresponding to the type of the target window; wherein the first operation includes at least any one of the following: Sending a first message to the electronic device; wherein the first message is used to instruct the electronic device to lock the screen of the electronic device; Sending a second message to the electronic device; wherein the second message is used to instruct the electronic device to restart the operating system of the electronic device; Send a third message to the target device; wherein the third message carries the user information and the generation time.
3. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the display content analysis method according to claim 1 are implemented.
4. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the display content analysis method according to claim 1 are implemented.
Citation Information
Patent Citations
Working state identification method, server, storage medium and device
CN112348089A
Bad information identification method of network terminal and local area network terminal equipment
CN112633313A
Task log analysis method and system, computer equipment and storage medium
CN112860651A
Generating summary of activity on computer GUI
US20160349928A1
