Software content analysis method and device based on OCR and NLP, storage medium

Abstract

The application discloses an OCR and NLP-based software content analysis method and device and a computer readable storage medium. The method comprises the following steps: obtaining a preset process white list when a first process is running, the white list recording a plurality of second processes, and the first process having monitoring authority of the second processes; determining a process corresponding to content displayed by a current page as a target process when a screen click signal is detected; intercepting a to-be-monitored image through the first process when the target process belongs to the second processes; inputting the to-be-monitored image into the first process, performing character recognition on the to-be-monitored image through an OCR model of the first process, and obtaining target characters; inputting the target characters into an NLP model of the first process to perform semantic recognition, and obtaining a content analysis result. The application analyzes the to-be-monitored image through the OCR model and the NLP model of the first process to obtain the content analysis result, so as to realize risk assessment on page display content of a software process, and further improve user experience of internet software.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a software content analysis method and device based on OCR and NLP and a storage medium. BACKGROUND

[0002] In recent years, the number of middle-aged and elderly users of social software and information type software and other Internet software has gradually increased. A large part of the content on the network comes from personal publication and may carry some undesirable information. The middle-aged and elderly people have limited ability to identify Internet information and are easily affected by some undesirable information. Although most Internet software can filter undesirable information according to some keywords, it is difficult to use simple keywords to check some undesirable information disguised in chat content, and it is impossible to guarantee user experience and network security. SUMMARY

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present application provide a software content analysis method and device based on OCR and NLP and a storage medium, which can effectively improve the user experience of Internet software.

[0005] In a first aspect, the embodiments of the present application provide a software content analysis method based on OCR and NLP, comprising:

[0006] When the first process is in a running state, a preset process white list is obtained, the white list records a plurality of second processes, and the first process has monitoring authority of the second processes;

[0007] When a screen click signal is detected, a process corresponding to the content displayed by the current page is determined as a target process;

[0008] When the target process belongs to the second process, a to-be-monitored image is intercepted through the first process, and the to-be-monitored image is a current page image;

[0009] The to-be-monitored image is input into the first process, and a target character is obtained through an OCR model preset in the first process for character recognition;

[0010] The target character is input into an NLP model preset in the first process for semantic recognition to obtain a content analysis result.

[0011] In some embodiments, the OCR model comprises a convolutional recurrent neural network (CRNN) comprising a convolutional neural network (CNN), a long short-term memory network (LSTM), and a transcription layer connected in sequence; and the inputting the image to be monitored into the first process comprises performing character recognition by the OCR model preset in the first process to obtain target characters, which comprises:

[0012] extracting image features from the image to be monitored by the CNN;

[0013] extracting sequence features from the image features by the LSTM to obtain a plurality of sequence features;

[0014] inputting the plurality of sequence features into the transcription layer for sequence prediction conversion to obtain a target sequence label, and determining the target sequence label as the target characters.

[0015] In some embodiments, the inputting the plurality of sequence features into the transcription layer for sequence prediction conversion to obtain a target sequence label comprises:

[0016] calculating sequence label prediction probabilities of each of the sequence features according to a preset CTC algorithm;

[0017] determining a first sequence label, which is a sequence label corresponding to the label prediction probability with the largest numerical value of each of the sequence features;

[0018] merging and processing each of the first sequence labels to obtain the target sequence label.

[0019] In some embodiments, the inputting the target characters into the NLP model preset in the first process for semantic recognition to obtain a content analysis result comprises:

[0020] obtaining a preset candidate dictionary comprising a plurality of candidate keywords, each of which corresponds to a risk score value, wherein the candidate keywords represent keywords containing risk content;

[0021] determining a target keyword from the plurality of candidate keywords according to the target characters;

[0022] inputting all the target keywords into the NLP model to obtain a keyword analysis result;

[0023] determining the content analysis result according to all the keyword analysis results.

[0024] In some embodiments, the determining a target keyword from the plurality of candidate keywords according to the target characters comprises:

[0025] perform character segmentation processing on the target character according to a preset forward maximum matching algorithm to obtain at least two to-be-matched characters;

[0026] calculate Euclidean distances between each of the to-be-matched characters and each of the candidate keywords;

[0027] when the Euclidean distance is less than a preset threshold, determine the candidate keyword corresponding to the Euclidean distance as the target keyword.

[0028] In some embodiments, the content analysis result includes a content risk assessment value, and determining the content analysis result according to all the keyword analysis results includes:

[0029] obtaining a target score value and a target quantity from the keyword analysis result, wherein the target score value is a risk score value corresponding to the target keyword, and the target quantity is a quantity of the same target keyword;

[0030] calculating a product of the target score value and the target quantity to obtain the content risk assessment value.

[0031] In some embodiments, after inputting the target character into an NLP model preset in the first process to perform semantic recognition and obtaining a content analysis result, the method further includes:

[0032] when the content risk assessment value is greater than a preset content risk threshold, determining process attribute information corresponding to the target process, and generating risk prompt information according to the process attribute information, the content risk assessment value, and the target keyword corresponding to the content risk assessment value;

[0033] determining user information corresponding to a target software, the target software being a software corresponding to the target process;

[0034] obtaining pre-set contact number information from the user information;

[0035] sending the risk prompt information according to the contact number information.

[0036] In a second aspect, an embodiment of the present application provides a software content analysis device based on OCR and NLP, comprising:

[0037] a process white list obtaining module, configured to, when a first process is in a running state, obtain a preset process white list, the white list recording a plurality of second processes, and the first process having monitoring authority of the second processes;

[0038] A target process determination module is configured to determine a process corresponding to content displayed by a current page as a target process when a screen click signal is detected.

[0039] A to-be-monitored image acquisition module is configured to acquire a to-be-monitored image through the first process when the target process belongs to the second process, the to-be-monitored image being a current page image.

[0040] A target character acquisition module is configured to input the to-be-monitored image into the first process, perform character recognition through an OCR model preconfigured in the first process, and obtain target characters.

[0041] A content analysis result acquisition module is configured to input the target characters into an NLP model preconfigured in the first process, perform semantic recognition, and obtain a content analysis result.

[0042] In a third aspect, an embodiment of the present application further provides an OCR and NLP-based software content analysis device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the OCR and NLP-based software content analysis method according to the first aspect when executing the computer program.

[0043] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program, and the computer program is used to execute the OCR and NLP-based software content analysis method according to the first aspect.

[0044] The embodiment of the present application comprises the following steps: when a first process is in a running state, a preconfigured process white list is acquired, the white list recording a plurality of second processes, and the first process having a monitoring permission of the second processes; when a screen click signal is detected, a process corresponding to content displayed by a current page is determined as a target process; when the target process belongs to the second process, a to-be-monitored image is acquired through the first process, the to-be-monitored image being a current page image; the to-be-monitored image is input into the first process, character recognition is performed through an OCR model preconfigured in the first process, and target characters are obtained; and the target characters are input into an NLP model preconfigured in the first process, semantic recognition is performed, and a content analysis result is obtained. According to the scheme provided by the embodiment of the present application, the to-be-monitored image of the target process can be acquired through the first process, and the content analysis result can be obtained by analyzing and processing the to-be-monitored image through the OCR model and the NLP model, so that risk assessment of page display content of a software process is realized, and user experience of internet software is improved.

[0045] Other features and advantages of the present application will be set forth in the description that follows, and in part will be apparent from the description, or can be learned by practice of the application. The purposes and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are used to explain the technical solutions of the application, and do not constitute a limitation on the technical solutions of the application.

[0047] Figure 1 is a step flow chart of the method for analyzing software content based on OCR and NLP provided by an embodiment of the application;

[0048] Figure 2 is a step flow chart of obtaining a target character provided by another embodiment of the application;

[0049] Figure 3 is a step flow chart of obtaining a target sequence label provided by another embodiment of the application;

[0050] Figure 4 is a step flow chart of obtaining a content analysis result provided by another embodiment of the application;

[0051] Figure 5 is a step flow chart of determining a target keyword provided by another embodiment of the application;

[0052] Figure 6 is a step flow chart of determining a content analysis result provided by another embodiment of the application;

[0053] Figure 7 is a step flow chart of sending a risk prompt information provided by another embodiment of the application;

[0054] Figure 8 is a module schematic diagram of the device for analyzing software content based on OCR and NLP provided by another embodiment of the application;

[0055] Figure 9 is a structural diagram of the device for analyzing software content based on OCR and NLP provided by another embodiment of the application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0057] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the sequence in the flowchart. The terms "first", "second", etc. in the specification, claims or above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0058] The application provides an OCR and NLP-based software content analysis method and device and a storage medium. The software content analysis method comprises: obtaining a preset process white list when a first process is in a running state, the white list recording a plurality of second processes, and the first process having monitoring authority of the second processes; determining a process corresponding to content displayed by a current page as a target process when a screen click signal is detected; intercepting a to-be-monitored image of the target process through the first process when the target process belongs to the second process, the to-be-monitored image being a current page image; inputting the to-be-monitored image into the first process, performing character recognition through an OCR model preset in the first process to obtain target characters; and inputting the target characters into an NLP model preset in the first process to perform semantic recognition and obtain a content analysis result. According to the scheme provided in the application, the to-be-monitored image of the target process can be intercepted through the first process, and the to-be-monitored image can be analyzed and processed through the OCR model and the NLP model to obtain the content analysis result, so that risk assessment of page display content of a software process is realized, and user experience of internet software is improved.

[0059] The embodiments of the application can compile, obtain and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0060] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, operation / interaction devices, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0061] NLP is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close connection with linguistic research. Natural Language Processing techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0062] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, tracking, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0063] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0064] like Figure 1 As shown, Figure 1 This is a flowchart illustrating the steps of a software content analysis method based on OCR and NLP provided in an embodiment of the present invention. The software content analysis method includes, but is not limited to, the following steps:

[0065] Step S110: While the first process is running, a preset process whitelist is obtained. The whitelist contains multiple second processes, and the first process has monitoring permissions for the second processes.

[0066] Step S120: When a screen click signal is detected, the process corresponding to the content displayed on the current page is determined as the target process;

[0067] Step S130: When the target process belongs to the second process, the image to be monitored is captured by the first process. The image to be monitored is the current page image.

[0068] Step S140: Input the image to be monitored into the first process, and perform text recognition through the OCR model preset in the first process to obtain the target character;

[0069] In step S150, the target character is input to the NLP model preset in the first process for semantic recognition to obtain a content analysis result.

[0070] It can be understood that the software content analysis method based on OCR and NLP provided by the embodiments of the present application can be applied to a terminal, the first process is a process in a monitoring software installed in the terminal, the process white list records a plurality of different second processes, and each second process belongs to a different software. Since the first process has monitoring authority over each second process in the process white list, when the first process is in a running state, the terminal can implement implementation content monitoring of the running second process through the first process and obtain a software content analysis result for the second process. The specific process can be as follows: when the terminal detects a screen click signal, a process corresponding to the content displayed on the current page of the terminal is determined as a target process; when it is detected that the target process belongs to a second process recorded in the process white list, a to-be-monitored image is intercepted through the first process, the to-be-monitored image is a current page image, the to-be-monitored image is input to the first process, text recognition is performed on the to-be-monitored image through an OCR model preset in the first process, target characters are obtained, the target characters are input to an NLP model preset in the first process for semantic recognition, and a content analysis result is obtained. Through the OCR model and the NLP model of the first process, the content analysis result is obtained by analyzing the to-be-monitored image, so as to implement risk assessment on the page display content of the software process, and thus the user experience of the Internet software is improved.

[0071] It should be noted that the first process in the embodiments of the present application can be a system monitoring software pre-installed in the terminal, or a monitoring software that can be selected and installed by the user of the terminal. The first process can provide background monitoring services for the system layer of the terminal (i.e., the first process corresponding to the monitoring software has monitoring authority over the second processes in the process white list), and the specific form of the first process is not limited in the embodiments of the present application. The system type required for installation of the monitoring software corresponding to the first process is not limited in the embodiments of the present application, and can be determined by a person skilled in the art according to actual needs.

[0072] It should be noted that the embodiments of the present application do not limit the time period for intercepting the to-be-monitored image through the first process. The to-be-monitored image can be a current page image of the target process that is intercepted in real time through the first process, or the to-be-monitored image can be a current page image of the target process that is obtained through the first process according to a preset time period, and the selection can be made by a person skilled in the art according to actual needs.

[0073] It should be noted that while the to-be-monitored image corresponding to the target process is obtained through the first process, target software information, usage time information, and the like corresponding to the target process are recorded, which provides an effective data basis for analyzing the content information corresponding to the to-be-monitored image and obtaining a more accurate content analysis result.

[0074] It should be noted that the OCR model can adopt a common type, and the specific selection of the model is not limited in the embodiment, and the OCR recognition can be realized. The target character can be all characters in the character region in the to-be-monitored image, and the character in the embodiment includes letters and numbers, which is not limited here.

[0075] In addition, in an embodiment, the OCR model includes a convolutional recurrent neural network (CRNN), and the CRNN includes a convolutional neural network (CNN), a long short-term memory (LSTM), and a transcription layer connected in sequence. Referring to Figure 2 , Figure 1 The step S140 in the embodiment shown includes but is not limited to the following steps:

[0076] Step S210, image feature extraction is performed on the to-be-monitored image through the CNN to obtain image features;

[0077] Step S220, sequence feature extraction is performed on the image features through the LSTM to obtain a plurality of sequence features;

[0078] Step S230, the plurality of sequence features are input into the transcription layer for sequence prediction conversion to obtain a target sequence label, and the target sequence label is determined as the target character.

[0079] In addition, referring to Figure 3 , in an embodiment, Figure 2 The step S230 in the embodiment shown includes but is not limited to the following steps:

[0080] Step S310, the sequence label prediction probability of each sequence feature is calculated according to a preset CTC algorithm;

[0081] Step S320, a first sequence label is determined, and the first sequence label is a sequence label corresponding to the label prediction probability with the largest value of each sequence feature;

[0082] Step S330, data merging processing is performed on each first sequence label to obtain a target sequence label.

[0083] It can be understood that, since the OCR model provided by the embodiment of the present application comprises a convolutional recurrent neural network (CRNN), the CRNN comprises a convolutional neural network (CNN), a long short-term memory (LSTM) and a transcription layer connected in sequence; the CNN is configured to perform image feature extraction on a to-be-monitored image input into the OCR model to obtain image features; the LSTM is configured to perform further sequence feature extraction on the image features processed by the CNN to obtain a plurality of sequence features; and the transcription layer is configured to convert the plurality of sequence features to obtain target characters. The following describes a process in which the OCR model preset in the first process performs character recognition on a to-be-monitored image to obtain target characters by taking a specific example: an expression of the image features obtained by performing image feature extraction on the to-be-monitored image by the CNN is as follows:

[0084] x = (x 1 ,x 2 ,...,x T );

[0085] Each column of the image features x is input into the LSTM as a time slice, and the size of the image features x is m*T; an expression of each column x t of the image features x is as follows:

[0086]

[0087] An expression of the LSTM is as follows:

[0088] y = NET w (x);

[0089] wherein w is a feature parameter of the LSTM, and wherein NET w represents a data transformation process of the LSTM between input and output, and NET w can be expressed as:

[0090] NET w :(R m ) T →(R n ) T ;

[0091] A softmax layer is connected after each time slice input into the LSTM, and is configured to perform normalization processing on each time slice; the sequence features output after the image features x are input into the LSTM are a posterior probability matrix, and a specific expression is as follows:

[0092] y = (y 1 ,y 2 ,...,y t ,...,y T );

[0093] wherein each column y of the sequence feature y t The expression of y is as follows:

[0094]

[0095] wherein n is the length of the character set to be recognized; subject to Each column y of the sequence feature y t is input into the argmax function to obtain the class information of the output character corresponding to each column y t , and then the sequence feature y is input into the transcription layer for sequence prediction conversion to obtain the target sequence label. The process can be as follows: calculating the sequence label prediction probability of each sequence feature according to a preset CTC algorithm; determining a first sequence label, the first sequence label being a sequence label corresponding to the largest label prediction probability of each sequence feature; performing data merging processing on each first sequence label to obtain the target sequence label, and determining the target sequence label as the target character.

[0096] It should be noted that the expression of the CTC algorithm can be as follows:

[0097]

[0098] wherein the expression of p (π|y) can be as follows:

[0099]

[0100] wherein is the probability of predicting the character π t at time t, and the character with the highest probability at the same timestamp in the above formula is taken as the predicted output. The output character set is the target character.

[0101] It should be noted that, in order to improve the accuracy of the content analysis result obtained by the OCR model, the OCR model can be trained by using a preset loss function. In the embodiments of the present application, the negative logarithm of the conditional probability is taken as the loss function of the OCR model, and the formula is as follows:

[0102]

[0103] In addition, with reference to Figure 4 , in an embodiment, Figure 1 The step S150 in the embodiment shown includes but is not limited to the following steps:

[0104] Step S410, obtaining a preset candidate dictionary, the candidate dictionary including a plurality of candidate keywords, each candidate keyword corresponding to a risk score value, wherein the candidate keyword represents a keyword containing risk content;

[0105] Step S420, determining the target keyword from the plurality of candidate keywords according to the target character;

[0106] Step S430, inputting all the target keywords into the NLP model to obtain a keyword analysis result;

[0107] Step S440, determining a content analysis result according to all the keyword analysis results.

[0108] It can be understood that, in order to improve the accuracy of NLP model recognition, a candidate dictionary with a plurality of candidate keywords can be preset, each candidate keyword corresponding to a risk score value, wherein the candidate keyword represents a keyword containing risk content. By matching the target character with the plurality of candidate keywords in the candidate dictionary, the target keyword is determined, which can ensure that the NLP model uses the correct dictionary for semantic recognition, avoids using a large general dictionary, and effectively improves the efficiency of semantic recognition.

[0109] It should be noted that the NLP model can be a common model, and the specific structure of the NLP model is not limited in the embodiment. As long as the semantic recognition can be realized based on the target dictionary, it is acceptable.

[0110] In addition, with reference to Figure 5 In an embodiment, Figure 4 The step S420 in the embodiment includes but is not limited to the following steps:

[0111] Step S510, performing character segmentation processing on the target character according to a preset forward maximum matching algorithm to obtain at least two to-be-matched characters;

[0112] Step S520, calculating the Euclidean distance between each to-be-matched character and each candidate keyword;

[0113] Step S530, when the Euclidean distance is smaller than a preset threshold, determining the candidate keyword corresponding to the Euclidean distance as the target keyword.

[0114] It can be understood that, by performing character segmentation processing on the target character according to the preset forward maximum matching algorithm, at least two to-be-matched characters are obtained. The Euclidean distance between each to-be-matched character and each candidate keyword is calculated. When the Euclidean distance is smaller than a preset threshold, it can be considered that the matching is successful, otherwise it is directly skipped as an unknown new word, which can effectively improve the recognition accuracy of the target keyword.

[0115] In addition, with reference to Figure 6 In an embodiment, the content analysis result includes a content risk assessment value, Figure 4 The step S440 in the embodiment includes but is not limited to the following steps:

[0116] Step S610, obtaining a target score value and a target quantity from the keyword analysis result, wherein the target score value is a risk score value corresponding to the target keyword, and the target quantity is a quantity of the same target keyword;

[0117] Step S620, calculating a product of the target score value and the target quantity to obtain a content risk assessment value.

[0118] It is worth noting that the keyword analysis result for the target keyword can be obtained through the NLP model, and the keyword analysis result can include the content risk assessment value. The calculation method of the content risk assessment value can be: obtaining a target score value and a target quantity from the keyword analysis result, wherein the target score value is a risk score value corresponding to the target keyword, and the target quantity is a quantity of the same target keyword (i.e., the frequency of occurrence of the target keyword in the target character); calculating a product of the target score value and the target quantity to obtain a content risk assessment value; for example, the frequency of occurrence of the target keyword A in the target character, i.e., the target quantity is 10, the target score value of the target keyword A is 10, and the content risk assessment value of the target character is the product of the target score value and the target quantity = 100.

[0119] In addition, with reference to Figure 7 In an embodiment, after step S150 in the embodiment shown in the figure is performed, the software content analysis method based on OCR and NLP provided by the embodiment of the present application further includes but is not limited to the following steps: Figure 1

[0120] Step S710, when the content risk assessment value is greater than a preset content risk threshold value, determining process attribute information corresponding to the target process, and generating risk prompt information according to the process attribute information, the content risk assessment value, and the target keyword corresponding to the content risk assessment value;

[0121] Step S720, determining user information corresponding to the target software, the target software being software corresponding to the target process;

[0122] Step S730, obtaining pre-set contact number information from the user information;

[0123] Step S740, sending the risk prompt information according to the contact number information.

[0124] ​It can be understood that when it is detected that the content risk evaluation value is greater than the preset content risk threshold value, it indicates that the current terminal user watches the risk content information through the target software, at this time, by determining the process attribute information of the target process corresponding to the current page image, the process attribute information can include the target software name corresponding to the target process, the time information of using the target software, and the keyword information corresponding to the current page image, and the like; and according to the process attribute information, the content risk evaluation value and the target keyword corresponding to the content risk evaluation value, the risk prompt information is generated, the user information corresponding to the target software is determined, the pre-set contact number information is obtained from the user information, and the risk prompt information is sent according to the contact number information; Because the identification ability of the middle-aged and old people for the Internet information is limited, they are easily affected by some bad information, in the case that the current terminal user is a middle-aged and old person, the user corresponding to the pre-set contact number information can be the children of the middle-aged and old people, in the case that the current terminal user watches the risk content information through the target software, according to the contact number information, the risk prompt information is sent to the children of the middle-aged and old people in time, so as to avoid the influence of the network bad information on the middle-aged and old people, and effectively guarantee the user experience of the Internet software.

[0125] In addition, with reference to Figure 8 , Figure 8 is a schematic diagram of a software content analysis device based on OCR and NLP provided by another embodiment of the present application. An embodiment of the present application further provides a software content analysis device based on OCR and NLP 800, which comprises:

[0126] A process white list acquisition module 810 is configured to acquire a preset process white list when the first process is in a running state. The white list records a plurality of second processes, and the first process has a monitoring permission of the second processes.

[0127] A target process determination module 820 is configured to determine a process corresponding to content displayed by a current page as a target process when a screen click signal is detected.

[0128] A to-be-monitored image acquisition module 830 is configured to acquire a to-be-monitored image through the first process when the target process belongs to the second process. The to-be-monitored image is a current page image.

[0129] A target character acquisition module 840 is configured to input the to-be-monitored image into the first process, perform character recognition through an OCR model preset in the first process, and obtain target characters.

[0130] The content analysis result obtaining module 850 is configured to input the target character into an NLP model preset in the first process to perform semantic recognition and obtain a content analysis result.

[0131] In addition, with reference to Figure 9 , Figure 9 is a structural diagram of an OCR and NLP-based software content analysis device according to another embodiment of the present application. The present application also provides an OCR and NLP-based software content analysis device 900, which comprises a memory 910, a processor 920, and a computer program stored in the memory 910 and executable on the processor 920.

[0132] The processor 920 and the memory 910 can be connected through a bus or other means.

[0133] The non-transitory software program and instructions required for implementing the OCR and NLP-based software content analysis method according to the above embodiments are stored in the memory 910, and when executed by the processor 920, the OCR and NLP-based software content analysis method according to the above embodiments is performed, for example, the method steps S110 to S150 in the above-described Figure 1 , the method steps S210 to S230 in the above-described Figure 2 , the method steps S310 to S330 in the above-described Figure 3 , the method steps S410 to S440 in the above-described Figure 4 , the method steps S510 to S530 in the above-described Figure 5 , the method steps S610 to S620 in the above-described Figure 6 , and the method steps S710 to S740 in the above-described Figure 7 .

[0134] The device embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0135] In addition, the present application also provides a computer readable storage medium storing computer executable instructions, which are executed by a processor or controller, for example, a processor 920 in the above-described OCR and NLP-based software content analysis device 900, so that the above-described processor executes the OCR and NLP-based software content analysis method according to the above embodiments, for example, executes the above-described Figure 1the method steps S110 to S150 in the method of Figure 2 the method steps S210 to S230 in the method of Figure 3 the method steps S310 to S330 in the method of Figure 4 the method steps S410 to S440 in the method of Figure 5 the method steps S510 to S530 in the method of Figure 6 the method steps S610 to S620 in the method of Figure 7 the method steps S710 to S740 in the method of. Those skilled in the art will understand that all or some of the steps in the above-disclosed methods, systems can be implemented as software, firmware, hardware, or suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those skilled in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves or other transport mechanisms, and includes any information delivery media.

[0136] The above description is specific to the preferred embodiments of the present application, but the present application is not limited to the above-described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and such equivalent modifications or substitutions are included in the scope of the present application as defined by the claims.

[0137] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable programs for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowcharts, and combinations of blocks in the block diagrams or flowcharts, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0138] It should be noted that in each of the specific embodiments of the present application, when it is necessary to perform relevant processing according to user information, user behavior data, user historical data, and user location information, and other data related to the identity or characteristics of the user, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards in the relevant countries and regions. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for enabling the embodiments of the present application to function normally will be obtained.

[0139] The units involved in the embodiments of the present application can be implemented in a software manner, or can be implemented in a hardware manner, and the described units can also be arranged in a processor. In some cases, the names of these units do not constitute a limitation on the units themselves.

[0140] It should be noted that although several modules or units of the device for action execution are mentioned in the foregoing detailed description, such a division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units.

[0141] Those skilled in the art can clearly understand that the example embodiments described in the specification can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) execute the methods according to the embodiments of the present application.

[0142] The terminal of the embodiment can include a radio frequency (RF) circuit, a memory, an input unit, a display unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a processor, and a power supply, etc. The RF circuit can be used for receiving and sending signals in the process of information or call, especially, receiving the downlink information of the base station and processing by the processor, and sending the uplink data to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to global system for mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc. The memory can be used to store software programs and modules, and the processor can execute various function applications and data processing of the terminal by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the terminal (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. The input unit can be used to receive input digital or character information, and generate key signal input related to the setting and function control of the terminal. Specifically, the input unit can include a touch panel and other input devices. The touch panel, also known as a touch screen, can collect touch operations (such as using a finger, a stylus, etc. Any suitable object or accessory on or near the touch panel, or operations near the touch panel) on it or near it, and drive the corresponding connection device according to the pre-set program. Optionally, the touch panel can include a touch detection device and a touch controller two parts.The touch detection device detects a touch position and detects a signal caused by a touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts the touch information into touch coordinates, and sends the touch coordinates to the processor. The touch controller can also receive commands from the processor and execute the commands. In addition, the touch panel can be implemented in various types such as a resistive type, a capacitive type, an infrared type, and a surface acoustic wave type. In addition to the touch panel, the input unit can include other input devices. Specifically, the other input devices can include one or more of a physical keyboard, function keys (such as volume control keys, on / off keys, etc.), a trackball, a mouse, a joystick, etc. The display unit can be used to display input information or provided information and various menus of the terminal. The display unit can include a display panel, which can be configured in the form of a liquid crystal display (LCD), an organic light emitting diode (OLED), etc. Further, the touch panel can cover the display panel, and when the touch panel detects a touch operation thereon or in the vicinity thereof, the touch panel transmits the touch event to the processor to determine a type of the touch event, and then the processor provides a corresponding visual output on the display panel according to the type of the touch event. The touch panel and the display panel are implemented as two independent components to perform input and output functions of the terminal, but in some embodiments, the touch panel and the display panel can be integrated to perform the input and output functions of the terminal. The terminal can also include at least one sensor such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel according to the brightness of ambient light, and the proximity sensor can turn off the display panel and / or the backlight when the terminal is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and when stationary, can detect the magnitude and direction of gravity, and can be used for applications such as recognizing the posture of the terminal (such as switching between landscape and portrait screens, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometers, taps), etc. As for other sensors that can be configured in the terminal, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., they will not be described here. The audio circuit, the speaker, and the microphone can provide an audio interface. The audio circuit can convert the received audio data into an electrical signal, transmit the electrical signal to the speaker, and convert the electrical signal into a sound signal output by the speaker. On the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit and converted into audio data. The audio data is output to the processor for processing, and then transmitted to another terminal via the RF circuit, or output to the memory for further processing.

[0143] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope of the application being indicated by the following claims.

[0144] It is to be understood that the application is not limited to the precise details of construction and the above-described embodiments and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should only be limited by the claims appended hereto.

[0145] The above is a specific description of the preferred embodiment of the present application, but the present application is not limited to the above-described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application. These equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A software content analysis method based on OCR and NLP, characterized in that, include: While the first process is running, a preset process whitelist is obtained. The whitelist contains multiple second processes, and the first process has monitoring permissions for the second processes. When a screen tap signal is detected, the process corresponding to the content currently displayed on the page is identified as the target process; When the target process belongs to the second process, the image to be monitored is captured by the first process, and the image to be monitored is the current page image. The image to be monitored is input into the first process, and the target character is obtained by performing text recognition through the OCR model preset in the first process; The target character is input into an NLP model preset in the first process for semantic recognition to obtain content analysis results; The content analysis results include a content risk assessment value. After inputting the target character into an NLP model preset in the first process for semantic recognition to obtain the content analysis results, the method further includes: When the content risk assessment value is greater than the preset content risk threshold, the process attribute information corresponding to the target process is determined, and risk warning information is generated based on the process attribute information, the content risk assessment value, and the target keyword corresponding to the content risk assessment value. Determine the user information corresponding to the target software, wherein the target software is the software corresponding to the target process; Obtain pre-set contact number information from the user information; The risk warning message will be sent based on the contact number information.

2. The method according to claim 1, characterized in that, The OCR model includes a convolutional recurrent neural network (CRNN), which comprises a convolutional neural network (CNN), a long short-term memory network (LSTM), and a transcription layer connected in sequence. The step of inputting the image to be monitored into the first process and performing text recognition using the OCR model preset in the first process to obtain the target character includes: The image features are obtained by extracting image features from the image to be monitored using the CNN; The image features are extracted using the LSTM to obtain multiple sequence features; Multiple sequence features are input into the transcription layer for sequence prediction and transformation to obtain target sequence labels, and the target sequence labels are determined as the target characters.

3. The method according to claim 2, characterized in that, The step of inputting multiple sequence features into the transcription layer for sequence prediction and transformation to obtain target sequence labels includes: The sequence label prediction probability of each sequence feature is calculated according to the preset CTC algorithm; Determine a first sequence label, which is the sequence label corresponding to the label prediction probability with the largest value among all the sequence features; The target sequence label is obtained by merging the data of each of the first sequence labels.

4. The method according to claim 1, characterized in that, The step of inputting the target character into an NLP model preset in the first process for semantic recognition to obtain content analysis results includes: Obtain a preset candidate dictionary, which includes multiple candidate keywords, each of which corresponds to a risk score value, wherein the candidate keywords represent keywords containing risky content; The target keyword is determined from a plurality of candidate keywords based on the target character; Input all the target keywords into the NLP model to obtain the keyword analysis results; The content analysis results are determined based on all the keyword analysis results.

5. The method according to claim 4, characterized in that, The step of determining the target keyword from a plurality of candidate keywords based on the target character includes: The target character is segmented according to the preset positive maximum matching algorithm to obtain at least two characters to be matched; Calculate the Euclidean distance between each of the characters to be matched and each of the candidate keywords; When the Euclidean distance is less than a preset threshold, the candidate keyword corresponding to the Euclidean distance is determined as the target keyword.

6. The method according to claim 4, characterized in that, The step of determining the content analysis result based on all the keyword analysis results includes: The target score and target quantity are obtained from the keyword analysis results, wherein the target score is the risk score corresponding to the target keyword, and the target quantity is the number of identical target keywords; The content risk assessment value is obtained by multiplying the target score by the target quantity.

7. A software content analysis device based on OCR and NLP, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the software content analysis method based on OCR and NLP as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing computer-executable instructions for performing the software content analysis method based on OCR and NLP as described in any one of claims 1 to 6.

Images