Data processing method, storage medium and computer terminal

Abstract

The application discloses a data processing method, a storage medium and a computer terminal. The method comprises the following steps: receiving a search instruction for first source data; obtaining a target index of the first source data through a target log space based on the search instruction; and calling the first source data from a source database based on the target index. The application solves the technical problem of low efficiency of problem troubleshooting of source data through a log in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and in particular, to a data processing method, a storage medium and a computer terminal. BACKGROUND

[0002] At present, a daily important part of work of a software developer is to handle online problems, and a general processing procedure is that the developer discovers problems in an online application program through monitoring or user feedback, then logs in a machine or a log system to check corresponding log data, analyzes and locates a general range according to the log data, opens local source code to find source code corresponding to the range, analyzes a cause of the problem in combination with the source code, and handles the problem, which needs to switch repeatedly between the log and the source code, and thus causes low problem checking efficiency.

[0003] For the above problems, no effective solution has been proposed. SUMMARY

[0004] Embodiments of the present application provide a data processing method, a storage medium and a computer terminal, to at least solve the technical problem of low problem checking efficiency of source data through a log in the related art.

[0005] According to an aspect of an embodiment of the present application, a data processing method is provided, including: receiving a retrieval instruction for first source data; obtaining a target index of the first source data through a target log space based on the retrieval instruction; and calling the first source data from a source database based on the target index.

[0006] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: receiving, by a cloud server, a retrieval instruction for first source data; obtaining, by the cloud server, a target index of the first source data through a target log space based on the retrieval instruction; and calling, by the cloud server, the first source data from a source database based on the target index.

[0007] According to another aspect of an embodiment of the present application, a data processing apparatus is also provided, including: a receiving module configured to receive a retrieval instruction for first source data; an obtaining module configured to obtain a target index of the first source data through a target log space based on the retrieval instruction; and a calling module configured to call the first source data from a source database based on the target index.

[0008] According to another aspect of an embodiment of the present application, a data processing apparatus is also provided, including: a receiving module configured to receive, by a cloud server, a retrieval instruction for first source data; an obtaining module configured to obtain, by the cloud server, a target index of the first source data through a target log space based on the retrieval instruction; and a calling module configured to call, by the cloud server, the first source data from a source database based on the target index.

[0009] According to another aspect of the embodiments of the present application, a storage medium is also provided, which comprises a stored program, wherein the program, when executed, controls a device in which the storage medium is located to perform the data processing method described above.

[0010] According to another aspect of the embodiments of the present application, a computer terminal is also provided, comprising a processor and a memory, wherein the processor is configured to execute a program stored in the memory, and the program, when executed, performs the data processing method described above.

[0011] In the embodiments of the present application, first, a search instruction for first source data is received; a target index of the first source data is obtained from a target log space based on the search instruction; and the first source data is called from a source database based on the target index, thereby improving the troubleshooting efficiency of the source data in the source database. It is easy to note that after receiving the search instruction for the first source data, the target index of the first source data is directly obtained from the target log space, and the first source data in the source data can be accurately located and called according to the target index, without the need for the user to switch between the log space and the source database, thereby improving the troubleshooting efficiency of the R&D or operation and maintenance personnel for the application program, making correct processing in a short time, thereby reducing the impact of the fault on the user, and further solving the technical problem of low efficiency of troubleshooting of the source data by the log in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are included to provide a further understanding of the present application and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0013] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing the data processing method according to an embodiment of the present application;

[0014] Figure 2 is a flowchart of a data processing method according to an embodiment of the present application;

[0015] Figure 3 is a flowchart of filtering a target word according to an embodiment of the present application;

[0016] Figure 4 is a flowchart of splitting first source data according to an embodiment of the present application;

[0017] Figure 5 is a structural schematic diagram of a code hosting system and a log system according to an embodiment of the present application;

[0018] Figure 6is a flow chart of another data processing method according to an embodiment of the present application;

[0019] Figure 7 is a schematic diagram of a data processing device according to an embodiment of the present application;

[0020] Figure 8 is a schematic diagram of another data processing device according to an embodiment of the present application;

[0021] Figure 9 is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiment of the present application will be described clearly and completely in combination with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, not all. Based on the embodiment in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the protection scope of the present application.

[0023] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0024] First, some of the nouns or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

[0025] Code hosting platform: is a platform that provides code data storage, historical version management, code review and other main research and development activities and services for programming developers. The code hosting platform is divided into local installation version and cloud product.

[0026] Log service: is a service that collects log data generated during the running of an application online, which solves the tool of mass data collection, storage, and fast query and analysis.

[0027] Source code: refers to the source code data developed by programmers.

[0028] Currently, the online problem positioning solutions are all based on online log data. There are two types of logs, the first type is detailed log data, and the developer customizes the output content; the second type is link tracking log, and the logs are concatenated according to the user request link; neither of them can be directly associated with the specific source code that causes the problem through the log data, so as to quickly analyze and troubleshoot the application. They need to frequently switch between the log system and the source code system, and the code needs to be downloaded to the local in advance to troubleshoot, which is low in efficiency and poor in process experience.

[0029] In order to improve the efficiency of online troubleshooting of code problems by R&D and operation personnel, the application provides a data processing method.

[0030] Embodiment 1

[0031] According to the embodiments of the application, a data processing method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] The method embodiment provided by the embodiment one of the application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the data processing method is shown. As shown in Figure 1 , the computer terminal 10 (or mobile device 10) can include one or more processors (the processor can include but is not limited to a microprocessor MCU or a programmable logic device FPGA processing device), a memory 104 for storing data, and a transmission module 106 for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, and it does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or fewer components than those shown in Figure 1 , or have a different configuration from Figure 1 .

[0033] It should be noted that the one or more processors and / or other data processing circuitry described above can be referred to herein generally as "data processing circuitry." The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. Furthermore, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any one of the other elements of the computer terminal 10 (or mobile device). As referred to in the embodiments of the present application, the data processing circuitry serves as a processor to control, for example, the selection of the variable resistance terminal path in connection with the interface.

[0034] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the data processing method in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the data processing method described above. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory disposed remotely with respect to the processor, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (NIC) that can be connected to other network devices through a base station so as to be able to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module used to communicate with the Internet in a wireless manner.

[0036] The display can be, for example, a touch screen type liquid crystal display (LCD) that can enable a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0037] It should be noted that in some optional embodiments, the above-mentioned Figure 1 The computer device (or mobile device) shown can include hardware elements (including circuitry), software elements (including computer code stored on a computer readable medium), or a combination of both hardware and software elements. It should be noted that Figure 1 is merely one example of a particular implementation and is intended to illustrate the types of components that can be present in the computer device (or mobile device) described above.

[0038] In the above operating environment, the application provides a data processing method as shown in Figure 2 Figure 2 is a flowchart of the data processing method according to an embodiment of the application.

[0039] In step S202, a search instruction for the first source data is received.

[0040] The first source data described above can be data stored in a source database. The first source data can be code type data, Chinese type data, or numerical type data, and the type of the first source data is not limited.

[0041] The search instruction described above can be generated according to target data. The target data can be part of the first source data. When the target data has a problem, a search instruction can be generated according to the target data, so that the complete data corresponding to the target data, that is, the first source data, can be retrieved according to the search instruction, so as to solve the problem of the target data by troubleshooting the first source data. The target data can be target code.

[0042] In an optional embodiment, the first source data can be data that needs to be retrieved to check whether a fault occurs. When a code segment or part of the first source data has a problem, a search instruction for the first source data can be generated according to the code segment or part of the first source data.

[0043] In another optional embodiment, the search instruction can realize multi-dimensional data search capability, including but not limited to general code segment search, file search, class segment search, and method segment search. In the search process by the search instruction, security check of the search content can also be performed, including sensitive word search interception and sensitive data desensitization.

[0044] In step S204, a target index of the first source data is acquired from a target log space based on the search instruction.

[0045] The log space described above is mainly used to collect log data generated by an application program or a system during running.

[0046] The target index described above can be stored in the target log space, and the target index described above can also be stored in the source database.

[0047] The target index described above can be an index constructed according to the first source data when the first source data is stored in the source database. The target index can be used to quickly retrieve the first source data corresponding to the search instruction according to log data in the log space. ​

[0048] In an optional embodiment, after obtaining the retrieval instruction, the corresponding log data can be queried according to the information recorded in the retrieval instruction, and the corresponding target index in the source database can be matched according to the corresponding log data, so as to retrieve the first source data from the source database according to the target index.

[0049] In step S206, the first source data is retrieved from the source database based on the target index.

[0050] The target index described above is constructed based on the structured data of the first source data when it is constructed, so that the first source data can be accurately retrieved from the source database through the target index, thereby improving the efficiency of obtaining the first source data.

[0051] The source database described above can contain source code.

[0052] In an optional embodiment, the first source data can be retrieved from the source database according to the target index, so that the user can repair the faulty code or data through the first source data.

[0053] In another optional embodiment, after the first source data is retrieved, the first source data can be displayed on the interface of the client to facilitate direct viewing by the user.

[0054] Through the above steps, first, the retrieval instruction for the first source data is received; the target index of the first source data is obtained from the target log space based on the retrieval instruction; and the first source data is retrieved from the source database based on the target index, thereby improving the troubleshooting efficiency of the source data in the source database. It is easy to note that after receiving the retrieval instruction for the first source data, the target index of the first source data is directly obtained from the target log space, and the first source data in the source data can be accurately located and retrieved according to the target index, without the need for the user to switch between the log space and the source database, so that the troubleshooting efficiency of the R&D or operation and maintenance personnel for the application program can be improved, the correct processing can be made in a short time, thereby reducing the impact of the fault on the user, and thereby solving the technical problem of low efficiency of problem troubleshooting of the source data through the log in the related art.

[0055] In the above embodiments of the present application, the method further includes: obtaining the first source data; structuring the first source data based on the target field to generate second source data, wherein the second source data is the source data corresponding to the target field in the first source data; constructing the target index based on the second source data; and synchronizing the target index to the target log space.

[0056] The first source data described above can be source code.

[0057] The target field described above can include, but is not limited to, a file name (File_name), a file path (File_path), a code repository path (Repo_path) where the file is located, a code committer (commiter), a code contributor (author), a programming language (language) corresponding to the code, a class set in an abstract syntax tree structure (Ast_class_set), a method set in the abstract syntax tree structure (Ast_method_set), and source code text content (File_bolob).

[0058] In an optional embodiment, a plurality of first source data can be collected by a source data collection module, the plurality of first source data can be read by task parallel processing, and unstructured first source data can be converted into structured second source data for subsequent use by an index construction module; the index construction module can roughly describe an abstract based on the structured second source data, and needs to divide the second source data to obtain data that needs to be retained in the second source data, and construct a target index based on the abstract and the retained data, and synchronize the target index to a target log space for subsequent use for querying.

[0059] In the above embodiments of the present application, the first source data is structured based on the target field to generate the second source data, including: determining a target abstract of the first source data based on the target field; obtaining a data type of the first source data; splitting the first source data based on the data type to obtain a target split result; and generating the second source data based on the target abstract and the target split result.

[0060] The target abstract described above includes, but is not limited to, a code file repository, a file path, and an author.

[0061] For source data being code text, since code is a special text, unlike natural language text, there is no specific language library, and different coding styles result in different code text content. In the present application, in order to construct a target index for different types of code text, a code tokenizer is provided by the code index construction module, which can accurately tokenize different types of code text. The tokenizer mainly solves the problem of retaining meaningless words in code with common semantics, handles various coding naming styles (such as camel case, underscore format, etc.), and syntax tree structure.

[0062] In an optional embodiment, a target summary of the first source data can be determined according to the target field, and then the data type of the first source data can be obtained. The first source data can be split according to the splitting mode corresponding to the different data types to obtain a target splitting result. The second source data can be generated according to the target summary and the target splitting result. Since the second source data is structured data, the target index described above can be quickly constructed through the second source data.

[0063] In the above embodiments of the present application, the first source data is split based on the data type to obtain a target splitting result, including: splitting the first source data based on the data type to obtain an initial splitting result; filtering the target words in the initial splitting result to generate the target splitting result, wherein the target words are stop words and / or sensitive words in the initial splitting result.

[0064] In an optional embodiment, the first source data can be split according to the data type to obtain an initial splitting result. After obtaining the initial splitting result, it can be detected whether the target words appear in the initial splitting result. If the target words appear, the target words can be filtered to generate the target splitting result. If the target words do not appear, the initial splitting result can be directly used as the target splitting result.

[0065] In another optional embodiment, before the first source data is split according to the data type, all strings in the first source data can be traversed, and then the strings in the first source data can be filtered. Optionally, special strings such as *, ¥, #, etc. can be filtered, and web tags can also be filtered.

[0066] Figure 3 A flowchart for filtering target words according to an embodiment of the present application is shown in FIG. 3, including the following steps: Figure 3

[0067] S31, obtaining at least one word in the initial splitting result;

[0068] S32, performing lowercase conversion on the at least one word to obtain a conversion result;

[0069] S33, judging whether there is a stop word in the conversion result based on the stop word list. If yes, step S34 is performed, and if no, step S35 is performed.

[0070] Optionally, the stop word list can be loaded through a programming language, and whether there is a stop word in the conversion result can be judged based on the stop word list.

[0071] S34, deleting the stop word in the at least one word to obtain a first deletion result;

[0072] ​S35, judging whether the sensitive word exists in the deletion result based on the sensitive word library, if yes, executing step S36, if no, executing step S37;

[0073] Optionally, the sensitive word library can be loaded first, and whether the sensitive word exists in the deletion result is judged based on the sensitive word library.

[0074] S36, deleting the sensitive word in the at least one word to obtain a second deletion result;

[0075] S37, generating a target split result according to the at least one word, the first deletion result and the second deletion result.

[0076] Optionally, the stop word and the sensitive word in the at least one word can be deleted according to the first deletion result and the second deletion result.

[0077] In another optional embodiment, when detecting whether the target word appears in the initial split result, a rule matching sensitive information recognition technology can be used for recognition, wherein the rule matching has good accuracy and expansibility; an information entropy algorithm can also be used, wherein the information entropy algorithm is used to measure the degree of disorder of code lines, and has good recognition effect on randomly generated keys and random identity information; template clustering and context semantic analysis methods can also be used for filtering, and common keywords are extracted according to the information entropy result aggregation, and the accuracy of the information entropy algorithm is improved combined with the context semantics and the current syntax structure.

[0078] In the above embodiments of the application, the first source data is split based on the data type to obtain an initial split result, including: in the case that the data type of the first source data is a first type, splitting the first source data by using a first tokenizer to obtain a target split result, wherein the first type is used to represent that the data type is a Chinese type.

[0079] The above-mentioned first type can be that the data type of the first source data is a Chinese type, that is, the first source data can be Chinese characters.

[0080] The above-mentioned first tokenizer can be a Chinese tokenizer.

[0081] In an optional embodiment, in the case that the data type of the first source data is a Chinese type, the first source data can be tokenized by using a Chinese tokenizer, and optionally, the Chinese tokenizer can split the first source data according to a Chinese word library to obtain the above-mentioned target split result.

[0082] In the above embodiment of the present application, the first source data is split based on the data type to obtain an initial split result, including: in the case where the data type of the first source data is a second type, the string of the first source data is split based on camel case to obtain a first string, wherein the second type is a format of the first source data as camel case; the first string is split using an underscore to obtain a second string; the second string is split using a second tokenizer to obtain the initial split result.

[0083] The camel case format can be a coding mode using the upper and lower cases of letters. The camel case format can be divided into a small camel case format and a large camel case format, wherein the first single word of the small camel case format can be a lower case letter, and the first letter of the second single word can be an upper case letter, for example: firstName; the first letter of each single word of the large camel case format can be an upper case letter, for example: FirstName.

[0084] The second tokenizer can be a standard tokenizer, wherein the standard tokenizer can be a tokenizer for processing English grammar.

[0085] In an optional embodiment, in the case where the data type of the first source data is camel case, the string of the first source data can be split according to the camel case in the first source data to obtain a first string; after obtaining the first string, the underscore in the first string can be searched, and the first string can be split according to the underscore to obtain a second string, so as to split the second string using the above-mentioned standard tokenizer to obtain the initial split result. If a sensitive word is found during the stage of inputting the first source data by the user, the sensitive word is directly intercepted, and if a sensitive word is found after obtaining the initial split result, desensitization processing can be performed.

[0086] In the above embodiment of the present application, the first source data is split based on the data type to obtain an initial split result, including: in the case where the data type of the first source data is a third type, the string of the first source data is split based on an underscore to obtain a third string, wherein the third type is a format of the first source data as underscore format; the third string is split using a second tokenizer to obtain the initial split result.

[0087] The underscore format refers to a form of splitting data using an underscore.

[0088] In an optional embodiment, in the case where the data type of the first source data is underscore format, the first source data can be directly split according to the underscore to obtain a third string, and the third string is split using a standard tokenizer to obtain the initial split result.

[0089] Figure 4As shown in a flowchart for splitting first source data according to an embodiment of the present application, the flowchart comprises the following steps: Figure 4

[0090] S40, obtaining first source data;

[0091] S41, judging whether the first source data is of Chinese type, if yes, executing step S42, if no, executing step S43;

[0092] S42, splitting the first source data by using a Chinese tokenizer to obtain an initial splitting result;

[0093] S43, judging whether the first source data is of camel case, if yes, executing step S44, if no, executing step S45;

[0094] S44, splitting a first string corresponding to the first source data according to camel case to obtain the first string;

[0095] S45, judging whether the first string is of underscore format, if yes, executing step S46, if no, executing step S47;

[0096] S46, splitting the first string by using underscore to obtain a second string;

[0097] S47, judging whether the first source data is of underscore format, if yes, executing step S48, if no, executing step S49;

[0098] S48, splitting the first source data by using underscore to obtain a third string;

[0099] S49, splitting the second string or the third string by using a standard tokenizer to obtain the initial splitting result.

[0100] In the above embodiment of the present application, the second string is split by using the second tokenizer to obtain the initial splitting result, which comprises: splitting the third string by using the second tokenizer based on space to obtain a fourth string; splitting the fourth string based on punctuation by using the second tokenizer to obtain the initial splitting result.

[0101] In an optional embodiment, the standard tokenizer can split the third string based on space to obtain a fourth string, so that there is no space in the fourth string obtained by splitting, and after obtaining the fourth string, the second tokenizer can further split the fourth string based on punctuation to obtain the initial splitting result, so that there is no punctuation in the initial splitting result obtained.

[0102] ​Similarly, the standard word segmenter can also split the second string by the same steps as described above to obtain the initial split result.

[0103] Figure 5 A structural diagram of a code hosting system and a log system according to an embodiment of the present application is shown in FIG. 1. The code hosting system includes the source database described above, and the log system includes the target log space described above. During the code development stage, a user can obtain and save the developed code data, i.e., the first source data, and construct a target index according to the first source data. The source database can be associated with the target index. During the code publishing process, the code in the source database can be published and run, and the running log data can be saved to the log space during the running process. When a problem occurs in the code, the target segment can be selected for code diagnosis according to the log data. The retrieval instruction can be generated according to the target segment, the target index can be obtained according to the retrieval instruction, and the first source data can be retrieved from the source database based on the target index.

[0104] According to the above embodiments of the present application, the key information in the log data can be used to accurately find the complete source code from the corresponding code repository. The one-stop online problem troubleshooting process from the log to the code can be completed without switching the context. The specific implementation process is as follows. During the daily development process, a developer will print logs in the code. The printed logs will generally record the following information: the time and location (file name, class name, method name, and code line information) of the fault occurrence, the cause of the fault, and the context information of the code running. When a fault occurs in the production environment, the developer will obtain the specific log data recorded in advance by querying the log. The system will use the location information recorded in the log, such as the file name, class name, and method name, as input information to search in the constructed code index data, and return the source code that meets the conditions to the developer, thereby realizing the quick association from the log to the source code. The entire process is processed in seconds and does not depend on whether the developer is familiar with the code, greatly improving the efficiency of online problem troubleshooting and positioning. Through the code search function, the complete source code can be accurately and quickly queried according to the recorded code segment in the log and displayed on a page for thread analysis by the troubleshooting personnel, thereby optimizing the troubleshooting process and improving the overall analysis efficiency.

[0105] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0106] Through the description of the above embodiments, those skilled in the art can clearly understand that the data processing method according to the above embodiments can be realized by means of software and necessary general hardware platforms, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method of each embodiment of the present application.

[0107] Embodiment 2

[0108] According to the embodiments of the present application, a data processing method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0109] Figure 6 is a data processing method flowchart according to an embodiment of the present application, as shown in Figure 6 The method can include the following steps:

[0110] Step S602, the cloud server receives a retrieval instruction for the first source data.

[0111] Step S604, the cloud server acquires the target index of the first source data through the target log space based on the retrieval instruction.

[0112] Step S606, the cloud server calls the first source data from the source database based on the target index.

[0113] In the above embodiments of the present application, the method further includes: the cloud server acquires the first source data; the cloud server performs structured processing on the first source data based on the target field to generate second source data, wherein the second source data is source data corresponding to the target field in the first source data; the cloud server constructs the target index based on the second source data; and the cloud server synchronizes the target index to the target log space.

[0114] In the above embodiments of the present application, the cloud server performs structured processing on the first source data based on the target field to generate second source data, including: the cloud server determines a target abstract of the first source data based on the target field; the cloud server acquires a data type of the first source data; the cloud server splits the first source data based on the data type to obtain a target splitting result; and the cloud server generates the second source data based on the target abstract and the target splitting result.

[0115] In the above embodiments of the application, the cloud server splits the first source data based on the data type to obtain a target splitting result, including: the cloud server splits the first source data based on the data type to obtain an initial splitting result; the cloud server filters a target word in the initial splitting result to generate the target splitting result, wherein the target word is a stop word and / or a sensitive word in the initial splitting result.

[0116] In the above embodiments of the application, the cloud server splits the first source data based on the data type to obtain an initial splitting result, including: in a case where the data type of the first source data is a first type, the cloud server splits the first source data by using a first tokenizer to obtain a target splitting result, wherein the first type is used to represent a Chinese type.

[0117] In the above embodiments of the application, the cloud server splits the first source data based on the data type to obtain an initial splitting result, including: in a case where the data type of the first source data is a second type, the cloud server splits a string of the first source data based on a camel case to obtain a first string, wherein the second type is a camel case format of the first source data; the cloud server splits the first string by using an underscore to obtain a second string; and the cloud server splits the second string by using a second tokenizer to obtain the initial splitting result.

[0118] In the above embodiments of the application, the cloud server splits the first source data based on the data type to obtain an initial splitting result, including: in a case where the data type of the first source data is a third type, the cloud server splits a string of the first source data based on an underscore to obtain a third string, wherein the third type is an underscore format of the first source data; and the cloud server splits the third string by using a second tokenizer to obtain the initial splitting result.

[0119] In the above embodiments of the application, the cloud server splits the second string by using the second tokenizer to obtain an initial splitting result, including: the cloud server splits the third string by using the second tokenizer based on a space to obtain a fourth string; and the cloud server splits the fourth string based on the second tokenizer based on a punctuation symbol to obtain the initial splitting result.

[0120] It should be noted that the preferred embodiments involved in the above embodiments of the application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0121] Embodiment 3

[0122] According to the embodiments of the application, a data processing device for implementing the above data processing method is also provided, such as Figure 7As shown, the apparatus 700 includes a receiving module 702, an obtaining module 704, and a calling module 706.

[0123] The receiving module is configured to receive a search instruction for first source data. The obtaining module is configured to obtain a target index of the first source data based on the search instruction through a target log space. The calling module is configured to call the first source data from a source database based on the target index.

[0124] It should be noted that the receiving module 702, the obtaining module 704, and the calling module 706 correspond to steps S202 to S206 in Embodiment 1, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the disclosure in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the apparatus.

[0125] In the above embodiments of the present application, the apparatus further includes a processing module, a constructing module, and a synchronizing module.

[0126] The obtaining module is further configured to obtain the first source data. The processing module is configured to perform structured processing on the first source data based on the target field to generate second source data, where the second source data is source data corresponding to the target field in the first source data. The constructing module is configured to construct the target index based on the second source data. The synchronizing module is configured to synchronize the target index to the target log space.

[0127] In the above embodiments of the present application, the processing module includes a determining unit, an obtaining unit, a splitting unit, and a generating unit.

[0128] The determining unit is configured to determine a target summary of the first source data based on the target field. The obtaining unit is configured to obtain a data type of the first source data. The splitting unit is configured to split the first source data based on the data type to obtain a target splitting result. The generating unit is configured to generate the second source data based on the target summary and the target splitting result.

[0129] In the above embodiments of the present application, the splitting unit includes a splitting subunit and a filtering subunit.

[0130] The splitting subunit is configured to split the first source data based on the data type to obtain an initial splitting result. The filtering subunit is configured to filter a target word in the initial splitting result to generate the target splitting result, where the target word is a stop word and / or a sensitive word in the initial splitting result.

[0131] In the above embodiments of the present application, the splitting unit is further configured to split the first source data by using a first tokenizer to obtain an initial splitting result, in a case where the data type of the first source data is a first type, wherein the first type is used to represent a Chinese type.

[0132] In the above embodiments of the present application, the splitting unit is further configured to split the string of the first source data based on a camel case to obtain a first string, in a case where the data type of the first source data is a second type, wherein the second type is a camel case format of the first source data; the splitting unit is further configured to split the first string by using an underscore to obtain a second string; and the splitting unit is further configured to split the second string by using a second tokenizer to obtain the initial splitting result.

[0133] In the above embodiments of the present application, the splitting unit is further configured to split the string of the first source data based on an underscore to obtain a third string, in a case where the data type of the first source data is a third type, wherein the third type is an underscore format of the first source data; and the splitting unit is further configured to split the third string by using the second tokenizer to obtain the initial splitting result.

[0134] In the above embodiments of the present application, the splitting unit is further configured to split the third string by using the second tokenizer based on a space to obtain a fourth string; and the splitting unit is further configured to split the fourth string based on the second tokenizer based on a punctuation symbol to obtain the initial splitting result.

[0135] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0136] Embodiment 4

[0137] According to the embodiments of the present application, a data processing device for implementing the above data processing method is further provided, as shown in Figure 8 The device 800 includes a receiving module 802, an obtaining module 804 and a calling module 806.

[0138] The receiving module is configured to receive a search instruction for the first source data through a cloud server; the obtaining module is configured to obtain a target index of the first source data through a target log space based on the search instruction through the cloud server; and the calling module is configured to call the first source data from a source database based on the target index through the cloud server.

[0139] It should be noted that the receiving module 802, the obtaining module 804, and the calling module 806 correspond to steps S602 to S606 in Embodiment 2, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the above-mentioned embodiments 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0140] Embodiment 5

[0141] The embodiments of the present application can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Alternatively, in the present embodiment, the above-mentioned computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0142] Alternatively, in the present embodiment, the above-mentioned computer terminal can be located in at least one network device of a plurality of network devices of a computer network.

[0143] In the present embodiment, the above-mentioned computer terminal can execute program codes of the following steps in the data processing method: receiving a search instruction for first source data; obtaining a target index of the first source data through a target log space based on the search instruction; and calling the first source data from a source database based on the target index.

[0144] Alternatively, Figure 9 is a structural block diagram of a computer terminal according to an embodiment of the present application. As shown in Figure 9 The computer terminal A can include one or more (only one is shown in the figure) processors, a memory.

[0145] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, i.e., implements the above-mentioned data processing method. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal A through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0146] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: receiving a search instruction for first source data; obtaining a target index of the first source data through a target log space based on the search instruction; and calling the first source data from a source database based on the target index.

[0147] Optionally, the processor can further execute program codes of the following steps: obtaining the first source data; performing a structural processing on the first source data based on the target field to generate second source data, wherein the second source data is source data in the first source data corresponding to the target field; constructing the target index based on the second source data; and synchronizing the target index to the target log space.

[0148] Optionally, the processor can further execute program codes of the following steps: determining a target summary of the first source data based on the target field; obtaining a data type of the first source data; splitting the first source data based on the data type to obtain a target splitting result; and generating the second source data based on the target summary and the target splitting result.

[0149] Optionally, the processor can further execute program codes of the following steps: splitting the first source data based on the data type to obtain an initial splitting result; filtering a target word in the initial splitting result to generate the target splitting result, wherein the target word is a stop word and / or a sensitive word in the initial splitting result.

[0150] Optionally, the processor can further execute program codes of the following steps: in a case where the data type of the first source data is a first type, splitting the first source data by using a first tokenizer to obtain an initial splitting result, wherein the first type is used to indicate that the data type is a Chinese type.

[0151] Optionally, the processor can further execute program codes of the following steps: in a case where the data type of the first source data is a second type, splitting a string of the first source data based on a camel case to obtain a first string, wherein the second type is a camel case format of the first source data; splitting the first string by using an underscore to obtain a second string; and splitting the second string by using a second tokenizer to obtain an initial splitting result.

[0152] Optionally, the processor can further execute program codes of the following steps: in a case where the data type of the first source data is a third type, splitting a string of the first source data based on an underscore to obtain a third string, wherein the third type is an underscore format of the first source data; and splitting the third string by using the second tokenizer to obtain an initial splitting result.

[0153] Optionally, the processor can further execute program codes of the following steps: splitting the third string by using the second tokenizer based on a space to obtain a fourth string; and splitting the fourth string by using the second tokenizer based on a punctuation symbol to obtain an initial splitting result.

[0154] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: the cloud server receives a retrieval instruction for the first source data; the cloud server obtains a target index of the first source data through the target log space based on the retrieval instruction; and the cloud server calls the first source data from the source database based on the target index.

[0155] With the embodiment of the application, first, a retrieval instruction for first source data is received; a target index of the first source data is obtained through a target log space based on the retrieval instruction; and the first source data is called from a source database based on the target index, thereby improving the troubleshooting efficiency of source data in the source database. It is easy to note that after receiving the retrieval instruction for the first source data, the target index of the first source data is directly obtained from the target log space, the first source data in the source data can be accurately located and called according to the target index, and the user does not need to switch the log space and the source database back and forth, thereby improving the troubleshooting efficiency of the application program by the R&D or operation and maintenance personnel, making correct processing in a short time, reducing the influence of the fault on the user, and further solving the technical problem of low efficiency of troubleshooting of source data by logs in the related art.

[0156] Those skilled in the art can understand that, Figure 9 The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or the like. Figure 9 It does not limit the structure of the electronic device. For example, the computer terminal 10 can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 9 For example, the computer terminal 10 can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 9 For example, the computer terminal 10 can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0157] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by programs instructing the related hardware of the terminal device, and the programs can be stored in a computer readable storage medium, which can include a flash disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc.

[0158] Embodiment 6

[0159] The embodiment of the application further provides a storage medium. Optionally, in the embodiment, the storage medium can be used to save the program code executed by the data processing method provided in Embodiment 1.

[0160] Optionally, in the embodiment, the storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0161] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: obtaining the first source data; performing structural processing on the first source data based on the target field to generate second source data, wherein the second source data is source data corresponding to the target field in the first source data; constructing the target index based on the second source data; and synchronizing the target index to the target log space.

[0162] Optionally, the storage medium is further configured to store program code for performing the following steps: determining a target summary of the first source data based on the target field; obtaining a data type of the first source data; splitting the first source data based on the data type to obtain a target splitting result; and generating the second source data based on the target summary and the target splitting result.

[0163] Optionally, the processor can further execute program code for performing the following steps: splitting the first source data based on the data type to obtain an initial splitting result; and filtering a target word in the initial splitting result to generate the target splitting result, wherein the target word is a stop word and / or a sensitive word in the initial splitting result.

[0164] Optionally, the storage medium is further configured to store program code for performing the following steps: in a case where the data type of the first source data is a first type, splitting the first source data by using a first tokenizer to obtain an initial splitting result, wherein the first type is used to indicate that the data type is a Chinese type.

[0165] Optionally, the storage medium is further configured to store program code for performing the following steps: in a case where the data type of the first source data is a second type, splitting a string of the first source data based on camel case to obtain a first string, wherein the second type is a camel case format of the first source data; splitting the first string by using an underscore to obtain a second string; and splitting the second string by using a second tokenizer to obtain the initial splitting result.

[0166] Optionally, the storage medium is further configured to store program code for performing the following steps: in a case where the data type of the first source data is a third type, splitting a string of the first source data based on an underscore to obtain a third string, wherein the third type is an underscore format of the first source data; and splitting the third string by using the second tokenizer to obtain the initial splitting result.

[0167] Optionally, the storage medium is further configured to store program code for splitting the third string based on spaces by the second tokenizer to obtain a fourth string, and splitting the fourth string based on punctuations by the second tokenizer to obtain an initial splitting result.

[0168] Optionally, in the embodiment, the storage medium is configured to store program code for receiving, by the cloud server, a retrieval instruction for the first source data, obtaining, by the cloud server, a target index of the first source data from the target log space based on the retrieval instruction, and calling, by the cloud server, the first source data from the source database based on the target index.

[0169] With the embodiment of the application, first, a retrieval instruction for the first source data is received, a target index of the first source data is obtained from the target log space based on the retrieval instruction, and the first source data is called from the source database based on the target index, thereby improving the troubleshooting efficiency of the source data in the source database. It is easy to note that after receiving the retrieval instruction for the first source data, the target index of the first source data is directly obtained from the target log space, the first source data in the source data can be accurately located and called according to the target index, and the user does not need to switch the log space and the source database back and forth, thereby improving the troubleshooting efficiency of the application program by the R&D or operation and maintenance personnel, making correct processing in a short time, reducing the impact of the fault on the user, and further solving the technical problem of low efficiency of troubleshooting of the source data by the log in the related art.

[0170] The serial numbers of the above embodiments of the application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0171] In the above embodiments of the application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0172] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0173] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0174] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0175] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the prior art that essentially contributes or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various program code storage media.

[0176] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A data processing method, characterized by, The method comprises: receiving a search instruction for first source data; obtaining a target index of the first source data based on the search instruction through a target log space; calling the first source data from a source database based on the target index; splitting the first source data based on a data type of the first source data to obtain a target split result; generating second source data based on a target abstract of the first source data and the target split result, wherein the target abstract is determined based on a target field; constructing a target index based on the second source data and synchronizing the target index to a target log space.

2. The method of claim 1, wherein, The method further comprises: obtaining first source data; structurally processing the first source data based on a target field to generate second source data, wherein the second source data is source data corresponding to the target field in the first source data; constructing a target index based on the second source data; synchronizing the target index to a target log space.

3. The method of claim 2, wherein, Structurally processing the first source data based on a target field to generate second source data comprises: determining a target abstract of the first source data based on the target field; obtaining a data type of the first source data; splitting the first source data based on the data type to obtain a target split result; generating the second source data based on the target abstract and the target split result.

4. The method of claim 3, wherein, Splitting the first source data based on the data type to obtain a target split result comprises: splitting the first source data based on the data type to obtain an initial split result; filtering a target word in the initial split result to generate the target split result, wherein the target word is a stop word and / or a sensitive word in the initial split result.

5. The method of claim 4, wherein, Splitting the first source data based on the data type to obtain an initial split result comprises: in a case where the data type of the first source data is a first type, splitting the first source data using a first tokenizer to obtain the target split result, wherein the first type indicates that the data type is a Chinese type.

6. The method of claim 4, wherein, Splitting the first source data based on the data type to obtain an initial split result comprises: in a case where the data type of the first source data is a second type, splitting a string of the first source data based on camel case to obtain a first string, wherein the second type indicates that the format of the first source data is camel case; splitting the first string using an underscore to obtain a second string; splitting the second string using a second tokenizer to obtain the initial split result.

7. The method of claim 6, wherein, Splitting the first source data based on the data type to obtain an initial split result comprises: in a case where the data type of the first source data is a third type, splitting a string of the first source data based on an underscore to obtain a third string, wherein the third type indicates that the format of the first source data is underscore format; splitting the third string using the second tokenizer to obtain the initial split result.

8. The method of claim 7, wherein, Splitting the second string by using the second word breaker to obtain the initial splitting result, comprising: Splitting the third string by using the second word breaker based on a space to obtain a fourth string; Splitting the fourth string based on a punctuation mark based on the second word breaker to obtain the initial splitting result.

9. A storage medium, characterized by The storage medium comprises a stored program, wherein the program controls a device where the storage medium is located to execute the data processing method in any one of claims 1 to 8 when the program is running.

10. A computer terminal, characterized in that Comprise: A processor and a memory, wherein the processor is used to run a program stored in the memory, and wherein the data processing method in any one of claims 1 to 8 is executed when the program is running.

Images