A method and device for automatically restoring protobuf data based on HOOK

Abstract

This application proposes a method and device for automatically restoring protobuf data based on HOOK. The method includes obtaining the source code after APP decompilation, analyzing the classes related to protobuf custom fields in the source code, and finding that the classes related to protobuf custom fields all inherit the com.google.protobuf.GeneratedMessageV3 parent class; using the HOOK framework to hook the com.google.protobuf.GeneratedMessageV3 parent class constructor, and triggering the parent class constructor when the subclass is called; filtering out official subclasses through feature recognition to obtain the protobuf custom class; again using the HOOK framework to hook the obtained protobuf custom class constructor, serialization function, and deserialization function, and finally obtaining the data before and after the structure is serialized, and outputting and displaying the obtained data. It can accurately identify the Java class of the custom message body compiled by protobuf in the app, perform batch hooking, automatically restore protobuf data, and improve data processing efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of computer data visualization, and in particular to a method and device for automatically restoring protobuf data based on HOOK. Background Art

[0002] In recent years, nearly every industry has been impacted to varying degrees by big data. Big data has unwittingly become a core competitive advantage for businesses and a key asset in their soft power. However, processing mobile app data presents significant challenges. Mobile phones capture vast amounts of data, and sometimes, only binary files are available, often without access to plaintext code. This makes direct application of this data challenging. Therefore, the development of effective data processing technologies is crucial in the industry. Protobuf offers advantages such as high transmission efficiency, a smaller serialized size compared to JSON and XML, cross-platform and multi-language support, robust message format upgrades and compatibility, and fast serialization and deserialization. In scenarios requiring large amounts of data, choosing protobuf can significantly reduce data volume, network I / O, and, consequently, transmission time. Consequently, the use of protobuf for storing data is growing.

[0003] In the daily analysis process, we often encounter data stored or transmitted as protobuf serialization results, but the official anonymous parsing algorithm can only perform basic parsing. It is impossible to know the specific meaning of each field value, let alone its field name. To obtain its field meaning and field name, we can only do a lot of reverse work and gradually analyze each message structure contained in the program, which brings a lot of difficulty to the work and is inefficient.

[0004] In response to the above technical problems, the present invention provides a method and device for automatically restoring protobuf data based on HOOK, which can accurately identify the Java class of the custom message body compiled by protobuf in the app, and perform batch hooks to obtain relevant key fields to automatically restore protobuf data, thereby improving data processing efficiency. Summary of the Invention

[0005] In order to solve the problems in the prior art that only basic parsing can be performed through anonymous parsing algorithms, and the specific meaning of each field value cannot be known, if one wants to obtain its field meaning and field name, one can only do a lot of reverse work to gradually analyze each message structure contained in the program, which is difficult and inefficient, this application provides a method and device for automatically restoring protobuf data based on HOOK to solve the above technical defects.

[0006] According to one aspect of the present invention, a method for automatically restoring protobuf data based on HOOK is proposed, the method comprising the following steps:

[0007] S1. Obtain the decompiled source code of the APP and analyze the classes related to the protobuf custom fields in the source code. The classes related to the protobuf custom fields all inherit the parent class com.google.protobuf.GeneratedMessageV3;

[0008] S2. Use the HOOK framework to hook the com.google.protobuf.GeneratedMessageV3 parent class constructor. When the subclass is called, the parent class constructor is triggered; and

[0009] S3. Filter out official subclasses through feature recognition and obtain protobuf custom classes.

[0010] In a specific embodiment, the method also includes step S4, again hooking the constructor, serialization function, and deserialization function of the protobuf custom class obtained in step S3 through the HOOK framework, and finally obtaining the data before and after serialization of the structure.

[0011] In a specific embodiment, in step S1, by statically reading and analyzing the classes related to the protobuf custom fields in the source code, it is learned that the classes related to the protobuf custom fields all inherit from the com.google.protobuf.GeneratedMessageV3 parent class.

[0012] In a specific embodiment, in step S3, by identifying the features of the official classes, classes whose names begin with com.google are filtered out, and finally the protobuf custom class is obtained.

[0013] In a specific embodiment, in step S1, a JAVA decompilation tool is used to open the mobile phone APP program to obtain the decompiled source code of the APP.

[0014] In the second aspect, the present application also proposes a device for automatically restoring protobuf data based on HOOK, which includes:

[0015] The acquisition module is used to obtain the decompiled source code of the APP and analyze the classes related to the protobuf custom fields in the source code. The classes related to the protobuf custom fields all inherit the com.google.protobuf.GeneratedMessageV3 parent class: the hook module uses the HOOK framework to hook the com.google.protobuf.GeneratedMessageV3 parent class constructor. When the subclass is called, the parent class constructor is triggered; and

[0016] The hook module uses the HOOK framework to hook the com.google.protobuf.GeneratedMessageV3 parent class constructor. When the subclass is called, the parent class constructor is triggered; and

[0017] The filtering module is used to filter out official subclasses through feature recognition and obtain protobuf custom classes.

[0018] In a specific embodiment, the hook module is used again to obtain the constructor, serialization function, and deserialization function of the protobuf custom class through the hook filtering module of the HOOK framework, and finally obtain the data before and after serialization of the structure.

[0019] In a specific embodiment, the acquisition module learns that the classes related to the protobuf custom fields are all inherited from the com.google.protobuf.GeneratedMessageV3 parent class by statically reading and analyzing the classes related to the protobuf custom fields in the source code.

[0020] In a specific embodiment, the filtering module filters out classes whose names begin with com.google by identifying features of official classes, and finally obtains the protobuf custom class.

[0021] In a third aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any of the above methods is implemented.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] The present invention is based on the HOOK framework and uses the HOOK framework to accurately identify each custom message structure through feature matching during program operation, and through the hook mode, associates the data before and after serialization and outputs it for display, so that in the dynamic analysis process, all custom message structure data can be directly previewed, thereby eliminating the reverse process of the app, greatly improving work efficiency and obtaining effective data. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0025] Figure 1 It is a flowchart of the method for automatically restoring protobuf data based on HOOK according to the present application;

[0026] Figure 2 This is a schematic diagram of the main framework of the method for automatically restoring protobuf data based on HOOK according to the present application;

[0027] Figure 3 It is a schematic diagram of a device for automatically restoring protobuf data based on HOOK according to the present application;

[0028] Figure 4 It is a structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0029] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0030] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] Figure 1 The flowchart of the method for automatically restoring protobuf data based on HOOK of the present application is shown; Figure 2 The main framework diagram of the method for automatically restoring protobuf data based on HOOK according to the present application is shown. Figure 1 and Figure 2 , the method comprises the following steps:

[0032] S1. Obtain the decompiled source code of the APP and analyze the protobuf-related classes in the source code;

[0033] JAVA decompilation is the process of reverse engineering a target executable program to retrieve its original code. In this example, a JAVA decompiler can be used to open a mobile app, obtain the decompiled source code, and analyze the protobuf-related classes in the source code. By statically reading and analyzing the protobuf-related classes in the source code, it can be determined that the classes related to protobuf custom fields all inherit from the com.google.protobuf.GeneratedMessageV3 parent class.

[0034] S2. Use the HOOK framework to hook the parent class constructor of com.google.protobuf.GeneratedMessageV3. When the subclass is called, the parent class constructor of com.google.protobuf.GeneratedMessageV3 is triggered.

[0035] Among them, the hook framework can be used to hook any function, monitor encrypted APIs or track private application code, and use the HOOK framework to hook the constructor of the com.google.protobuf.GeneratedMessageV3 class for monitoring.

[0036] S3. By identifying the characteristics of official classes, we filter out classes whose names start with com.google, so as to accurately identify the classes of custom protobuf message bodies and finally obtain the custom protobuf data.

[0037] After the above method, many subclasses will be obtained, some of which are not needed, such as official subclasses. Therefore, the present invention can filter out classes whose class names start with com.google by identifying the characteristics of official classes, so as to accurately identify the classes of custom protobuf message bodies.

[0038] S4. Again, the constructor, serialization function, and deserialization function of the protobuf custom class obtained in step S3 are hooked through the HOOK framework, and finally the data before and after serialization of the structure are obtained, and the obtained data is output and displayed.

[0039] The present invention is based on the HOOK framework and uses the HOOK framework to accurately identify each custom message structure through feature matching during program operation, and through the hook mode, associates the data before and after serialization and outputs it for display, so that in the dynamic analysis process, all custom message structure data can be directly previewed, thereby eliminating the reverse process of the app, greatly improving work efficiency and obtaining effective data.

[0040] Further references Figure 3 As an implementation of the above method, the present application provides an embodiment of a hierarchically groupable rectangular association diagram layout device. Figure 1 Corresponding to the method embodiment shown, the system can be specifically applied to various electronic devices. The apparatus 300 includes the following modules:

[0041] The acquisition module 310 is used to obtain the decompiled source code of the APP and analyze the protobuf related classes in the source code;

[0042] The hook module 320 is used to use the HOOK framework to hook the parent class constructor, and trigger the parent class constructor when the subclass is called.

[0043] In this embodiment, in the hook module, the HOOK framework hooks the parent class constructor of com.google.protobuf.GeneratedMessageV3, and when the subclass is called, the parent class constructor of com.google.protobuf.GeneratedMessageV3 is triggered.

[0044] The filtering module 330 is used to filter out classes whose class names begin with com.google through feature recognition, and finally obtain the protobuf custom class.

[0045] The hook module 340 is hooked again to obtain the constructor, serialization function, and deserialization function of the protobuf custom class obtained by the hook filtering module of the HOOK framework, and finally obtain the data before and after serialization of the structure.

[0046] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements any of the above methods.

[0047] Reference below Figure 4 , which shows a structural diagram of a computer system 400 suitable for implementing an electronic device of an embodiment of the present application. Figure 4The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0048] like Figure 4 As shown, computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage unit 408 into a random access memory (RAM) 403. Various programs and data required for the operation of system 400 are also stored in RAM 403. CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0049] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a liquid crystal display (LCD) and a speaker; a storage section 408 including a hard disk; and a communication section 409 including a network interface card such as a LAN card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read from the removable media can be installed in the storage section 408 as needed.

[0050] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 409, and / or installed from a removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the method of the present application are performed.

[0051] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable medium, or any combination of the two. Computer-readable media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0052] The computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0053] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0054] The modules involved in the embodiments of the present application may be implemented in software or in hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor includes a receiving module, an acquisition module, a determination module, a calculation module, and a generation module. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the receiving unit may also be described as a "module that obtains preset, target user configuration information in response to determining that the verification request information includes a user name, a request time, a user signature code, and a client application code."

[0055] As another aspect, the present application also provides a computer-readable medium, which may be included in the server described in the above embodiment; or may exist independently and not be assembled into the server. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the server, the server: receives a verification request message sent by a client of a target user; in response to determining that the verification request message includes a user name, a request time, a user signature code, and a client application code, obtains preset configuration information of the target user, wherein the configuration information includes a preset user password corresponding to the user name; determines whether the verification request message is valid based on the request time, and in response to determining that it is valid, determines whether a preset storage area includes a user signature code; in response to determining that it is not included, stores the user signature code in a preset storage area, and calculates a server application code based on the user password, the request time, and the user signature code; in response to determining that the server application code matches the client application code, generates verification success information for indicating that the verification request is a legitimate request.

[0056] In addition, the computer-readable medium may be included in the terminal device described in the above embodiment, or may exist independently without being incorporated into the terminal device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the terminal device, the terminal device: obtains user information input by the target user, wherein the user information includes a user name and a user password; generates a user signature code representing the target user based on the user information; determines a request time; calculates a client application code based on the user password, the request time, and the user signature code; generates verification request information including the user name, the request time, the user signature code, and the client application code; and sends the verification request information to the server.

[0057] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for automatically restoring protobuf data based on HOOK, characterized in that: The following steps are involved: S1. Obtain the decompiled source code of the APP and analyze the classes related to the protobuf custom fields in the source code. The classes related to the protobuf custom fields all inherit the com.google.protobuf.GeneratedMessageV3 parent class. S2. Use the HOOK framework to hook the com.google.protobuf.GeneratedMessageV3 parent class constructor. When the subclass is called, the parent class constructor is triggered; and S3. Filter out official subclasses through feature recognition and obtain protobuf custom classes; S4, again through the HOOK framework hook step S3 obtained by the protobuf custom class constructor, serialization function or deserialization function, and finally obtain the data before and after structure serialization.

2. The method for automatically restoring protobuf data based on HOOK according to claim 1, characterized in that: In step S1, by statically reading and analyzing the protobuf custom field-related classes in the source code, it is learned that the protobuf custom field-related classes all inherit from the com.google.protobuf.GeneratedMessageV3 parent class.

3. The method for automatically restoring protobuf data based on HOOK according to claim 1, characterized in that: In step S3, by identifying the features of the official classes, the classes whose names start with com.google are filtered out, and finally the protobuf custom classes are obtained.

4. The method for automatically restoring protobuf data based on HOOK according to claim 1, characterized in that: In step S1, a JAVA decompilation tool is used to open the mobile phone APP program to obtain the decompiled source code of the APP.

5. A device for automatically restoring protobuf data based on HOOK, characterized in that: The device includes: an acquisition module for acquiring the decompiled source code of the APP, analyzing the classes related to the protobuf custom field in the source code, wherein the classes related to the protobuf custom field all inherit the com.google.protobuf.GeneratedMessageV3 parent class; a hook module for using the HOOK framework to hook the com.google.protobuf.GeneratedMessageV3 parent class constructor, and triggering the parent class constructor when the subclass is called; and The filtering module is used to filter out official subclasses through feature recognition and obtain protobuf custom classes; The hook module is used to obtain the constructor, serialization function, and deserialization function of the protobuf custom class through the hook filtering module of the HOOK framework again, and finally obtain the data before and after serialization of the structure.

6. The device for automatically restoring protobuf data based on HOOK according to claim 5, characterized in that: The acquisition module statically reads and analyzes the protobuf custom field-related classes in the source code, and learns that the protobuf custom field-related classes all inherit from the com.google.protobuf.GeneratedMessageV3 parent class.

7. The device for automatically restoring protobuf data based on HOOK according to claim 5, characterized in that: The filtering module filters out classes whose names start with com.google by identifying the characteristics of official classes, and finally obtains the protobuf custom class.

8. A computer-readable storage medium, wherein a computer program is stored in the medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images