A software-defined method and system for cloud network streaming data

Abstract

The present invention discloses a software definition method and system for streaming data on a cloud network end, in the field of mobile Internet technology, the method includes: positioning the function processing logic of streaming data in an application, the application being an application for which an access interface for streaming data is to be made; displaying the function processing logic so that a developer can load a Hook interface code written based on the function processing logic into the application, and in the process of the application running the Hook interface code, passing the function and parameters of the determined application that need to be hooked to the Hook to the Android virtual machine in the Android operating system, so that the Android virtual machine can obtain the streaming data generated in the operation of the application in real time based on the Hook technology, and output the streaming data as a Hook result. In this way, the present invention faces the application software under the cloud network end, completes the software definition process of streaming data in the application, and can conveniently access the streaming data through the interface.

Description

Technical Field

[0001] The present invention relates to the field of mobile Internet technology, and in particular to a software definition method and system for cloud network end streaming data. Background Art

[0002] After entering the era of mobile Internet, users cannot consume content, obtain information, and live their daily lives without mobile applications. Cloud, network, and terminal are very popular concepts in the field of Internet applications. Cloud refers to cloud computing and the infrastructure and resources used to support cloud computing. Various types of information are gathered in the cloud computing center and then processed and diverted according to needs. Network refers to the network. "Network" not only includes the original "Internet", but also extends to the field of "Internet of Things". The network's carrying capacity is constantly improving and new value is continuously being explored. Terminal refers to the terminal, which can be divided into hardware terminals and software terminals. Hardware terminals are computers, mobile phones, various sensors and interactive terminals, etc.; software terminals are various APPs, web login interfaces, software terminal programs, etc.

[0003] Mobile applications have accumulated a large amount of service and user usage data. With the rapid development of mobile Internet, various mobile applications have emerged in an endless stream. Due to the inherent closedness of mobile applications, a large number of "information islands" have been formed, making data analysis more complicated. Big data research and analysis are faced with the dilemma of no available data. It is of great value to extract, open, analyze and integrate data from various information system clients, realize the interconnection between multiple systems, break the "information islands" between systems, and further realize data sharing, information aggregation, and system integration. The rapid development of artificial intelligence and data mining based on data requires more data sharing and opening. Therefore, the opening of business data of information systems is increasingly showing its broad application prospects.

[0004] As one of the most mainstream operating systems, Android operating system has the world's largest mobile market share. After entering the 5G era, technologies such as the Internet of Things and live broadcasting have developed rapidly. With faster and more stable network speeds as support, more and more physical devices can also become intelligent by connecting to the Internet. Mobile applications such as live broadcasting, online meetings, and chat tools are also becoming popular. The popularity of these applications has also led to the generation of a large amount of streaming data. Streaming data in a big data environment has the following characteristics: real-time, volatility, disorder, burstiness, and infinity. These data exist inside the application and are difficult to obtain, forming an "information island". In related technologies, one idea to solve the problem of "data islands" in mobile applications is to expose data access interfaces through APIs (Application Programming Interfaces) based on software-defined methods, so that developers can customize data services for mobile applications on demand. By opening the API interface of mobile applications, different applications can achieve data interconnection and interoperability through calls, thereby breaking the "information island" of applications. The current interactive mode of interoperability belongs to the pull mode (i.e. polling, which means that no matter whether there is new data on the server side, the client sends a request for query at regular intervals. The result of polling may be that there is a new update on the server side, or there may be no change). It is suitable for obtaining data stored on the server side from the client, such as takeaway orders, song lists, news details, etc. However, the streaming data generated in real time by mobile applications such as live broadcasts, online meetings, and chat tools is not stored on the server side for a long time, and the generation time is uncertain, which makes the design and development of access interfaces for such streaming data a problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a software-defined method and system for cloud network streaming data to overcome the above-mentioned problems that need to be solved urgently.

[0006] In order to solve the above problems, the present invention discloses a software definition method for cloud network end streaming data, the method comprising:

[0007] Locating the function processing logic of the streaming data in the application program, wherein the application program is an application program for which an access interface of the streaming data is to be created;

[0008] The function processing logic is displayed so that the developer can write the Hook interface code based on the function processing logic and load the Hook interface code into the application;

[0009] During the process of the application program running the Hook interface code, determining the function and parameters of the Hook required by the application program;

[0010] The function and parameters that need to be Hooked are passed to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine obtains the streaming data generated during the operation of the application in real time based on the Hook technology, and outputs the streaming data as a Hook result.

[0011] Furthermore, locate the function processing logic of streaming data in the application, including:

[0012] For the application program for which a streaming data interface is to be created, a stack model record of the application program function execution sequence corresponding to the user behavior is obtained;

[0013] Abstracting the stack model in the stack model record into control flows, and obtaining a control flow tree abstracted into a multi-branch tree structure;

[0014] The control flow tree is recursively traversed by a pre-order traversal method of the tree. During the traversal process, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model to obtain a set of control flow sequences with the highest similarity; wherein the similarity analysis model is obtained by collecting streaming processing objects involved in the historical streaming processing development of Android, extracting stack model features related to the streaming processing logic in the streaming processing objects, and abstracting the stack model features into a tree structure; the stack model features include intra-class features and inter-class features, wherein the intra-class features represent the attributes of a streaming processing function, and the inter-class features represent the calling relationship between similar streaming processing functions;

[0015] The function processing logic is displayed, including:

[0016] While traversing the control flow tree, similar control flows obtained by calculation are marked, and the marked control flow sequences are used as function processing logic for streaming data in the application and displayed on the front-end interface of the application.

[0017] Further, for the application program for which the streaming data interface is to be created, a stack model record of the application program function execution sequence corresponding to the user behavior is obtained, including:

[0018] Start the application program for which the streaming data interface is to be created;

[0019] Loading the stack model acquisition tool into the Android virtual machine through the dex loading method;

[0020] The stack model acquisition tool is used to obtain stack information of the application function execution sequence corresponding to the user behavior, and the stack information is aggregated to obtain the stack model record.

[0021] Furthermore, the stack model in the stack model record is abstracted into control flows, and a control flow tree abstracted into a multi-branch tree structure is obtained, including:

[0022] Abstracting the stack model in the stack model record into control flows, wherein each control flow includes four parts: a call type, a call method, a call method parameter, and a return structure;

[0023] Each control flow is associated with its parent control flow and its child control flow respectively, and a control flow tree abstracted into a multi-branch tree structure is obtained.

[0024] Furthermore, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model, including:

[0025] Input any control flow node in the control flow tree and the intra-class feature dictionary into the similarity analysis model, wherein each element feature in the intra-class feature dictionary has a weight value, and the size of the weight value is positively correlated with the frequency of occurrence of the element feature;

[0026] In the similarity analysis model, for each input control flow node, if the control flow node contains an element feature in the intra-class feature dictionary, the intra-class feature similarity score of the control flow node increases;

[0027] The similarity analysis model outputs the intra-class feature similarity score of the control flow node.

[0028] Furthermore, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model, including:

[0029] Inputting two control flow nodes currently undergoing recursive judgment in the control flow tree into the similarity analysis model; in the tree structure of the similarity analysis model, similar control flow nodes include socket processing class functions and data packet processing class functions;

[0030] In the similarity analysis model, determining whether the two control flow nodes are the socket processing class function and the data packet processing class function respectively;

[0031] If the two control flow nodes are the socket processing class function and the data packet processing class function respectively, determining whether the two control flow nodes are in a parent-child node relationship or a brother node relationship;

[0032] If the two control flow nodes are in a parent-child node relationship or a brother node relationship, the two control flow nodes are marked as control flows with high inter-class feature similarity, and the IDs of the two control flow nodes are output.

[0033] Furthermore, the method further comprises:

[0034] Receiving a Hook request for the streaming data access interface of the application program transmitted by the developer through an Http request; the Hook request includes the function and parameters that the application program needs to hook;

[0035] Passing the Hook request to the streaming interface of the corresponding process of the application program through Android Binder cross-process communication;

[0036] The step of passing the function and parameters that need to be hooked to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine obtains the streaming data generated during the operation of the application in real time based on the Hook technology includes:

[0037] The streaming interface of the process corresponding to the application calls the behavior reflection mechanism in the Java framework layer of the Android operating system according to the Hook request, and passes the function and parameters that the application needs to Hook to the Android virtual machine, so that the Android virtual machine starts the Hook mode based on the Hook technology.

[0038] Furthermore, the method further comprises:

[0039] During the Hook mode, the generated streaming data is captured by HookMethodHandler, which is used to package the streaming data into streaming data blocks;

[0040] Each streaming data block passed by HookMethodHandler is used as a trigger event Event, and the streaming data block is converted into a Json data packet and sent to the corresponding Topic through pubEvent, so that the external data processing platform can receive the Json data packet after subscribing to the Topic.

[0041] Based on the same inventive concept, the present invention discloses a software-defined system for cloud network end streaming data, the system comprising:

[0042] A function locating module, used for locating the function processing logic of the streaming data in the application program, wherein the application program is an application program for which an access interface of the streaming data is to be created;

[0043] A function display module is used to display the function processing logic so that the developer can write the Hook interface code based on the function processing logic and load the Hook interface code into the application;

[0044] A Hook parameter acquisition module, used to determine the function and parameters of the Hook required by the application during the process of the application running the Hook interface code;

[0045] The Hook parameter transfer module is used to pass the function and parameters that need to be Hooked to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine can obtain the streaming data generated during the operation of the application in real time based on the Hook technology, and output the streaming data as a Hook result.

[0046] Furthermore, the function positioning module includes:

[0047] A stack model record acquisition submodule, used to acquire, for the application program for which a streaming data interface is to be produced, a stack model record of the application program function execution sequence corresponding to the user behavior;

[0048] A control flow tree abstraction submodule is used to abstract the stack model in the stack model record into control flows, and obtain a control flow tree abstracted into a multi-branch tree structure;

[0049] A similarity calculation submodule is used to recursively traverse the control flow tree by a pre-order traversal method of the tree. During the traversal process, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model to obtain a set of control flow sequences with the highest similarity; wherein the similarity analysis model is obtained by collecting streaming objects involved in the historical streaming development of Android, extracting the stack model features of the streaming logic in the streaming objects, and abstracting the stack model features into a tree structure; the stack model features include intra-class features and inter-class features, wherein the intra-class features represent the attributes of a streaming function, and the inter-class features represent the calling relationship between similar streaming functions;

[0050] The function display module includes: a control flow sequence marking submodule, which is used to mark similar control flows calculated while traversing the control flow tree, use the marked control flow sequence as the function processing logic of the streaming data in the application, and display it on the front-end interface of the application.

[0051] The present invention includes the following advantages:

[0052] The present invention is based on the idea of ​​software definition. First, starting from the commonly used Android streaming data processing development framework and library functions, combined with the stack models of various applications (also called applications) involving streaming data in different fields, some stack model features of Android application development streaming processing are extracted, and a similarity analysis model is established; then, based on the similarity analysis model, the function processing logic of streaming data in the application to be used to make a streaming data access interface can be located, and the function processing logic can be displayed for reference by developers. Based on the function processing logic and manual analysis method, the developer can write a Hook interface code for the streaming data of the application; after the Hook interface code is loaded into the application, in the process of the application running the Hook interface code, the function and parameters of the application that need to be hooked can be determined; and then, by passing the function and parameters that need to be hooked to an Android virtual machine in the Android operating system, the Android virtual machine can obtain the streaming data generated in the running of the application in real time based on the Hook technology, and output the streaming data as a Hook result. In this way, the present invention completes the software definition process of streaming data in the application software under the cloud network end, can conveniently access the streaming data through the interface, and can also output the streaming data in the application to the outside through the access interface for analysis and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 It is a flowchart of the steps of a software definition method for cloud network end streaming data of the present invention;

[0055] Figure 2 It is a structural schematic diagram of a streaming data interface development framework of the present invention;

[0056] Figure 3 It is a pseudo code diagram of the control flow tree ControlFlow data structure of the stack model of the application program of the present invention when it is running;

[0057] Figure 4 It is a schematic diagram of the parent-child node relationship or the brother node relationship of the present invention;

[0058] Figure 5 It is the overall pseudo code diagram of the application service positioning module of the present invention;

[0059] Figure 6.1 It is a timing diagram of the Hook processing submodule of the present invention;

[0060] Figure 6.2 is a timing diagram of the behavior interpreter of the present invention;

[0061] Figure 6.3 It is an architecture diagram of the behavior reflection submodule of the present invention;

[0062] Figure 7.1 This example requires capturing a screenshot of the interface of Trace logging into the live broadcast room of short video software A;

[0063] Figure 7.2 This is a screenshot of the showStream positioning result of the live studio stack model in this example;

[0064] Figure 7.3 This is a screenshot of the control flow of the live studio stack model No. 4498 in this instance;

[0065] Figure 7.4 This is a screenshot of the live comment streaming data returned by the Hook interface in this example;

[0066] Figure 8 It is a functional module diagram of a software-defined system for cloud network streaming data according to the present invention. DETAILED DESCRIPTION

[0067] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] At present, based on the software-defined method, we can use the resource reflection mechanism inside the information system to generate interoperability interfaces through monitoring and control outside the information system to achieve efficient interoperability of business data and functions. The current interactive mode of interoperability belongs to the pull mode (i.e. polling. Polling means that regardless of whether there is new data on the server, the client sends a request for query at regular intervals. The result of polling may be that there is a new update on the server, or there may be no change). It is suitable for obtaining data stored on the server from the client, such as takeaway orders, song lists, news details and other data. However, the aforementioned real-time streaming data is often not stored on the server for a long time, but exists on the server in the form of cache and message queues. The Android interoperability interface in the streaming data scenario has the following challenges:

[0069] (1) Volatility and burstiness of data. The data generated by live broadcast type Android applications are volatile and cannot be obtained from the mobile application client when the live broadcast room is closed. Therefore, the present invention needs to interoperate in real time during the live broadcast process. How to enable the interoperability interface to monitor streaming data in real time and output the data stream to where the developer needs it is a major challenge. In addition, for chat room type Android applications, the data generated is bursty, the time when the data is generated is uncertain, and the amount of data generated is also uncertain. For example: when big news occurs, the data flow rate generated by chat room apps will be very fast, while the data flow rate generated during weekday working hours will usually be slower. Therefore, how to enable the interoperability interface to have the ability to match the flow rate and run in a push mode (push mode refers to message push, which is the server actively sending messages to the client, and the server is the first to know the data change) faces challenges.

[0070] (2) Lightweight streaming data interfaces. The non-streaming data API interface of Android applications developed based on the pull model has a low access frequency. Each request can return data obtained from the application client in the form of pages. Therefore, the non-streaming data interface only needs to have availability and stability. For streaming data interfaces, since real-time monitoring is required, the interface is required to be as lightweight as possible and cannot affect the operation of the original Android application and cause crashes.

[0071] (3) Automation of streaming data interface development. Android application development under the A / S (Application / Server, A / S) architecture is characterized by diverse development modes, diverse development languages, and diverse development frameworks. This results in relatively complex implementation of Android application functions, deep call levels of the runtime stack model, huge function execution sequences recorded by the stack model, and complex heap model objects. This makes it very difficult to analyze the function logic of streaming processing related functions. Therefore, how to simplify the development steps of streaming data interfaces and automate interface development is also very challenging.

[0072] Therefore, how to meet the above challenges and design and develop an interoperable interface for streaming data has become a problem that needs to be solved urgently. In view of this, the present invention proposes a software definition method for streaming data on the cloud network side, referring to Figure 1 , the method may include the following steps:

[0073] Step S101, locating the function processing logic of the streaming data in the application, the application being the application for which the access interface of the streaming data is to be created;

[0074] Step S102, displaying the function processing logic so that the developer can write the Hook interface code based on the function processing logic and load the Hook interface code into the application;

[0075] Step S103, during the process of the application program running the Hook interface code, determining the function and parameters of the Hook that the application program needs;

[0076] Step S104, passing the function and parameters that need to be hooked to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine obtains the streaming data generated during the operation of the application in real time based on the Hook technology, and outputs the streaming data as a Hook result.

[0077] To implement the above method, refer to Figure 2 The present invention designs a streaming data interface development framework, which includes a function positioning module, a streaming data acquisition module and an interface interaction module. The present invention implements the entire streaming data acquisition module and a part of the interface interaction module in the Android operating system kernel, and the other part of the interface interaction module is implemented in the user state and embedded in the Yuanbang App. The function positioning module is implemented in the user state and provided to the developer in the form of an analysis tool. In the specific implementation process, the developer determines the minimum set required for the final development of the streaming data access interface according to the auxiliary results given by the function positioning module and the results of manual analysis, that is, obtains the function processing logic of the streaming data in the application; then the developer writes the Hook interface code (the Hook interface code includes the function and parameters required for Hook) through the code editor, and packages the Hook interface code into a Dex file and loads it into the application of the streaming data access interface to be made; the interface interaction module interacts with the Dex file loaded by the application, and in the interaction process, obtains the function and parameters required for the application to Hook, and then passes the function and parameters required for Hook to the Android virtual machine (also referred to as the Android app virtual machine), so that the streaming data acquisition module in the Android virtual machine starts the Hook process.

[0078] Based on the above development framework, in order to realize the location of the function processing logic of the streaming data in the application in step S101, a similarity analysis model is pre-established in the function location module of the present invention. The method for establishing the similarity analysis model is as follows:

[0079] Step A1, determining the streaming processing objects involved in the historical streaming processing development of Android;

[0080] Step A2, extracting stack model features related to the stream processing logic in the stream processing object, the stack model features include intra-class features and inter-class features, wherein the intra-class features represent the attributes of a stream processing function, and the inter-class features represent the calling relationship between similar stream processing functions;

[0081] Step A3: abstract the stack model features into a tree structure to obtain a similarity analysis model.

[0082] Considering the problem that the video streaming data generated by push and pull streaming in streaming scenarios such as live broadcast is of little value, the present invention focuses on the streaming data generated by communication in these streaming scenarios. On this basis, the present invention narrows the establishment of the Android streaming data processing logic similarity analysis model to the communication of Android Socket, and the collected streaming data is divided into two steps: feature extraction of commonly used Socket frameworks and feature extraction of mainstream streaming applications, that is, the streaming processing objects of the present invention include Android Socket frameworks and network libraries, as well as applications with streaming data. Among them, the commonly used Socket frameworks and network libraries are shown in Table 1.1:

[0083] Table 1.1 Commonly used Android Socket frameworks and network libraries

[0084]

[0085] When the Netty framework performs Socket communication, it first needs to build a Bootstrap node and set up a ChannelHandler for connection operations. After the connection is successfully established, a ChannelFuture object will be obtained, indicating an I / O operation that has not yet occurred. Then, data is received and sent through the methods provided by the Channel object. In TCP transmission, Netty will put the transmitted data packet into the ByteBuf data structure. After the Android client receives the data, it can use the StringDecoder and LineBasedFrameDecoder provided by Netyy to disassemble and process the data packet.

[0086] Java.net network library is also a commonly used network library for Android Socket. Its principle is to put the sender and receiver into one thread to run. First, you also need to use the IP and port to construct a Socket instance and establish a connection. After the connection is successfully established, you need to send and receive data streams through the socket.getOutputStream method and socket.getInputStream.

[0087] OkHttp is a very popular Android Http library, which also provides Socket functions. Its communication process is similar to the above two. First, use the OkHttpClinet instance to build a WebSocket instance and establish a connection, then rewrite the onMessage, onFailure and other methods of the WebSocketListener interface to receive data, and use the webSocket.close method to disconnect when closing the communication. Similarly, the data it transmits is also a byte stream.

[0088] AndroidAsync is a NIO (non-blocking IO) network library. When using AndroidAsync to create a Socket connection, you first need to use AsyncHttpClient.getDefaultInstance to build a Socket instance, then override the setDataCallback method of AsyncSocket to process the received Socket data, and override the onComplete method to send data.

[0089] The socket processing logic of the Mina framework is similar to the network library above. Use NioSocketConnector to establish a connection, and then configure MinaClientHandler to process data. Implement the messageReceived and messageSent methods in MinaClientHandler to process the sending and receiving of messages. After the connection is established, the connection maintenance work is handled by the underlying framework.

[0090] After determining the streaming object, the present invention can use the function call graph tool provided by the code editor IDEA to perform static scanning, obtain the function call relationship graph of the streaming object, and obtain the corresponding stack model. Among them, by collecting the Socket-related processing processes of the above-mentioned streaming frameworks and network libraries such as Netty, Mina, Java.net, OkHttp, and AndroidAsync, and analyzing the function call relationship graph of the above-mentioned streaming objects, the stack model features of the streaming logic in the streaming object can be extracted. Then, the stack model features are abstracted into a tree structure model to obtain a similarity analysis model.

[0091] In the present invention, for the intra-class features, extracting the stack model features related to the streaming processing logic in the streaming processing object may include the following steps:

[0092] Step B1, counting the frequencies of occurrence of element feature class names, method names, and parameter names in the streaming logic of the streaming object;

[0093] Step B2, classify and label the element features according to the frequency of occurrence of different element feature class names, method names, and parameter names, and determine the intra-class stack model features; the categories include socket processing class and data processing class; wherein the intra-class stack model features are recorded in the intra-class feature dictionary.

[0094] The present invention takes the Android Socket framework and network library as an example of a streaming object. The Android Socket framework and network library mentioned above are installed through IntelliJIDEA, and then the streaming processing related methods are located to count the frequency of class names, method names, and parameter names in the methods. Taking Netty as an example, the netty-all-4.1.43.Final.jar package is installed in the environment, and then according to the official documents, the Netty Socket programming related methods are checked, and finally the following key classes are determined, from which the keywords of class names, method names, and parameter names are counted. Some of the intra-class feature keywords collected by the present invention can be shown in Table 1.2:

[0095] Table 1.2 Intra-class feature extraction

[0096]

[0097] The present invention constructs an intra-class feature dictionary through frequency statistics as shown in Table 1.3. The feature dictionary is divided into two categories, the first category represents socket processing class features, and the second category represents data processing class features, which are respectively recorded as Class A and Class B. Weights are given to each feature tag based on the statistical frequency, and the higher the frequency, the greater the weight. The weight range is 1 to 3 points. Table 1.3 shows 11 intra-class feature keywords with higher occurrence frequencies.

[0098] Table 1.3 Intra-class feature dictionary

[0099] Feature name (case insensitive) Weight category socket 3 A stream 3 A client 1 A proto 2 B message 2 A buffer 1 B decode 1 B channel 1 A protocol 2 B jsonobject 1 B parse 1 B

[0100] In view of the inter-class features, the present invention can abstract the Socket processing process into the following steps: 1. Establishing a socket connection; 2. Customizing the handle function; 3. Transmitting data packets; 4. Decoding data packets; 5. Returning results. Therefore, for the judgment of inter-class similarity, the present invention can start with the order-pre and order-post relationship of the control flow. For example: control flow A represents the processing of data packets in the Socket, and control flow B represents the transmission of data packets. Then, through the existence of B in the post-order control flow of A, it can be determined that the parent control flow of AB is very likely to be a Socket type control flow, and the similarity is relatively high.

[0101] Applications with streaming data can be software such as Kuaishou, bilibili, Tonghuashun, Taobao, Douyu, etc. For this type of streaming processing object, the present invention can start from the perspective of streaming interface implementation, and obtain the stack model record file of the listed application services through the network software behavior reflection technology framework. There is a note in the acquisition process, because the streaming function of the application transmits a large amount of data and the network communication is frequent. Therefore, in order to avoid the application service stack model record file being too large, the classes in the acquisition process can be filtered, and the Android interface related operation classes can usually be filtered out, such as com.airbnb.lottie, etc. Since the streaming processing object of this type of application with streaming data generally encapsulates an internal Socket processing class in its program, the extraction process of the stack model features of the streaming processing logic can refer to the processing process of the above-mentioned Android Socket framework and network library, which is a streaming processing object, and the present invention will not go into details here.

[0102] Based on the similarity analysis model of the present invention, the implementation method of step S101 in the function positioning module can be as follows:

[0103] Sub-step S101-1, for the application program for which a streaming data interface is to be created, obtaining a stack model record of the application program function execution sequence corresponding to the user behavior;

[0104] In the present invention, the execution process of sub-step S101-1 mainly includes the following steps:

[0105] Start the application program for which the streaming data interface is to be created;

[0106] Load the stack model acquisition tool into the Android virtual machine through the dex loading method;

[0107] The stack model acquisition tool is used to obtain the stack information of the application function execution sequence corresponding to the user behavior, and the stack information is summarized to obtain the stack model record.

[0108] In the present invention, the developer starts the corresponding application that needs to make a streaming processing interface (streaming data interface), and then loads the stack model acquisition tool into the Android AppDalvik virtual machine where the application is located in advance through the dex loading method. Then the developer triggers the acquisition process in the stack model acquisition tool and performs corresponding operations. For example, for streaming data, the developer can stay in the streaming data display interface for a period of time, allowing the stack model acquisition tool to record all stack information during this period of time. After this process is completed, the developer stops the acquisition and recycles the relevant stack model records of the application function execution sequence. These record information will eventually be output to a trace file, and finally the stack model records will be summarized.

[0109] Sub-step S101-2, abstracting the stack model in the stack model record into control flows, and obtaining a control flow tree abstracted into a multi-branch tree structure;

[0110] In the present invention, sub-step S101-2 may include the following steps: abstracting the stack model in the stack model record into control flows, wherein each control flow includes four parts: call type, call method, call method parameters, and return structure; associating each control flow with its parent control flow and its child control flow respectively to obtain a control flow tree abstracted into a multi-branch tree structure.

[0111] Sub-step S101-3, recursively traverse the control flow tree by pre-order traversal of the tree. During the traversal process, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model to obtain a set of control flow sequences with the highest similarity; wherein, the similarity analysis model is obtained by collecting streaming processing objects involved in Android's historical streaming processing development, extracting the stack model features of the streaming processing logic in the streaming processing objects, and abstracting the stack model features into a tree structure; the stack model features include intra-class features and inter-class features, wherein the intra-class features represent the attributes of a streaming processing function, and the inter-class features represent the calling relationship between similar streaming processing functions.

[0112] In the present invention, the node Node of the tree represents a function call, and the edge represents its sub-function call, that is, the sub-function is called during the execution of the Node node. When the application is running, the control flow tree ControlFlow data structure of its stack model is as follows: Figure 3 As shown in Figure 1, the pre-order traversal process starts from the root control flow node of the control flow tree and recursively traverses all sub-group flows subFlows of the control flow node.

[0113] During the traversal process, the intra-class feature similarity and inter-class feature similarity are calculated for each ControlFlow.

[0114] In one embodiment of the present invention, calculating the intra-class feature similarity of each control flow in the control flow tree using a pre-established similarity analysis model may include the following sub-steps:

[0115] Input any control flow node in the control flow tree and the intra-class feature dictionary into the similarity analysis model. Each element feature in the intra-class feature dictionary has a weight value, and the size of the weight value is positively correlated with the frequency of occurrence of the element feature;

[0116] In the similarity analysis model, for each input control flow node, if the control flow node contains an element feature in the intra-class feature dictionary, the intra-class feature similarity score of the control flow node increases;

[0117] The similarity analysis model outputs the intra-class feature similarity scores of control flow nodes.

[0118] In one embodiment of the present invention, calculating the inter-class feature similarity of each control flow in the control flow tree using a pre-established similarity analysis model may include the following sub-steps:

[0119] Input two control flow nodes currently being recursively judged in the control flow tree into the similarity analysis model; in the tree structure of the similarity analysis model, similar control flow nodes include socket processing class functions and data packet processing class functions;

[0120] In the similarity analysis model, it is determined whether the two control flow nodes are respectively a socket processing class function and a data packet processing class function;

[0121] If the two control flow nodes are respectively a socket processing class function and a data packet processing class function, then determine whether the two control flow nodes are a parent-child node relationship or a brother node relationship;

[0122] If two control flow nodes are in a parent-child node relationship or a brother node relationship, the two control flow nodes are marked as control flows with high inter-class feature similarity, and the IDs of the two control flow nodes are output.

[0123] In the present invention, the schematic diagram of two control flow nodes being a parent-child node relationship or a brother node relationship is as follows: Figure 4 See Figure 4 Specifically, if node A is the ancestor of node B, recursively judge in the pre-order traversal control flow tree structure. If it exists, add weight to the control flow class similarity variable. For the subtree structure where nodes A and B are siblings, the order is determined by the time of the A and B class methods in the call stack. If the call time of A is earlier than that of B, the weight is also added to the class similarity variable. Finally, a list of marked high-similarity ControlFlow node IDs is returned for subsequent screening.

[0124] Finally, based on the calculation method of the intra-class feature similarity and inter-class feature similarity, the present invention obtains the overall pseudo code of the application service positioning module as follows: Figure 5 By rewriting the traversal method in the Visitor class, the calculation of the similarity analysis module is combined with the traversal process. If the obtained similarity value is greater than a (a is adjusted based on experience), it is determined to be an Android streaming related control flow, that is, a set of control flow sequences with the highest similarity is obtained in the present invention.

[0125] In step S102, the present invention displays the function processing logic, which may include the following steps:

[0126] While traversing the control flow tree, similar control flows obtained by calculation are marked, and the marked control flow sequences are used as the function processing logic of the streaming data in the application and displayed on the front-end interface of the application.

[0127] In the present invention, the application service location module is implemented in the project TraceParser. In order to display it in the interface, the front-end and back-end parts are implemented separately. The front-end interface uses the streaming function of viewing the location to process the control flow, which is implemented in Javascript and HTML. The back-end is based on the Java Servlet architecture code and is responsible for reading the stack model record file and returning the results after similarity analysis.

[0128] The front-end code includes nativecfviewer.js and nativecfviewer.html. The html file implements the style of the button showStream in the interface window. The Js file implements the coloring and display of related control flows. For example: each control flow tree corresponds to a blue rectangular bar. The control flow calculated and marked by the similarity analysis model on the back end will be displayed. After clicking the showStream button, the showStream method of nativecfviewer.js will be called to implement the .ajax asynchronous call. NativeCFReaderServlet.java implemented in the Servlet framework receives the front-end request through @APIshowStream. After the back-end calculation returns the control flow ID with a high similarity, nativecfviewer.js receives the Response parameter and colors the three control flow blocks with high similarity to assist developers in quick screening and analysis.

[0129] The backend code is implemented in the TraceReader.java file. The TraceReader class reads a heap model trace file, then adds a showStream method to traverse the stack model, and embeds the similarity analysis module code into the stack model traversal process. ShowStream will finally return an Integer array, recording the nodes whose similarity value calculation value is greater than 3.

[0130] The present invention can facilitate developers to analyze results and flexibly customize the access interface of Android streaming data by locating the function processing logic of streaming data. Next, the developer can write the Hook interface code based on the function processing logic, and package the Hook interface code into a Dex file and load it into the application. Then, in the process of the application running the Dex file, the developer can obtain the function and parameters that the application needs to hook Hook, so that the developer can start the Hook request. Specifically, the present invention can implement the Hook request through the following method:

[0131] Receive the Hook request for the streaming data access interface of the application transmitted by the developer through the HTTP request; the Hook request includes the function and parameters that the application needs to hook;

[0132] The Hook request is passed to the streaming interface of the corresponding process of the application through Android Binder cross-process communication.

[0133] Step S104 can be specifically implemented by the following steps:

[0134] The streaming interface of the corresponding process of the application calls the behavior reflection mechanism in the Java framework layer of the Android operating system according to the Hook request, and passes the function and parameters that the application needs to Hook to the Android virtual machine, so that the Android virtual machine starts the Hook mode based on the Hook technology.

[0135] Since the streaming data acquisition process disclosed in step S103 and step S104 of the present invention is implemented based on the streaming data acquisition module and the interface interaction module in the streaming data interface development framework, in order to clearly explain the implementation logic of step S103 and step S104, the present invention will now explain the implementation of the Android operating system and the streaming data acquisition module and the interface interaction module.

[0136] In the development framework of the present invention, the Android operating system includes underlying hardware, a Linux kernel, an Android Runtime layer, a Java framework layer and an Application layer, wherein: an interface interaction module is implemented in the Application layer and the Linux kernel, the Application layer includes a remote interface management application and an application program for accessing an interface of streaming data to be produced, the remote interface management application is used to receive network requests for interface interoperation and forward them to the underlying layer, status monitoring, and interact with the application program for data streams; a streaming data acquisition module is implemented in the Android Runtime layer, the Android Runtime layer is located in the Android virtual machine, and is used to complete the functions of parsing interface Hook commands, monitoring, and interpreting and executing; the Java framework layer is connected to the Android Runtime layer and the Application layer respectively, the Java framework layer includes a behavior reflection submodule, the behavior reflection submodule is used to utilize the reflection mechanism of Java to transmit the functions and parameters that need to be Hooked that are passed by the interface interaction module to the ART / Dalvik virtual machine; wherein the Android virtual machine is an ART / Dalvik virtual machine, and the ART / Dalvik virtual machine is used to compile the dex bytecode in the Dex file into CPU operation instructions and submit them to the underlying hardware for execution.

[0137] In the present invention, each module in the Android Runtime layer jointly completes the core function of Hook. Among them, the Android Runtime layer includes an interface submodule, an optimizer-reoptimizer, a Hook processing submodule and a behavior interpreter, which are implemented in the ART / Dalvik virtual machine and implement Hook by monitoring the process of interpretation and execution. Specifically:

[0138] Interface submodule: The interface submodule completes the encapsulation of the Hook function and provides a calling channel for the Java framework layer. Finally, the upper-level module will call the Hook module in the virtual machine through the Java reflection mechanism.

[0139] Optimizer-deoptimizer: After Android 4.4, Android Runtime will load native instructions that can be executed by the CPU through the ART virtual machine. Therefore, in order to implement the Hook operation, the present invention needs to deoptimize the CPU native instructions into bytecodes. The optimizer-deoptimizer implements the class screening mechanism in the two-level screening mechanism, which can deoptimize the specified class into dex bytecodes at runtime, and then execute it through the interpreter executor provided by the ART virtual machine, so as to realize the monitoring and Hook of the specified function. For the classes that have not been deoptimized, the CPU native instructions are still installed to run, which improves the execution efficiency during the Hook process. When the interface submodule transmits the start Hook command, the optimizer-deoptimizer is triggered, and the function that the application needs to hook is deoptimized into bytecodes. When the interface submodule sends the end Hook command, the optimizer-deoptimizer re-optimizes the class that has been deoptimized into bytecodes into CPU native instructions, reduces system overhead, and improves the monitoring speed.

[0140] Hook processing submodule: The Hook processing submodule is implemented in the Android virtual machine, and additional code is inserted by plugging. The module timing diagram is as follows Figure 6.1 As shown. After receiving the startHooking command from the interface submodule, the Hook processing submodule converts the incoming Hook method into the corresponding artMethod in the ART virtual machine, and inserts the function list that needs to be Hooked and the processing function HookHandler of the Hook function in the function list into the DexFileExtra object. In order to hook multiple functions at the same time, the startHook method is called at the same time to enter behavior_hook; among them, behavior_hook uses the optimizer-deoptimizer to deoptimize the functions that need to be Hooked in the function list, and then constructs a BehaviorHook instance, which contains the function list inserted in DexFileExtra. After that, behavior_hook registers a listener in the behavior interpreter, which can listen to the Hook function in DexFileExtra. In addition, when the Hook processing submodule receives the StopHooking instruction from the interface submodule, it sets the mark bit of the BehaviorHook instance to empty and deletes the previously added Hook function in the DexFileExtra object. At the same time, it starts the optimizer-deoptimizer to convert the Dex bytecode into CPU operation instructions and restore it to the native instructions when the application is executed. In this way, the ART / Dalvik virtual machine (Android virtual machine) can run the application in the previous way.

[0141] Behavior interpreter: The behavior interpreter is mainly responsible for interpreting and executing the dex bytecode, and monitoring the Hook function during the interpretation and execution process. The module timing diagram is as follows Figure 6.2 As shown. The present invention inserts a stub in the behavior interpreter to listen to the Enter and Exit events of the function. When the Enter or Exit event is triggered, the behavior interpreter will jump to the corresponding HookHandle function for processing and then return. In this way, the present invention can customize the processing logic when the Hook function is triggered, and realize the processing of Hook to the specified function. Specifically: when the Enter of the Hook function is triggered, it means that the next function of the behavior interpreter execution pointer is the Hook function. The present invention obtains the HookMethodHandler method loaded in the Java framework layer through the reflection mechanism of C++ through JNI call, and the method contains the processing function before the function enters that is customized by the developer when developing the Hook interface. When the Exit of the Hook function is triggered, it means that the behavior interpreter has executed the Hook function and the Hook function returns the result. At this time, the present invention also calls the processing method after the function exits in HookMethodHandler through JNI to collect and process the return value.

[0142] The present invention implements the behavior reflection submodule in the Java framework layer, referring to Figure 6.3 , the behavior reflection submodule includes a behavior reflection class loader and a behavior reflection interface. The main function of the behavior reflection submodule is to load the user-defined interface code, and then implement the function of passing the Hook function to the ART / Dalvik virtual machine. During the implementation process: after the application process is started, the behavior reflection class loader will check whether the application has loaded the Dex file of the corresponding streaming data access interface; if so, it will load the interface tool class YancloudPub.dex and the dex bytecode in the Dex file. After successful loading, the application registers the interface in the remote interface management application through the Android Binder mechanism, thereby providing a channel for interface calling. The logic of calling the Hook interface of the behavior reflection submodule is implemented in ApiStarter.java. Therefore, the behavior reflection interface is used to call ApiStarter in YancloudPub.dex after receiving the Hook call command, so as to reflectively call the Hook method in the ART / Dalvik virtual machine to start Hook.

[0143] In the present invention, the remote interface management application in the Application layer is used to receive the streaming data access interface Hook request transmitted by the developer through the Http request, and transmit the streaming data access interface Hook request to the streaming interface of the corresponding process of the application through the Android Binder cross-process communication, so that after the streaming interface receives the request, it calls the behavior reflection interface submodule, thereby triggering the Android virtual machine to start the Hook mode. Among them, the remote interface management application can be the present invention Figure 2 The Yuanbang App shown is an Android management application for interface management and data transmission in the runtime reflection framework of the network software application behavior. Using this management application, the present invention realizes a pipeline for streaming data transmission within the application program, and can output the function parameters and function execution results of the streaming data obtained in the Hook mode as Hook results through the Android Binder mechanism.

[0144] Software definition refers to the use of various traditional resources in a programmable manner. The present invention, facing the application software under the cloud network, realizes the software definition of the streaming data in the application, and can output the streaming data in the application through the corresponding interface. In order to conveniently analyze the streaming data, the present invention also proposes the following method:

[0145] During the Hook mode, the generated streaming data is captured by HookMethodHandler, which is used to package the streaming data into streaming data blocks;

[0146] Each streaming data block passed by HookMethodHandler is used as a trigger event Event, and the streaming data block is converted into a Json data packet and sent to the corresponding Topic through pubEvent, so that the external data processing platform can receive the Json data packet after subscribing to the Topic. This method can be implemented by establishing a long connection between the remote interface management application and the external data processing platform. Through the communication relationship between the remote interface management application and the external data processing platform, the streaming data in the application can be output to the external data processing platform for analysis, extracting the important value of the streaming data and breaking the "information island".

[0147] In order to verify the effectiveness of the software-defined method for cloud network streaming data of the present invention, which can output the function parameters and function execution results of streaming data as Hook results, thereby completing the development of the streaming data access interface in the application, the present invention uses an example to illustrate.

[0148] It is determined that short video software A is taken as a research example of the present invention.

[0149] During the live broadcast of the short video software A, the audience can make real-time comments in the comment area or interact by giving gifts and other rewards. The audience's interactive content will be displayed in real time in the live broadcast interface, which has the characteristics of a streaming scene. In addition, the data generated during the live broadcast cannot be obtained through the client after the live broadcast room is closed, and can only be obtained through the company's internal server. Therefore, the comment data stream of the live broadcast room of short video software A meets the streaming application scenario of the present invention.

[0150] Interface requirement: Real-time acquisition of streaming data of audience comments generated during the live broadcast of short video software A.

[0151] The collection of application service information stack related to the live broadcast room of short video software A is divided into the following steps:

[0152] (1) Environmental preparation. First, install the short video software A application on an Android phone that has been flashed with a customized system. Then run DexPacker.java in IDEA to generate the dex file corresponding to the short video software AApk (hereinafter referred to as dy.dex). After that, use the Android adb tool to copy the dy.dex file to the specified directory sdcard / apiminier. Next, install and open the remote interface management application - Yuanbang App on the mobile phone, and then configure the mobile phone and computer in the same local area network. Start the short video software AApp, and the behavior reflection class loader will load the dy.dex file into the Android virtual machine.

[0153] Open the YancloudPub project in IDEA, configure the tracestarter.conf configuration file for capturing the application service stack model, and set TaskType to 0. Run TraceStarter.java. If the result returns the package name com.ss.android.ugc.aweme of the corresponding short video software AApp, it means that the environment for capturing Trace has been successfully set up.

[0154] (2) Capture Trace. Based on step (1), this example re-modifies TaskType in tracestarter.conf to 1, indicating the capture Trace task. According to the interface requirements, log in to the account of short video software A in advance, and then enter a live broadcast room interface. Figure 7.1The following is the interface of the live broadcast room of the short video software A, which needs to be captured by Trace in this example. Keep the live broadcast room page on the phone, and then run TraceStarter.java to start collecting the stack model. During the collection process, no operation is required on the phone, just observe whether the comments in the live broadcast room are updated. After the comments in the live broadcast room are updated, write down the content of the comments, and trigger TraceStarter.java again to stop the collection. The Android Studio console will output the progress of the stack record download. It should be noted that the capture process of the trace cannot be too long, because the amount of data generated during the live broadcast is large, and the corresponding collected stack model record file will be very large. If the capture process stays in the live broadcast room for too long, it may cause excessive memory usage and then the application service crashes.

[0155] (3) Collect the application service stack model. After waiting for the trace file and hprof file to be downloaded, the stack model record is collected successfully. The file named com.ss.android.ugc.aweme_1609120835_11575_0.trace represents the collected stack model record, and the file named com.ss.android.ugc.aweme_1609120835_11575_1.hprof represents the corresponding heap model record. At this point, the stack model collection of the short video software A live broadcast room is completed.

[0156] After collecting the live broadcast room stack model of short video software A, run the TraceParser project to view and analyze the stack model. This example can be compared through manual positioning and application service streaming processing function positioning, so as to verify the effect of step S101 of the present invention on positioning the function processing logic of streaming data in the application.

[0157] After setting the trace file directory, open the local web page where the TraceParser project is started. Load the trace file of short video software A, and then click showStream. After similarity analysis and application service positioning, the positioning results are as follows: Figure 7.2 As shown in the figure. The relevant control flows screened manually are as follows Figure 7.3 As shown. Finally, it is determined that the key control flow for developing the streaming interface is the control flow with ID 4498 and TID 246. This control flow belongs to the 5 control flows located by the application service flow processing logic. It can be seen that the key control flow of the streaming interface can be accurately located by adopting the solution of the present invention.

[0158] Next, according to the control flow No. 4498 given by the function positioning module, it is found through manual analysis that the messages generated during the live broadcast are encapsulated by ProtoApiResult, such as Figure 7.3As shown. The short video software AApp parses the message through the decodeWebSocketPayload method. The parameter passed in is the message PayloadItem object. The function will parse the byte message and then encapsulate it into ProtoApiResult. Therefore, you only need to hook the internal processing of the function in the red box to output the comment data when parsing the message. Finally, this example determines that the Hook method is com.bytedance.android.daagd().

[0159] Next, we start writing the Hook interface in the interface interaction module. Run ProjectGenerator.java to generate the interface project template of short video software A. An ExamPorter.java file will be automatically generated in the template. This file is used to write the interface. This example customizes a HookHandle class to implement the before and after functions in HookCallback. Here, this example can use the toString method in after to print out the comment data generated by the Hook function. In ExamPorter.java, implement a runHook method and an endHook method. The runHook method obtains the class name and method name of com.bytedance.android.daagd() through reflection, and then calls the startHook method of APIPorter to perform the Hook operation. After the interface code is written, run DexPacker.java to package and generate a Dex file, and then import it into the mobile phone.

[0160] Import the Hook interface bytecode (code) of the short video software A, and then restart the short video software AApp. After entering a live broadcast room, enter the interface call command in the browser to run the Hook interface bytecode. You can find that the live broadcast room data generated in real time by the console is as follows Figure 7.4 As shown in the figure, the left side is the live broadcast in progress, and the right side is the live broadcast room comment data stream generated in real time by the streaming data access interface of the application. In this example, since the data format encapsulation is omitted, the data stream generated on the right is the most initial data. Developers can customize the Hook processing function HookHandle according to their needs to achieve the unification of data format.

[0161] The above results show that the software definition method for cloud network streaming data designed by the present invention can effectively realize the development of Android application streaming data interface.

[0162] Based on the same inventive concept, Figure 8The present invention also discloses a software-defined system for cloud network end streaming data, which may include:

[0163] A function locating module 801 is used to locate the function processing logic of the streaming data in the application program, where the application program is an application program for which the access interface of the streaming data is to be created;

[0164] The function display module 802 is used to display the function processing logic so that the developer can write the Hook interface code based on the function processing logic and load the Hook interface code into the application program;

[0165] The Hook parameter acquisition module 803 is used to determine the function and parameters of the Hook that the application needs when the application runs the Hook interface code;

[0166] The Hook parameter transfer module 804 is used to transfer the function and parameters that need to be Hooked to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine can obtain the streaming data generated during the operation of the application in real time based on the Hook technology, and output the streaming data as a Hook result.

[0167] Furthermore, the function location module 701 may include:

[0168] A stack model record acquisition submodule is used to acquire a stack model record of an application function execution sequence corresponding to a user behavior for an application to which a streaming data interface is to be created;

[0169] The control flow tree abstraction submodule is used to abstract the stack model in the stack model record into control flows, and obtain a control flow tree abstracted into a multi-branch tree structure;

[0170] The similarity calculation submodule is used to recursively traverse the control flow tree by pre-order traversal of the tree. During the traversal process, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model to obtain a set of control flow sequences with the highest similarity. The similarity analysis model is obtained by collecting the streaming objects involved in the historical streaming development of Android, extracting the stack model features of the streaming logic in the streaming objects, and abstracting the stack model features into a tree structure. The stack model features include intra-class features and inter-class features, wherein the intra-class features represent the attributes of a streaming function, and the inter-class features represent the calling relationship between similar streaming functions.

[0171] Function display module 702, including:

[0172] The control flow sequence marking submodule is used to mark similar control flows calculated while traversing the control flow tree, use the marked control flow sequence as the function processing logic of streaming data in the application, and display it on the front-end interface of the application.

[0173] Furthermore, the stack model record acquisition submodule includes:

[0174] An application program starting unit, used for starting the application program for which a streaming data interface is to be created;

[0175] A collection tool loading unit is used to load the stack model collection tool into the Android virtual machine through the dex loading method;

[0176] The stack information acquisition unit is used to use the stack model acquisition tool to acquire the stack information of the application function execution sequence corresponding to the user behavior, and to summarize the stack information to obtain the stack model record.

[0177] Furthermore, the control flow tree abstract submodule includes:

[0178] The control flow abstraction unit is used to abstract the stack model in the stack model record into control flows, where each control flow contains four parts: call type, call method, call method parameters, and return structure;

[0179] The control flow association unit is used to associate each control flow with its parent control flow and its child control flow respectively, so as to obtain a control flow tree abstracted into a multi-branch tree structure.

[0180] Furthermore, the similarity calculation submodule includes:

[0181] An intra-class feature calculation unit is used to input any control flow node in the control flow tree and the intra-class feature dictionary into a similarity analysis model, wherein each element feature in the intra-class feature dictionary has a weight value, and the size of the weight value is positively correlated with the frequency of occurrence of the element feature;

[0182] In the similarity analysis model, for each input control flow node, if the control flow node contains an element feature in the intra-class feature dictionary, the intra-class feature similarity score of the control flow node increases; the similarity analysis model outputs the intra-class feature similarity score of the control flow node.

[0183] Furthermore, the similarity calculation submodule includes:

[0184] The inter-class feature calculation unit is used to input two control flow nodes currently undergoing recursive judgment in the control flow tree into a similarity analysis model; in the tree structure of the similarity analysis model, similar control flow nodes include socket processing class functions and data packet processing class functions;

[0185] In the similarity analysis model, determine whether the two control flow nodes are respectively a socket processing class function and a data packet processing class function; if the two control flow nodes are respectively a socket processing class function and a data packet processing class function, determine whether the two control flow nodes are in a parent-child node relationship or a brother node relationship; if the two control flow nodes are in a parent-child node relationship or a brother node relationship, mark the two control flow nodes as control flows with high inter-class feature similarity, and output the IDs of the two control flow nodes.

[0186] Furthermore, the system also includes:

[0187] The Hook request receiving module is used to receive the Hook request for the streaming data access interface of the application program transmitted by the developer through the Http request; the Hook request includes the function and parameters that the application program needs to hook;

[0188] The Hook request delivery module is used to pass the Hook request to the streaming interface of the corresponding process of the application through Android Binder cross-process communication;

[0189] The Hook parameter transfer module 704 includes:

[0190] The transfer execution submodule is configured in the streaming interface of the corresponding process of the application. It is used to call the behavior reflection mechanism in the Java framework layer of the Android operating system according to the Hook request, and pass the functions and parameters that the application needs to Hook to the Android virtual machine, so that the Android virtual machine can start the Hook mode based on the Hook technology.

[0191] Furthermore, the system also includes:

[0192] The streaming data packaging module is used to capture the streaming data generated during the Hook mode by HookMethodHandler, and HookMethodHandler is used to package the streaming data into streaming data blocks;

[0193] The streaming data sending module is used to treat each streaming data block passed by HookMethodHandler as a trigger event Event, and convert the streaming data block into a Json data packet and send it to the corresponding Topic through pubEvent, so that the external data processing platform can receive the Json data packet after subscribing to the Topic.

[0194] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0195] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0196] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0197] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0199] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0200] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0201] The above is a detailed introduction to a software-defined method and system for cloud network streaming data provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A software definition method for cloud network streaming data, characterized in that: The method comprises: Locating the function processing logic of the streaming data in the application program, wherein the application program is an application program for which an access interface of the streaming data is to be created; The function processing logic is displayed so that the developer can write the Hook interface code based on the function processing logic, and load the Hook interface code into the terminal where the application is located; During the process of the application program running the Hook interface code, determining the function and parameters of the Hook required by the application program; Passing the function and parameters that need to be hooked to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine obtains the streaming data generated during the operation of the application in real time based on the Hook technology, and outputs the streaming data as a Hook result; The positioning of the function processing logic of the streaming data in the application includes: For the application program for which the access interface of the streaming data is to be created, a stack model record of the application program function execution sequence corresponding to the user behavior is obtained; Abstracting the stack model in the stack model record into control flows, and obtaining a control flow tree abstracted into a multi-branch tree structure; The control flow tree is recursively traversed by the pre-order traversal method of the multi-branch tree. During the traversal process, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model to obtain a set of control flow sequences with the highest similarity; wherein the similarity analysis model is obtained by collecting streaming processing objects involved in the historical streaming processing development of Android, extracting stack model features related to the streaming processing logic in the streaming processing objects, and abstracting the stack model features into a tree structure; the stack model features include intra-class features and inter-class features, wherein the intra-class features represent the attributes of a streaming processing function, and the inter-class features represent the calling relationship between similar streaming processing functions; The function processing logic is displayed, including: While traversing the control flow tree, similar control flow sequences obtained by calculation are marked, and the marked control flow sequences are used as function processing logic of streaming data in the application and displayed on the front-end interface of the application.

2. The method according to claim 1, characterized in that: For an application program for which a streaming data access interface is to be created, a stack model record of the application program function execution sequence corresponding to a user behavior is obtained, including: Start the application program for accessing the streaming data to be produced; Loading the stack model acquisition tool into the Android virtual machine through the dex loading method; The stack model acquisition tool is used to obtain stack information of the application function execution sequence corresponding to the user behavior, and the stack information is aggregated to obtain the stack model record.

3. The method according to claim 1, characterized in that The stack model in the stack model record is abstracted into control flows, and a control flow tree abstracted into a multi-branch tree structure is obtained, including: Abstracting the stack model in the stack model record into control flows, wherein each control flow includes four parts: a call type, a call method, a call method parameter, and a return structure; Each control flow is associated with its parent control flow and its child control flow respectively, and a control flow tree abstracted into a multi-branch tree structure is obtained.

4. The method according to claim 1, characterized in that The similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model, including: Input any control flow node in the control flow tree and the intra-class feature dictionary into the similarity analysis model, wherein each element feature in the intra-class feature dictionary has a weight value, and the size of the weight value is positively correlated with the frequency of occurrence of the element feature; In the similarity analysis model, for each input control flow node, if the control flow node contains an element feature in the intra-class feature dictionary, the intra-class feature similarity score of the control flow node increases; The similarity analysis model outputs the intra-class feature similarity score of the control flow node.

5. The method according to claim 1, characterized in that The similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model, including: Inputting two control flow nodes currently undergoing recursive judgment in the control flow tree into the similarity analysis model; in the tree structure of the similarity analysis model, similar control flow nodes include socket processing class functions and data packet processing class functions; In the similarity analysis model, determining whether the two control flow nodes are the socket processing class function and the data packet processing class function respectively; If the two control flow nodes are the socket processing class function and the data packet processing class function respectively, determining whether the two control flow nodes are in a parent-child node relationship or a brother node relationship; If the two control flow nodes are in a parent-child node relationship or a brother node relationship, the two control flow nodes are marked as control flows with high inter-class feature similarity, and the IDs of the two control flow nodes are output.

6. The method according to claim 1, characterized in that The method further comprises: Receiving a Hook request for the streaming data access interface of the application program transmitted by the developer through an Http request; the Hook request includes the function and parameters that the application program needs to hook; Passing the Hook request to the streaming interface of the corresponding process of the application program through Android Binder cross-process communication; The step of passing the function and parameters that need to be hooked to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine obtains the streaming data generated during the operation of the application in real time based on the Hook technology includes: The streaming interface of the process corresponding to the application calls the behavior reflection mechanism in the Java framework layer of the Android operating system according to the Hook request, and passes the function and parameters that the application needs to Hook to the Android virtual machine, so that the Android virtual machine starts the Hook mode based on the Hook technology.

7. The method according to claim 6, characterized in that The method further comprises: During the Hook mode, the generated streaming data is captured by HookMethodHandler, which is used to package the streaming data into streaming data blocks; Each streaming data block passed by HookMethodHandler is used as a trigger event Event, and the streaming data block is converted into a Json data packet and sent to the corresponding Topic through pubEvent, so that the external data processing platform can receive the Json data packet after subscribing to the Topic.

8. A software-defined system for cloud network streaming data, characterized in that: The system comprises: A function locating module, used for locating the function processing logic of the streaming data in the application program, wherein the application program is an application program for which an access interface of the streaming data is to be created; A function display module is used to display the function processing logic so that the developer can write the Hook interface code based on the function processing logic and load the Hook interface code into the application; A Hook parameter acquisition module, used to determine the function and parameters of the Hook required by the application during the process of the application running the Hook interface code; A Hook parameter transfer module is used to transfer the function and parameters that need to be Hooked to the Android virtual machine on the terminal where the application is located, so that the Android virtual machine obtains the streaming data generated during the operation of the application in real time based on the Hook technology, and outputs the streaming data as a Hook result; Wherein, the function positioning module is used for: For the application program for which the access interface of the streaming data is to be created, a stack model record of the application program function execution sequence corresponding to the user behavior is obtained; Abstracting the stack model in the stack model record into control flows, and obtaining a control flow tree abstracted into a multi-branch tree structure; The control flow tree is recursively traversed by the pre-order traversal method of the multi-branch tree. During the traversal process, the similarity of each control flow in the control flow tree is calculated using a pre-established similarity analysis model to obtain a set of control flow sequences with the highest similarity; wherein the similarity analysis model is obtained by collecting streaming processing objects involved in the historical streaming processing development of Android, extracting stack model features related to the streaming processing logic in the streaming processing objects, and abstracting the stack model features into a tree structure; the stack model features include intra-class features and inter-class features, wherein the intra-class features represent the attributes of a streaming processing function, and the inter-class features represent the calling relationship between similar streaming processing functions; The function display module is used to: While traversing the control flow tree, similar control flow sequences obtained by calculation are marked, and the marked control flow sequences are used as function processing logic of streaming data in the application and displayed on the front-end interface of the application.

Images