Data Acquisition Method, System, Device and Storage Medium

Abstract

Embodiments of the present application disclose a data acquisition method, system, device, and storage medium, which are used to solve the problem of low data acquisition efficiency in the prior art. The method includes: obtaining attribute information and acquisition strategy information of at least one acquisition object; the attribute information includes identification information of the acquisition object; creating a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition strategy information; the data acquisition model includes identification information and acquisition strategy information corresponding to each acquisition object respectively; deploying the data acquisition model to at least one cloud processor corresponding to the acquisition object; and using the data acquisition model to perform data acquisition on the acquisition object in the cloud processor. This technical solution can achieve the effect of automatically performing data acquisition on the acquisition object in the cloud processor using the data acquisition model, improving the flexibility of data acquisition and the data acquisition efficiency.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data collection method, system, device, and storage medium. Background Art

[0002] Currently, cloud computing technology has developed rapidly, especially the rapid development of public clouds and public cloud-based services, which has led to the continuous expansion of the scale of machines and services in cloud computing data centers. How to quickly and efficiently collect and process the monitoring data of machines and services across the network is a very important issue in the field of cloud monitoring. Deploying collection agents across the network to collect monitoring data through the agent method, and combining a task distribution server and a collection device to achieve flexible collection of network-wide monitoring data is the mainstream method in the prior art.

[0003] In the prior art, a monitoring data collection system includes a collection device, a task distribution server, and a collection device. In this monitoring data collection system, the collection device is used to collect data, the collection device is used to issue collection instructions, the task distribution server is used to receive the collection instructions from the collection device and issue them to the collection device, and then the collection device collects data at a fixed cycle according to the collection instructions. Using this technology, combined with the frequent changes in the monitoring data collection objects and monitoring strategies in large-scale business scenarios, it is necessary to frequently issue collection instructions to a large number of collection devices through the collection device, and the configuration and issuance efficiency of large-scale collection instructions will be relatively low. At the same time, the performance of the task distribution server also affects the issuance efficiency of the collection instructions, further resulting in a relatively low collection efficiency of the monitoring data. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a data collection method, system, device, and storage medium to solve the problem of low data collection efficiency in the prior art.

[0005] To solve the above technical problems, the embodiments of this application are implemented as follows:

[0006] On the one hand, the embodiments of this application provide a data collection method, including:

[0007] Obtain the attribute information and collection strategy information of at least one collection object; the attribute information includes the identification information of the collection object;

[0008] According to the attribute information and the collection strategy information, create a data collection model corresponding to the collection object; the data collection model includes the identification information and the collection strategy information respectively corresponding to each collection object;

[0009] Deploy the data collection model to at least one cloud processor corresponding to the collection object;

[0010] Using the data collection model, collect data for the collection object in the cloud processor.

[0011] On the other hand, an embodiment of the present application provides a data collection system, including a collection model management device and at least one data collection device; wherein:

[0012] The collection model management device is configured to obtain attribute information and collection strategy information of at least one collection object; create a data collection model corresponding to the collection object according to the attribute information and the collection strategy information, and send the data collection model to the data collection device; the attribute information includes identification information of the collection object; the data collection model includes identification information corresponding to each of the collection objects and the collection strategy information;

[0013] The data collection device is configured to deploy the data collection model to at least one cloud processor corresponding to the collection object; and use the data collection model to collect data for the collection object in the cloud processor.

[0014] On yet another aspect, an embodiment of the present application provides a data collection device, including a processor and a memory electrically connected to the processor, the memory storing a computer program, and the processor is configured to call and execute the computer program from the memory to implement the above data collection method.

[0015] On yet another aspect, an embodiment of the present application provides a storage medium for storing a computer program, and the computer program can be executed by a processor to implement the above data collection method.

[0016] By adopting the technical solution of the embodiment of the present application, by obtaining the attribute information and collection strategy information of at least one collection object, creating a data collection model corresponding to the collection object according to the attribute information and the collection strategy information, and deploying the data collection model to at least one cloud processor corresponding to the collection object, so as to use the data collection model to collect data for the collection object in the cloud processor. Compared with the prior art in which data collection depends on the received collection instructions, this technical solution can achieve the effect of automatically collecting data for the collection object in the cloud processor by creating a data collection model and deploying the data collection model to the cloud processor corresponding to the collection object, improving the flexibility of data collection, and since the data collection model is deployed locally in the cloud processor, there is no need to wait for the receipt of the data collection model when collecting data, thus improving the data collection efficiency. Description of the Drawings

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0018] Figure 1 is a schematic block diagram of a data acquisition system according to an embodiment of the present application;

[0019] Figure 2 is a schematic flowchart of a data acquisition method according to an embodiment of the present application;

[0020] Figure 3 is a schematic flowchart of a data acquisition method according to another embodiment of the present application;

[0021] Figure 4 is a schematic flowchart of a method for creating and distributing a data acquisition model according to an embodiment of the present application;

[0022] Figure 5 is a schematic block diagram of a data acquisition device according to an embodiment of the present application;

[0023] Figure 6 is a schematic block diagram of a collection model management device according to an embodiment of the present application;

[0024] Figure 7 is a schematic block diagram of a data acquisition system according to another embodiment of the present application;

[0025] Figure 8 is a schematic structural diagram of a data acquisition device according to an embodiment of the present application. Detailed implementation manners

[0026] The embodiments of the present application provide a data acquisition method, system, device and storage medium to solve the problem of low data acquisition efficiency in the prior art.

[0027] To enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0028] The data acquisition method provided by one or more embodiments of the present application can be applied to a data acquisition system, that is, the data acquisition system is used to execute the data acquisition method. Figure 1 It is a schematic block diagram of a data acquisition system according to an embodiment of the present application. As Figure 1 shown, the system includes an acquisition model management device 10 and at least one data acquisition device 20. Among them, the acquisition model management device 10 is respectively connected to each data acquisition device 20.

[0029] It should be noted that according to various data acquisition scenarios in practical applications, the data acquisition system may include one or more data acquisition devices 20. As Figure 1 shown, a data acquisition system including 2 data acquisition devices 20 is schematically shown.

[0030] In this embodiment, the acquisition model management device 10 is used to obtain the attribute information and acquisition strategy information of at least one acquisition object, create a data acquisition model corresponding to the acquisition object according to the attribute information and acquisition strategy information, and send the data acquisition model to the data acquisition device 20. Among them, the attribute information includes the identification information of the acquisition object. The data acquisition model includes the identification information and acquisition strategy information corresponding to each acquisition object.

[0031] In this embodiment, the data acquisition device 20 is used to deploy the data acquisition model in at least one cloud processor corresponding to the acquisition object, and use the data acquisition model to perform data acquisition on the acquisition object in the cloud processor.

[0032] Optionally, the data acquisition device is a monitoring data acquisition device in the agent mode. In the agent mode, the data acquisition device is a part of the cloud processor (i.e., the cloud host), and is responsible for collecting monitoring data of the cloud host and the systems, containers, and business programs deployed on the cloud host. Optionally, the data acquisition device can also be a device independent of the cloud processor, and each data acquisition device is respectively connected to a cloud processor. Thus, based on the connection relationship between the data acquisition device and the cloud processor, the data acquisition device can deploy the data acquisition model in its corresponding cloud processor.

[0033] In addition, the data acquisition system may further include a task distribution device, so that the acquisition model management device sends the data acquisition model to the task distribution device, and the task distribution device distributes the data acquisition model to the data acquisition device. Among them, the acquisition model management device is connected to the task distribution device, and the task distribution device is respectively connected to each data acquisition device. This connection relationship is not shown in Figure 1 shown.

[0034] Figure 2 It is a schematic flowchart of a data acquisition method according to an embodiment of the present application, applied to such asFigure 1 The data acquisition system shown, such as Figure 2 shown, the method includes:

[0035] S202, obtaining the attribute information and acquisition policy information of at least one acquisition object.

[0036] Among them, the attribute information may include the identification information of the acquisition object. In this embodiment, the information acquisition unit in the acquisition model management device can obtain asset data from the asset configuration management system, and process the obtained asset data to obtain the acquisition object set and the association relationship between each acquisition object. The acquisition policy information input by the user can be received through the acquisition policy management unit in the acquisition model management device.

[0037] Among them, for the data acquisition system, the asset configuration management system is an external system. The asset configuration management system can be used to perform asset data configuration management on the cloud computing data center (i.e., each cloud processor). The asset data may include the basic asset information, application asset information, logical asset information, association information between the same type of assets, and mapping information between different types of assets of the cloud processor. The basic asset information may include the physical machine information, switch information, firewall information, server information, etc. of the cloud processor, the application asset information may include the business program information deployed on the cloud processor, etc., and the logical asset information may include the organizational structure of the cloud processor and its roles, responsibilities, etc.

[0038] When the information acquisition unit processes the obtained asset data, it can automatically execute the processing process according to the processing policy pre-configured by the user. The processing policy may include cleaning, format conversion, conversion according to business agreement rules, etc. The association relationship between each acquisition object may include the association information between the same type of assets and the mapping information between different types of assets.

[0039] Optionally, the acquisition object may be the basic asset information and the application asset information. When the acquisition object is the application asset information, for example, when the acquisition object is the business program information deployed on the cloud processor, the identification information of the acquisition object may be the program package name, class name, method name, etc. of the business program.

[0040] S204, creating a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition policy information.

[0041] Among them, the data acquisition model includes the identification information and acquisition policy information corresponding to each acquisition object. In this embodiment, the identification information corresponding to each acquisition object can be used as the model entity, the association relationship between each acquisition object can be used as the association relationship, and the acquisition policy information can be used as the attribute of the model entity to construct the data acquisition model.

[0042] S206, deploy the data acquisition model to at least one cloud processor corresponding to the acquisition object.

[0043] In this embodiment, by storing the data acquisition model locally in each cloud processor, there is no need to obtain the data acquisition model from the task distribution device or the acquisition model management device every time, making the data acquisition more flexible and avoiding the impact of the distribution efficiency of the task distribution device or the acquisition model management device on the data acquisition efficiency.

[0044] S208, use the data acquisition model to perform data acquisition on the acquisition object in the cloud processor.

[0045] Adopting the technical solution of the embodiment of the present application, by obtaining the attribute information and acquisition policy information of at least one acquisition object, creating a data acquisition model corresponding to the acquisition object according to the attribute information and acquisition policy information, and deploying the data acquisition model to at least one cloud processor corresponding to the acquisition object, so as to use the data acquisition model to perform data acquisition on the acquisition object in the cloud processor. Compared with the prior art in which data acquisition depends on the received acquisition instructions, this technical solution can achieve the effect of automatically performing data acquisition on the acquisition object in the cloud processor by creating a data acquisition model and deploying the data acquisition model to the cloud processor corresponding to the acquisition object, improving the flexibility of data acquisition. And since the data acquisition model is deployed locally in the cloud processor, there is no need to wait to receive the data acquisition model when performing data acquisition, thus improving the data acquisition efficiency.

[0046] In one embodiment, the acquisition policy information may include the scanning configuration information and acquisition configuration information corresponding to each acquisition object. The scanning configuration information may include the scanning frequency and / or scanning method. The acquisition configuration information may include the acquisition frequency and / or acquisition method.

[0047] Among them, the acquisition method may define through which interface to acquire the corresponding acquisition object. The acquisition method may include acquisition through the tcp interface, acquisition through the http interface, direct detection acquisition, acquisition through the udp interface, etc. The acquisition configuration information may further include the acquisition content, the sending method corresponding to the acquired data, etc. For example, when the acquisition object is a physical machine, the acquisition content in the acquisition configuration information may be configured to include CPU (Central Processing Unit) information, memory information, disk information, etc. The sending method may define through which interface to send the acquired data to the data processing system (for the data acquisition system, the data processing system is an external system). The sending method may include sending by email, sending by http, sending by udp, sending by text message, etc.

[0048] In this embodiment, for the above-mentioned S208, the data acquisition model is used to acquire data from the acquisition objects in the cloud processor, which can be specifically executed as the following steps A1 - A3:

[0049] Step A1: Scan the acquisition objects in the cloud processor according to the scan configuration information.

[0050] Among them, according to the scan frequencies and scan methods of the acquisition objects in the scan configuration information, each acquisition object can be scanned according to the corresponding scan method in each scan cycle.

[0051] Step A2: When the first acquisition object is scanned in the cloud processor, create a data acquisition task corresponding to the first acquisition object.

[0052] Among them, the data acquisition task includes the acquisition configuration information corresponding to the first acquisition object. In this embodiment, when an acquisition object is scanned, the data acquisition task corresponding to the acquisition object can be automatically created. If the acquisition object is not scanned, the data acquisition task corresponding to the acquisition object is not created.

[0053] Step A3: Based on the data acquisition task, acquire data from the first acquisition object.

[0054] In this embodiment, through the data acquisition model deployed in the cloud processor, the data acquisition from the acquisition objects in the cloud processor can be automated, which improves the flexibility of data acquisition and the data acquisition efficiency.

[0055] In one embodiment, after executing step A2 to create the data acquisition task corresponding to the first acquisition object, according to the scan configuration information, continue to scan the acquisition objects in the cloud processor, and based on the scan result, determine whether the first acquisition object exists in the cloud processor; if not, delete the data acquisition task corresponding to the first acquisition object; if so, retain the data acquisition task corresponding to the first acquisition object.

[0056] In this embodiment, due to the real-time variability of the acquisition objects in the cloud processor, by scanning the acquisition objects in each scan cycle according to the scan frequency in the scan configuration information, the acquisition objects with changes in the cloud processor can be detected in a timely manner, so as to add or delete the data acquisition tasks of the corresponding acquisition objects, achieving the effect of timely updating the data acquisition tasks, and thus ensuring the accuracy of data acquisition.

[0057] In one embodiment, after executing the above-mentioned S204 to create the data acquisition model corresponding to the acquisition object, the data acquisition model can be sent to the task distribution device, so that when it is detected that the data acquisition model is not deployed in at least one cloud processor, the data acquisition model is sent to at least one cloud processor in which the data acquisition model is not deployed through the task distribution device.

[0058] Among them, the data acquisition model can carry a version identifier. When the task distribution device receives the data acquisition model of the latest version, it can actively send the data acquisition model to each cloud processor.

[0059] In one embodiment, the cloud processor can report specified information to the task distribution device. The specified information can include a registration request, a running status information, the version of the data acquisition model, etc. When the task distribution device receives the registration request reported by the cloud processor, it can register the cloud processor, and then send the registration completion information and the data acquisition model to the cloud processor. When the task distribution device receives that the running status information reported by the cloud processor is the running status and the data acquisition model is reported, it compares whether the version of the data acquisition model reported by the cloud processor is lower than the version of the data acquisition model stored in the task distribution device. When it is determined that the version of the received data acquisition model is lower than the version of the data acquisition model stored in the task distribution device, the data acquisition model is sent to the cloud processor.

[0060] In this embodiment, by sending the data acquisition model to the task distribution device, the data acquisition model is sent to the cloud processors without deploying the data acquisition model through the task distribution device, ensuring that each cloud processor can receive the data acquisition model, thus ensuring the smooth execution of data acquisition and improving the data acquisition efficiency.

[0061] In one embodiment, the attribute information further includes the association relationship between each acquisition object. The data acquisition model includes the acquisition policy information corresponding to the second acquisition object and the association relationship between the second acquisition object and the third acquisition object. The above S208, using the data acquisition model to perform data acquisition on the acquisition objects in the cloud processor, can be specifically executed as the following steps B1 - B2:

[0062] Step B1, according to the acquisition policy information corresponding to the second acquisition object, perform data acquisition on the second acquisition object to obtain the first target data corresponding to the second acquisition object.

[0063] Step B2, according to the association relationship between the second acquisition object and the third acquisition object and the first target data, determine the second target data corresponding to the third acquisition object.

[0064] For example, the third collection object is a class in the service program information deployed on the cloud processor, the second collection object is multiple methods in this class, the second collection object corresponds to collection policy information, and the third collection object does not have corresponding collection policy information (that is, the data corresponding to the third collection object cannot be directly collected), and the association relationship between the second collection object and the third collection object is an inclusion relationship. Then, after collecting data for the second collection object according to the collection policy information corresponding to the second collection object to obtain the first target data corresponding to the second collection object, the second target data corresponding to the third collection object can be determined according to the inclusion relationship between the second collection object and the third collection object and the first target data.

[0065] In this embodiment, through the data collection model deployed in the cloud processor, it is possible to automatically collect data for the collection objects in the cloud processor, and realize the data collection for the collection objects whose data cannot be directly collected, which not only improves the data collection efficiency but also ensures the comprehensiveness of the collected data.

[0066] Figure 3 It is a schematic flowchart of a data collection method according to another embodiment of the present application, which is applied to a data collection system as Figure 1 shown, as Figure 3 shown, the method includes:

[0067] S301, the data collection device scans the collection objects in the cloud processor according to the scan configuration information in the data collection model.

[0068] Among them, the data collection device can receive the data collection model sent by the task distribution device and deploy the data collection model in the cloud processor. The data collection model may include identification information and collection policy information corresponding to each collection object respectively. The collection policy information includes scan configuration information and collection configuration information corresponding to each collection object. The scan configuration information includes scan frequency and / or scan method. The collection configuration information includes collection frequency and / or collection method.

[0069] S302, when the data collection device scans that the cloud processor includes the first collection object, create a data collection task corresponding to the first collection object.

[0070] Among them, the data collection task includes the collection configuration information corresponding to the first collection object. In one embodiment, after creating the data collection task corresponding to the first collection object, the collection objects in the cloud processor can be continuously scanned according to the scan configuration information, and according to the scan result, it is judged whether the first collection object exists in the cloud processor; if not, the data collection task corresponding to the first collection object is deleted; if so, the data collection task corresponding to the first collection object is retained.

[0071] In S303, the data acquisition device performs data acquisition on the first acquisition object based on the data acquisition task.

[0072] In one embodiment, the data acquisition model may further include the association relationships between the acquisition objects. When the data acquisition model includes the acquisition strategy information corresponding to the second acquisition object and the association relationship between the second acquisition object and the third acquisition object, data acquisition may be performed on the second acquisition object according to the acquisition strategy information corresponding to the second acquisition object to obtain the first target data corresponding to the second acquisition object, and then the second target data corresponding to the third acquisition object may be determined according to the association relationship between the second acquisition object and the third acquisition object and the first target data.

[0073] The specific processes of the above S301 - S303 have been described in detail in the above embodiments and will not be elaborated here.

[0074] In this embodiment, the data acquisition device uses the data acquisition model deployed in the cloud processor to automatically perform data acquisition on the acquisition objects in the cloud processor, improving the flexibility of data acquisition. Moreover, since the data acquisition model is deployed locally in the cloud processor, there is no need to wait to receive the data acquisition model during data acquisition, thus improving the data acquisition efficiency.

[0075] Figure 4 It is a schematic flowchart of the creation and distribution of a data acquisition model according to an embodiment of the present application, applied to the Figure 1 acquisition model management device shown in Figure 4 As shown, the method includes:

[0076] In S401, the acquisition model management device obtains the attribute information of at least one acquisition object and obtains the acquisition strategy information corresponding to at least one acquisition object.

[0077] Among them, the attribute information may include the identification information of the acquisition object and the association relationships between the acquisition objects.

[0078] In S402, the acquisition model management device creates a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition strategy information.

[0079] In S403, the acquisition model management device sends the data acquisition model to the task distribution device so that the task distribution device distributes the data acquisition model to the data acquisition device.

[0080] Among them, when the task distribution device detects that the data collection model is not deployed in at least one cloud processor, it sends the data collection model to at least one cloud processor where the data collection model is not deployed. Thus, the data collection device deploys the received data collection model in the corresponding cloud processor and uses the data collection model to collect data from the collection objects in the cloud processor.

[0081] The specific processes of S401 - S403 above have been described in detail in the above embodiments and will not be elaborated here.

[0082] In this embodiment, the collection model management device can automatically create a data collection model and send the data collection model to the task distribution device. Thus, the task distribution device distributes the data collection model to the data collection device, ensuring that each data collection device can receive the data collection model, thereby ensuring the smooth execution of data collection and improving the data collection efficiency.

[0083] The above is the data collection method and system provided by the embodiments of this application. Next, the Figure 1 shown data collection system will be described in detail with reference to the accompanying drawings.

[0084] In one embodiment, the data collection system further includes a task distribution device. In this embodiment, the collection model management device 10 is further configured to send the data collection model to the task distribution device. The task distribution device is configured to distribute the data collection model to the data collection device 20.

[0085] Among them, the data collection device 20 is further configured to report specified information to the task distribution device. The specified information may include a registration request of the data collection device 20, operation status information, the version of the data collection model, etc. Thus, the task distribution device determines the information to be returned to the data collection device 20 according to the received specified information.

[0086] For example, when the task distribution device receives the registration request of the data collection device, it determines to return the registration completion information to the data collection device 20 and distribute the data collection model. When the task distribution device receives the operation status information indicating the running status and the version of the received data collection model is lower than the version of the data collection model stored in the task distribution device, it determines to return the latest version of the data collection model to the data collection device 20.

[0087] In this embodiment, the collection model management device sends the data collection model to the task distribution device, and thus the task distribution device distributes the data collection model to the data collection device, ensuring that each data collection device can receive the data collection model, thereby ensuring the smooth execution of data collection and improving the data collection efficiency.

[0088] In one embodiment, the acquisition policy information includes the scanning configuration information and the acquisition configuration information corresponding to each acquisition object. The scanning configuration information includes the scanning frequency and / or the scanning method. The acquisition configuration information includes the acquisition frequency and / or the acquisition method. Figure 5 is a schematic block diagram of a data acquisition device according to an embodiment of the present application, as Figure 5 shown, the data acquisition device 20 may include a scanning unit 201, an acquisition unit 202, a receiving unit 203, and a first storage unit 204. Among them, the receiving unit 203 is connected to the scanning unit 201, the scanning unit 201 is connected to the acquisition unit 202, and the scanning unit 201, the acquisition unit 202, and the receiving unit 203 are respectively connected to the first storage unit 204.

[0089] In this embodiment, the scanning unit 201 is configured to scan the acquisition object in the cloud processor according to the scanning configuration information. When the first acquisition object is scanned in the cloud processor, a data acquisition task corresponding to the first acquisition object is created. Among them, the data acquisition task includes the acquisition configuration information corresponding to the first acquisition object.

[0090] In this embodiment, the acquisition unit 202 is configured to perform data acquisition on the first acquisition object based on the data acquisition task. The receiving unit 203 is configured to receive the data acquisition model sent by the task distribution device and send the data acquisition model to the first storage unit 204. The first storage unit 204 is configured to deploy the data acquisition model in the cloud processor.

[0091] In addition, the data acquisition device 20 further includes a sending unit, configured to send the data acquired by the acquisition unit 202 to an external data processing system to implement the processing of the acquired data. The sending unit may be connected to the acquisition unit 202 and connected to the first storage unit 204, and this connection relationship is not shown in Figure 5 shown.

[0092] In this embodiment, the data acquisition device uses the data acquisition model deployed in the cloud processor to automatically perform data acquisition on the acquisition object in the cloud processor, improving the flexibility of data acquisition and the data acquisition efficiency.

[0093] Figure 6 is a schematic block diagram of an acquisition model management device according to an embodiment of the present application, as Figure 6 shown, the acquisition model management device 10 may include an information acquisition unit 101, an acquisition policy management unit 102, an acquisition model management unit 103, and a sending unit 104. Among them, the information acquisition unit 101 and the acquisition policy management unit 102 are respectively connected to the acquisition model management unit 103, and the acquisition model management unit 103 is connected to the sending unit 104.

[0094] In this embodiment, the information acquisition unit 101 is configured to acquire the attribute information of at least one acquisition object and send the attribute information to the acquisition model management unit 103. The acquisition policy management unit 102 is configured to acquire the acquisition policy information corresponding to at least one acquisition object and send the acquisition policy information to the acquisition model management unit 103. The acquisition model management unit 103 is configured to create a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition policy information, and send the data acquisition model to the distribution unit 104. The distribution unit 104 is configured to send the data acquisition model to the task distribution device.

[0095] In addition, the acquisition model management device 10 further includes a second storage unit, which is configured to persistently store the attribute information of the acquisition object, the acquisition policy information corresponding to the acquisition object, and the data acquisition model corresponding to the acquisition object. The second storage unit can be respectively connected to the information acquisition unit 101, the acquisition policy management unit 102, and the acquisition model management unit 103, and is connected to the distribution unit 104. This connection relationship is not shown in Figure 6 Thus, the acquisition model management unit 103 sends the data acquisition model to the second storage unit. After the distribution unit 104 acquires the data acquisition model from the second storage unit, it sends the data acquisition model to the task distribution device.

[0096] In this embodiment, the acquisition model management device realizes the effect of automatically generating a data acquisition model by acquiring the attribute information and the acquisition policy information of at least one acquisition object, creating a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition policy information, and sending the data acquisition model to the task distribution device, providing a model basis for improving the data acquisition efficiency.

[0097] Figure 7 is a schematic block diagram of a data acquisition system according to another embodiment of the present application. As Figure 7 shown, the system includes an acquisition model management device 10, a task distribution device 30, and at least one data acquisition device 20. In this embodiment, the connection relationship between the data acquisition device 20 and the cloud processor is not limited. For example, each data acquisition device 20 is respectively a part of each cloud processor, or each data acquisition device 20 is respectively connected to a cloud processor (not shown in Figure 7 ). Figure 7 A data acquisition system including 2 data acquisition devices 20 is schematically shown. Among them, the acquisition model management device 10 is connected to the task distribution device 30, and the task distribution device 30 is respectively connected to each data acquisition device 20.

[0098] In this embodiment, the acquisition model management device 10 is configured to obtain the attribute information and acquisition policy information of at least one acquisition object, create a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition policy information, and send the data acquisition model to the task distribution device 30. Among them, the attribute information includes the identification information of the acquisition object. The data acquisition model includes the identification information and acquisition policy information respectively corresponding to each acquisition object.

[0099] In this embodiment, the task distribution device 30 is configured to send the data acquisition model to the data acquisition device 20. The data acquisition device 20 is configured to deploy the data acquisition model in at least one cloud processor corresponding to the acquisition object, and use the data acquisition model to perform data acquisition on the acquisition object in the cloud processor.

[0100] For the specific execution units respectively corresponding to the acquisition model management device 10 and the data acquisition device 20, please refer to Figure 5 and Figure 6 the embodiments shown. To avoid repetition, they will not be elaborated here.

[0101] By using the system of the embodiment of the present application, the acquisition model management device creates a data acquisition model corresponding to the acquisition object by obtaining the attribute information and acquisition policy information of at least one acquisition object, and sends the data acquisition model to the task distribution device. The task distribution device sends the data acquisition model to the data acquisition device, and the data acquisition device deploys the data acquisition model in at least one cloud processor corresponding to the acquisition object, so as to use the data acquisition model to perform data acquisition on the acquisition object in the cloud processor. Compared with the prior art in which the data acquisition device depends on the received acquisition instruction for data acquisition, the data acquisition device in this system can achieve the effect of automatically performing data acquisition on the acquisition object in the cloud processor by using the data acquisition model, improving the flexibility of data acquisition. And since the data acquisition model is deployed locally in the cloud processor, there is no need to wait to receive the data acquisition model when performing data acquisition, thus improving the data acquisition efficiency.

[0102] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.

[0103] Those skilled in the art should understand that Figure 1 and Figure 7The data acquisition system in [it] can be used to implement the data acquisition method described above. The detailed description thereof should be similar to that in the method part above. To avoid redundancy, it will not be elaborated here.

[0104] Based on the same idea, the embodiment of the present application also provides a data acquisition device, as Figure 8 shown. The data acquisition device may vary greatly due to different configurations or performances, and may include one or more processors 801 and a memory 802. One or more application programs or data may be stored in the memory 802. Among them, the memory 802 may be transient storage or persistent storage. The application programs stored in the memory 802 may include one or more modules (not shown in the figure), and each module may include a series of computer-executable instructions in the data acquisition device. Further, the processor 801 may be configured to communicate with the memory 802 and execute a series of computer-executable instructions in the memory 802 on the data acquisition device. The data acquisition device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input / output interfaces 805, and one or more keyboards 806.

[0105] Specifically in this embodiment, the data acquisition device includes a memory and one or more programs, where one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the data acquisition device, and is configured to be executed by one or more processors. The one or more programs include the following computer-executable instructions:

[0106] Obtain the attribute information and acquisition strategy information of at least one acquisition object; the attribute information includes the identification information of the acquisition object;

[0107] Create a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition strategy information; the data acquisition model includes the identification information and the acquisition strategy information respectively corresponding to each acquisition object;

[0108] Deploy the data acquisition model to at least one cloud processor corresponding to the acquisition object;

[0109] Use the data acquisition model to perform data acquisition on the acquisition object in the cloud processor.

[0110] Using the device according to the embodiments of the present application, by obtaining the attribute information and collection strategy information of at least one collection object, creating a data collection model corresponding to the collection object according to the attribute information and collection strategy information, and deploying the data collection model in at least one cloud processor corresponding to the collection object, so as to use the data collection model to collect data from the collection object in the cloud processor. Compared with the prior art in which data collection depends on the received collection instructions, the device can achieve the effect of automatically collecting data from the collection object in the cloud processor by creating a data collection model and deploying the data collection model in the cloud processor corresponding to the collection object, improving the flexibility of data collection. And since the data collection model is deployed locally in the cloud processor, there is no need to wait for the data collection model to be received during data collection, thus improving the data collection efficiency.

[0111] The embodiments of the present application also propose a storage medium that stores one or more computer programs. The one or more computer programs include instructions that, when executed by an electronic device including a plurality of application programs, can enable the electronic device to execute each process of the above data collection method embodiment and achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0112] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0113] For the convenience of description, when describing the above device, it is divided into various units according to functions for description. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0114] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0116] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0118] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0119] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0120] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0121] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0122] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0123] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0124] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and variations can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A data acquisition method, characterized in that, Including: Obtaining attribute information and collection strategy information of at least one collection object; The attribute information includes identification information of the collection object and the association relationship between the collection objects; According to the attribute information and the collection strategy information, creating a data collection model corresponding to the collection object; the data collection model includes identification information corresponding to each collection object, the collection strategy information, and the association relationship between the collection objects; Deploying the data collection model to at least one cloud processor corresponding to the collection object; Using the data collection model to collect data from the collection object in the cloud processor; The using the data collection model to collect data from the collection object in the cloud processor includes: collecting data from the second collection object according to the collection strategy information corresponding to the second collection object to obtain first target data corresponding to the second collection object; Determining second target data corresponding to the third collection object according to the association relationship between the second collection object and the third collection object and the first target data.

2. The method according to claim 1, wherein After creating the data collection model corresponding to the collection object, it further includes: Sending the data collection model to a task distribution device; When it is detected that the data collection model is not deployed in at least one of the cloud processors, sending the data collection model to at least one of the cloud processors where the data collection model is not deployed through the task distribution device.

3. A data acquisition system, characterized in that, Including a collection model management device and at least one data collection device; wherein: The collection model management device is used to obtain attribute information and collection strategy information of at least one collection object; according to the attribute information and the collection strategy information, creating a data collection model corresponding to the collection object, and sending the data collection model to the data collection device; the attribute information includes identification information of the collection object and the association relationship between the collection objects; the data collection model includes identification information corresponding to each collection object, the collection strategy information, and the association relationship between the collection objects; The data collection device is used to deploy the data collection model to at least one cloud processor corresponding to the collection object; and using the data collection model to collect data from the collection object in the cloud processor; The using the data collection model to collect data from the collection object in the cloud processor includes: collecting data from the second collection object according to the collection strategy information corresponding to the second collection object to obtain first target data corresponding to the second collection object; determining second target data corresponding to the third collection object according to the association relationship between the second collection object and the third collection object and the first target data.

4. The system according to claim 3, wherein The system further includes a task distribution device; The collection model management device is further used to send the data collection model to the task distribution device; The task distribution device is used to send the data acquisition model to the data acquisition device.

5. The system according to claim 4, characterized in that, The data acquisition device further includes: A receiving unit, configured to receive the data acquisition model sent by the task distribution device and send the data acquisition model to the first storage unit; The first storage unit is configured to deploy the data acquisition model in the cloud processor.

6. The system according to claim 4, wherein The acquisition model management device includes: An information acquisition unit, configured to acquire the attribute information of the at least one acquisition object and send the attribute information to the acquisition model management unit; An acquisition strategy management unit, configured to acquire the acquisition strategy information corresponding to the at least one acquisition object and send the acquisition strategy information to the acquisition model management unit; The acquisition model management unit is configured to create a data acquisition model corresponding to the acquisition object according to the attribute information and the acquisition strategy information, and send the data acquisition model to the sending unit; The sending unit is configured to send the data acquisition model to the task distribution device.

7. A data acquisition device, characterized in that, It includes a processor and a memory electrically connected to the processor. The memory stores a computer program, and the processor is configured to call and execute the computer program from the memory to implement the data acquisition method according to any one of claims 1-2.

8. A storage medium, characterized in that, The storage medium is used to store a computer program, and the computer program is executed by a processor to implement the data acquisition method according to any one of claims 1-2.

Images