A data analysis method, device, equipment and medium

Abstract

The present invention discloses a data analysis method, device, equipment and medium. The method comprises: determining overload data in the data to be processed according to the available analysis capability of a local analysis unit and the analysis capability required by the data to be processed; generating self-explanatory data for the overload data, so that the overload data can be analyzed according to the self-explanatory data by an available analysis unit in a data analysis system. The technical solution of the embodiment of the present invention solves the problem of load balancing and time-sharing processing of data by the analysis unit.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular, to a data analysis method, device, equipment and medium. Background Art

[0002] In the security industry, the business carried by the intelligent analysis system is not only large in scale but also diverse in types. Among them, the analysis system is the core system of intelligent security. How to make full use of the existing analysis capabilities to process more data has become the basic demand of users.

[0003] In the prior art, the most common way for analysis systems to perform data analysis is real-time processing. When the workload of real-time data exceeds the analysis capability of the analysis unit, part of the data to be analyzed will be discarded to ensure that it does not affect subsequent business.

[0004] However, adopting a discarding strategy for real-time data that cannot be processed in time may cause the loss of some key data and incomplete data, thus affecting the analysis and processing of key business. Summary of the invention

[0005] The present invention provides a data analysis method, device, equipment and medium, which solve the problems of data load balancing and data time-sharing processing in analysis units.

[0006] In a first aspect, an embodiment of the present invention provides a data analysis method, which is performed by an analysis unit of a data analysis system, and the method includes:

[0007] Determining overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed;

[0008] Self-explanatory data is generated for the overload data, so that the overload data can be analyzed according to the self-explanatory data by an available analysis unit in a data analysis system.

[0009] In a second aspect, an embodiment of the present invention further provides a data analysis method, which is executed by a central server in a data analysis system, and the method includes:

[0010] generating overload data and self-explanatory data of the overload data by an analysis unit of a data analysis system; wherein the overload data is determined by the analysis unit according to the available analysis capacity of the analysis unit and the analysis capacity required for the data to be processed;

[0011] An available analysis unit is allocated to the overload data, so that the overload data is analyzed by the available analysis unit according to the self-explanatory data.

[0012] In a third aspect, an embodiment of the present invention further provides a data analysis device, which is configured in an analysis unit of a data analysis system, and the device includes:

[0013] An overload data determination module, used to determine overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed;

[0014] The self-explanatory data generating module is used to generate self-explanatory data for the overload data, so as to analyze the overload data according to the self-explanatory data available in the data analyzing system.

[0015] In a fourth aspect, an embodiment of the present invention further provides a data analysis device, which is configured in a central server for data analysis, and the device includes:

[0016] A generation module is used to generate overload data and self-explanatory data of the overload data through an analysis unit of a data analysis system; wherein the overload data is determined by the analysis unit according to the available analysis capability of the analysis unit and the analysis capability required for the data to be processed; and an allocation module is used to allocate an available analysis unit to the overload data so as to analyze the overload data according to the self-explanatory data through the available analysis unit.

[0017] In a fifth aspect, an embodiment of the present invention further provides a device, the device comprising:

[0018] one or more processors;

[0019] a storage device for storing one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement a data analysis method as described in any one of the embodiments of the present invention.

[0021] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data analysis method as described in any one of the embodiments of the present invention.

[0022] The present invention determines the overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed, and generates self-explanatory data for the overload data, so that the overload data is analyzed according to the self-explanatory data by the available analysis unit in the data analysis system. The technical solution of the embodiment of the present invention realizes the problems of load balancing and data time-sharing processing when the analysis unit processes data by generating self-explanatory data for the overload data and analyzing and processing the overload data by the available analysis unit in the data analysis system. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of a data analysis method provided by an embodiment of the present invention;

[0024] Figure 2 A network diagram of a data analysis system provided by an embodiment of the present invention;

[0025] Figure 3 A flowchart of another data analysis method provided by an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a data analysis method provided by an embodiment of the present invention;

[0027] Figure 5 A flowchart of another data analysis method provided by an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the structure of a data analysis device provided by an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of the structure of another data analysis device provided by an embodiment of the present invention;

[0030] Figure 8 A schematic diagram of the structure of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0032] Figure 1 This is a flow chart of a data analysis method provided by an embodiment of the present invention. This embodiment can be applied to situations where data is analyzed. Typically, this method can be applied to situations where an analysis unit in an intelligent analysis system in the security industry analyzes data. This method can be executed by a data analysis device configured in an analysis unit of the data analysis system, and the device can be implemented in the form of acquisition software and / or hardware. Figure 1 , the method may specifically include the following steps:

[0033] Step 110: Determine overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed.

[0034] Among them, the analysis unit is the actual computing unit that performs intelligent analysis. The available analysis capabilities of a single analysis unit are limited. Usually, due to authorization and hardware design limitations, a single analysis unit can only perform analysis of one business type at the same time.

[0035] See Figure 2 , Figure 2 A network diagram of a data analysis system provided by an embodiment of the present invention. The computing nodes correspond to the analysis units in the data analysis system, and the central server is the management server of the analysis unit cluster, which is mainly responsible for analysis unit management and workload scheduling. Users can configure the central server of the data analysis system through the client, and the central server of the analysis system assigns tasks to the analysis units, and the analysis units perform real-time analysis on the pending data assigned by the central server.

[0036] If the analysis capability required for the data to be processed is greater than the available analysis capability of the local analysis unit, the overload data in the data to be processed is determined based on the difference between the above analysis capability and the available analysis capability, and the local analysis unit analyzes other data except the overload data in real time.

[0037] Step 120: Generate self-explanatory data for the overload data, so that the overload data can be analyzed according to the self-explanatory data by an available analysis unit in a data analysis system.

[0038] Wherein, the self-explanatory data includes the task type of the overload data;

[0039] Alternatively, the self-explanatory data includes at least one of time information, source information, and forwarding information of the overload data, and a task type.

[0040] Exemplarily, the task type is an independent intelligent analysis service, which may include data analysis types such as face analysis, video structuring, and behavior analysis. The time information is the creation time of the overload data, and the source information is the overload data acquisition path. The priority order of the overload data can be determined according to the time information and / or source information, which is used to determine the processing order of the overload data. Optionally, the self-explanatory data of the overload data also includes the analysis unit information initially assigned to the overload data.

[0041] The step of generating self-explanatory data for the overload data includes:

[0042] Generate self-explanatory data in at least one of the following forms for the overload data: xml, json, memory block, text.

[0043] By adding context information to the overload data to form self-explanatory data, the overload data can contain the context information of its own operation, so that the overload data can be correctly analyzed and processed according to the self-explanatory data on any analysis unit.

[0044] The available analysis unit is any analysis unit in the current data analysis system whose available analysis capability is greater than or equal to the analysis capability required by the overload data.

[0045] After the self-explanatory data of the overload data is generated, as an optional implementation, if there is an available analysis unit in the current data processing system, the self-explanatory data of the overload data is directly sent to the available analysis unit, and the available analysis unit analyzes the overload data based on the self-explanatory data of the overload data. As another optional implementation, if there is no available analysis unit temporarily in the current data processing system, the self-explanatory data of the overload data can be saved in a cache server to complete the caching of the self-explanatory data of the overload data. Optionally, the overload data and context information are encapsulated together to form the self-explanatory data, and the self-explanatory data of each overload data can be stored in the cache service in order of priority.

[0046] The technical solution of this embodiment determines the overloaded data in the data to be processed based on the available analysis capabilities of the local analysis unit and the analysis capabilities required for the data to be processed, and generates self-explanatory data for the overloaded data, so that the overloaded data can be analyzed according to the self-explanatory data by the available analysis units in the data analysis system, thereby achieving load balancing when the analysis unit processes data and the problem of data time-sharing processing.

[0047] Based on the above embodiment, the method further comprises: receiving self-explanatory data of overload data of any analysis unit;

[0048] The overload data is analyzed based on the self-explanatory data of the received overload data.

[0049] Optionally, the available analysis unit can directly receive the self-explanatory data of the overload data from other analysis units, or directly obtain the corresponding self-explanatory data of the overload data from the cache server. After obtaining the self-explanatory data of the overload data, the available analysis unit analyzes the overload data according to the self-explanatory data of the overload data.

[0050] Optionally, the available analysis unit can directly receive self-explanatory data of overload data from other analysis units, which can specifically include that each analysis unit can notify each other of its own available analysis capabilities, without the need for the central server to participate in monitoring, and completely independent of the central server. When self-explanatory data of overload data is generated on any analysis unit, at the same time or before the self-explanatory data of the overload data is generated, if the analysis unit has already learned that there is an available analysis unit in the current data processing system that has the analysis capabilities required by the overload data, then the self-explanatory data of the overload data is directly sent to the available analysis unit, so that the overload data can be analyzed by the available analysis unit based on the self-explanatory data of the overload data; the central server can also be used to monitor the available analysis capabilities of each analysis unit in real time. If the central server has detected that there are other analysis units that have the analysis capabilities required by the overload data at the same time or before the self-explanatory data of the overload data is generated, then the self-explanatory data of the overload data does not need to be stored and can be directly sent to the available analysis unit for analysis.

[0051] Specifically, the overload data is analyzed according to the self-explanatory data of the received overload data, including:

[0052] According to the task type of the received overload data, the overload data is analyzed based on an analysis algorithm associated with the task type.

[0053] Exemplarily, if the task type of the overload data received by the available analysis unit is face analysis, the available analysis unit may analyze the overload task using an analysis algorithm for face analysis.

[0054] Based on the above embodiment, after analyzing the overload data based on the analysis algorithm associated with the task, the method further includes:

[0055] Determining a storage server for the overload data according to the received forwarding information of the overload data;

[0056] The analysis result of the overload data is forwarded to the storage server.

[0057] The forwarding information of the overload data includes the storage server of the overload data analysis result. After processing the overload data, the analysis unit will forward the analysis result of the overload data to the corresponding storage server according to the forwarding information of the overload data.

[0058] Figure 3This is a flow chart of another data analysis method provided by an embodiment of the present invention. This embodiment can be applied to the case of analyzing data. Typically, this method can be applied to the case of analyzing data by an analysis unit in an intelligent analysis system of the security industry. This method can be executed by a data analysis device configured in a central server of the data analysis system. The device can be implemented in the form of acquisition software and / or hardware. Figure 3 , the method may specifically include the following steps:

[0059] Step 210: Generate overload data and self-explanatory data of the overload data through the analysis unit of the data analysis system; wherein the overload data is determined by the analysis unit according to the available analysis capability of the analysis unit and the analysis capability required for the data to be processed.

[0060] Step 220: Allocate an available analysis unit to the overload data, so that the overload data is analyzed by the available analysis unit according to the self-explanatory data.

[0061] In this embodiment, when the analysis unit processes the data to be processed assigned by the central server, the analysis unit reports the data to the central server in real time. The central server can monitor the available analysis capabilities of the analysis unit in real time. When an analysis unit with the analysis capabilities required for the overload data is detected, the analysis unit is used as an available analysis unit for the overload data, and a control instruction is sent to the available analysis unit to instruct the available analysis unit to obtain self-explanatory data of the overload data from the data cache server, so that the available analysis unit analyzes the overload data based on the self-explanatory data.

[0062] Optionally, if the central server detects that other analysis units are available while listening to the overload data of the analysis unit, the self-explanatory data of the overload data at this time may not be stored but directly sent to the available analysis unit for analysis.

[0063] For example, see Figure 4 In the data analysis system, if the current workload of the analysis unit 1 exceeds its own processing capacity, the analysis unit 1 generates self-explanatory data for the overloaded data in the data to be analyzed, and sends the self-explanatory data of the overloaded data to the cache server for storage.

[0064] Wherein, the self-explanatory data includes the task type of the overload data;

[0065] Alternatively, the self-explanatory data includes time information, at least one of source information and forwarding information of the overload data, and a task type.

[0066] Specifically, if the central server detects that the remaining analysis capacity of other analysis units 2 in the current data analysis system is greater than or equal to the analysis capacity required for the overload data, it sends a control signal to the analysis unit 2 to instruct the analysis unit 2 to obtain the self-explanatory data of the overload data from the data cache server, and analyze the overload data based on the self-explanatory data of the overload data.

[0067] When the workload of the analysis unit 1 drops to a point where the available analysis capacity is greater than the required analysis capacity of the overloaded data, the analysis unit 1 will automatically connect to the central cache server, obtain the self-explanatory data of the overloaded data, and continue to analyze and process the overloaded data based on the self-explanatory data.

[0068] The technical solution of the embodiment of the present invention generates overload data and self-explanatory data of the overload data through the analysis unit in the data analysis system, and allocates available analysis units to the overload data. By scheduling the data to be processed by the analysis unit, the analysis work can be transferred from the high-load analysis unit to the low-load analysis unit, thereby achieving a real-time data processing load balancing effect. Due to the existence of the cache server, when the workload of the entire analysis system is relatively high, the self-explanatory data to be analyzed can still be saved, and the historical data in the cache can be analyzed after the workload is reduced, thereby achieving the effect of data time-sharing processing.

[0069] Figure 5 This is a flow chart of another data analysis method provided by an embodiment of the present invention. Based on the above embodiment, this embodiment further refines step 220. Figure 5 , the method may specifically include the following steps:

[0070] Step 310: Generate overload data and self-explanatory data of the overload data through the analysis unit of the data analysis system; wherein the overload data is determined by the analysis unit according to the available analysis capability of the analysis unit and the analysis capability required for the data to be processed.

[0071] Step 320: Allocate available analysis units to the overload data according to the available analysis capabilities of the analysis units in the data analysis system and the analysis capabilities and priorities required by the overload data, so that the overload data can be analyzed by the available analysis units according to the self-explanatory data.

[0072] Optionally, before allocating available analysis units to the overload data, the method further includes:

[0073] The priority of the overload data is determined according to the time information and / or source information of the overload data.

[0074] In this embodiment, the central server allocates available analysis units to the overload data from analysis units whose available analysis capabilities are greater than or equal to the analysis capabilities required by the overload data.

[0075] If there are multiple overload data in the current cache server, available analysis units are allocated to overload data with high priority and whose required analysis capacity is close to the available analysis capacity of the analysis unit according to the required analysis capacity and priority of the overload data.

[0076] The technical solution of this embodiment allocates available analysis units to overloaded data according to the available analysis capabilities of the analysis units in the data analysis system, as well as the analysis capabilities and priorities required for the overloaded data, thereby achieving load balancing of the data analysis system while ensuring priority processing of important data.

[0077] Figure 6 The present invention provides a structural diagram of a data analysis device provided in an embodiment of the present invention, wherein the device is configured in an analysis unit of a data analysis system. Figure 6 , the device may specifically include:

[0078] The overload data determination module 410 is used to determine the overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed.

[0079] The self-explanatory data generating module 420 is used to generate self-explanatory data for the overload data, so that the overload data can be analyzed according to the self-explanatory data by an available analysis unit in the data analysis system.

[0080] Wherein, the self-explanatory data includes the task type of the overload data;

[0081] Alternatively, the self-explanatory data includes time information, at least one of source information and forwarding information of the overload data, and a task type.

[0082] The self-explanatory data generation module 420 is also specifically used to generate self-explanatory data in at least one of the following forms for the overload data: xml, json, memory block, text.

[0083] Based on the above embodiment, the device further includes an overload data analysis module and a storage module.

[0084] Wherein, the overload data analysis module is used to receive self-explanatory data of overload data of any analysis unit;

[0085] The overload data is analyzed based on the self-explanatory data of the received overload data.

[0086] Specifically, the overload data analysis module is further used to: analyze the overload data according to the task type of the received overload data and based on an analysis algorithm associated with the task type.

[0087] The storage module is used to: determine a storage server for the overload data according to the forwarding information of the received overload data;

[0088] The analysis result of the overload data is forwarded to the storage server.

[0089] A data analysis device provided in an embodiment of the present invention can execute a data analysis method provided in any embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method, which will not be described in detail.

[0090] Figure 7 This is a schematic diagram of the structure of another data analysis device provided by an embodiment of the present invention, and the device is configured in a central server of a data analysis system. Figure 7 , the device specifically comprises:

[0091] The generating module 510 is used to generate overload data and self-explanatory data of the overload data by the analyzing unit of the data analyzing system; wherein the overload data is determined by the analyzing unit according to the available analyzing capability of the analyzing unit and the analyzing capability required by the data to be processed. The allocating module 520 is used to allocate the available analyzing unit to the overload data, so that the available analyzing unit analyzes the overload data according to the self-explanatory data.

[0092] Wherein, the self-explanatory data includes the task type of the overload data;

[0093] Alternatively, the self-explanatory data includes time information, at least one of source information and forwarding information of the overload data, and a task type.

[0094] Based on the above embodiment, the device further includes: a priority determination module, configured to determine the priority of the overload data according to the time information and / or source information of the overload data.

[0095] Specifically, the allocation module 520 is used to allocate available analysis units to the overloaded data according to the available analysis capabilities of the analysis units in the data analysis system and the analysis capabilities and priorities required for the overloaded data, so as to analyze the overloaded data according to the self-explanatory data through the available analysis units.

[0096] A data analysis device provided in an embodiment of the present invention can execute a data analysis method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0097] Figure 8A schematic diagram of the structure of a device provided in an embodiment of the present invention. Figure 8 A block diagram of an exemplary device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 8 The device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0098] like Figure 8 As shown, device 12 is in the form of a general purpose computing device. Components of device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0099] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus architectures. Device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by device 12, including volatile and non-volatile media, removable and non-removable media.

[0100] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0101] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0102] The device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the device 12, and / or may communicate with any device that enables the device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the device 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0103] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a data analysis method provided by an embodiment of the present invention.

[0104] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, a data analysis method provided in the embodiment of the present invention is implemented. The method includes:

[0105] Determining overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed;

[0106] Generate self-explanatory data for the overload data so that the overload data can be analyzed by an available analysis unit in the data analysis system according to the self-explanatory data. And / or:

[0107] generating overload data and self-explanatory data of the overload data by an analysis unit of a data analysis system; wherein the overload data is determined by the analysis unit according to the available analysis capacity of the analysis unit and the analysis capacity required for the data to be processed;

[0108] If yes, an available analysis unit is allocated to the overload data, so that the overload data is analyzed by the available analysis unit according to the self-explanatory data.

[0109] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0110] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0111] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0112] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0113] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data analysis method, characterized in that: Executed by an analysis unit of a data analysis system, the method includes: Determining overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed; generating self-explanatory data for the overload data, so as to analyze the overload data according to the self-explanatory data by an available analysis unit in a data analysis system; Wherein, the available analysis unit directly receives the self-explanatory data from other analysis units, or directly obtains the self-explanatory data from a cache server; The self-explanatory data includes the overload data and context information; The self-explanatory data includes the task type of the overload data; Alternatively, the self-explanatory data includes at least one of time information, source information and forwarding information of the overload data, and a task type; The task types are mutually independent intelligent analysis services, and the task types include face analysis, video structuring, and behavior analysis.

2. The method according to claim 1, characterized in that Generating self-explanatory data for the overload data, including: Generate self-explanatory data in at least one of the following forms for the overload data: xml, json, memory block, text.

3. The method according to claim 1, characterized in that The method further comprises: Receiving self-interpreting data of overload data of any analysis unit; The overload data is analyzed based on the self-explanatory data of the received overload data.

4. The method according to claim 3, characterized in that The overload data is analyzed according to the self-explanatory data of the received overload data, including: According to the task type of the received overload data, the overload data is analyzed based on an analysis algorithm associated with the task type.

5. The method according to claim 4, characterized in that After analyzing the overload data based on the analysis algorithm associated with the task, it also includes: Determining a storage server for the overload data according to the received forwarding information of the overload data; The analysis result of the overload data is forwarded to the storage server.

6. A data analysis method, characterized in that: Executed by a central server of a data analysis system, the method comprises: generating overload data and self-explanatory data of the overload data by an analysis unit of a data analysis system; wherein the overload data is determined by the analysis unit according to the available analysis capacity of the analysis unit and the analysis capacity required for the data to be processed; allocating an available analysis unit to the overload data, so as to analyze the overload data according to the self-explanatory data by the available analysis unit; Wherein, the available analysis unit directly receives the self-explanatory data from other analysis units, or directly obtains the self-explanatory data from a cache server; The self-explanatory data includes the overload data and context information; The self-explanatory data includes the task type of the overload data; Alternatively, the self-explanatory data includes at least one of time information, source information and forwarding information of the overload data, and a task type; The task types are mutually independent intelligent analysis services, and the task types include face analysis, video structuring, and behavior analysis.

7. The method according to claim 6, characterized in that Before allocating the available analysis unit to the overload data, the method further includes: The priority of the overload data is determined according to the time information and / or source information of the overload data.

8. The method according to claim 7, characterized in that Allocating an available analysis unit to the overload data so as to analyze the overload data according to the self-explanatory data by the available analysis unit comprises: According to the available analysis capabilities of the analysis units in the data analysis system, the analysis capabilities required for the overload data and the priority, available analysis units are allocated to the overload data so that the overload data is analyzed according to the self-explanatory data by the available analysis units.

9. A data analysis device, characterized in that: An analysis unit configured in a data analysis system, the device comprising: An overload data determination module, used to determine overload data in the data to be processed according to the available analysis capability of the local analysis unit and the analysis capability required by the data to be processed; a self-explanatory data generating module, configured to generate self-explanatory data for the overload data, so as to analyze the overload data according to the self-explanatory data through an available analysis unit in a data analysis system; Wherein, the available analysis unit directly receives the self-explanatory data from other analysis units, or directly obtains the self-explanatory data from a cache server; The self-explanatory data includes the overload data and context information; The self-explanatory data includes the task type of the overload data; Alternatively, the self-explanatory data includes at least one of time information, source information and forwarding information of the overload data, and a task type; The task types are mutually independent intelligent analysis services, and the task types include face analysis, video structuring, and behavior analysis.

10. A data analysis device, characterized in that: The central server configured for data analysis includes: A generating module, configured to generate overload data and self-explanatory data of the overload data by an analyzing unit of a data analyzing system; wherein the overload data is determined by the analyzing unit according to an available analyzing capability of the analyzing unit and an analyzing capability required by the data to be processed; an allocation module, configured to allocate an available analysis unit to the overload data, so as to analyze the overload data according to the self-explanatory data through the available analysis unit; Wherein, the available analysis unit directly receives the self-explanatory data from other analysis units, or directly obtains the self-explanatory data from a cache server; The self-explanatory data includes the overload data and context information; The self-explanatory data includes the task type of the overload data; Alternatively, the self-explanatory data includes at least one of time information, source information and forwarding information of the overload data, and a task type; The task types are mutually independent intelligent analysis services, and the task types include face analysis, video structuring, and behavior analysis.

11. A device, characterized in that: The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement a data analysis method as described in any one of claims 1-5 or claims 6-8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a data analysis method as described in any one of claims 1 to 5 or claims 6 to 8 is implemented.

Images