Running log processing method, electronic equipment and storage medium

Abstract

The invention discloses a running log processing method, electronic equipment and a storage medium. The method comprises the steps of generating a target operation log of network equipment based on interaction data of the network equipment and a network platform in a process of interaction between the network equipment and the network platform; the target running log is fragmented based on the identification information of the network device, at least one log fragment is obtained, and different log fragments contain different identification information corresponding to log data; and storing the at least one log fragment based on the generation time of the log data contained in the at least one log fragment. The technical problem that the query efficiency of the running log is low in the related technology is solved.

Description

Technical Field

[0001] This application relates to the field of processing operation logs. Specifically, it relates to a method for processing operation logs, an electronic device, and a storage medium. Background Art

[0002] The Internet of Things platform bears the connection and message communication services of a large number of devices, and the cloud operation logs are a powerful means to record device behaviors and troubleshoot problems. The interaction behaviors of a large number of devices with the Internet of Things platform generate a large amount of operation logs, which brings great pressure to the storage and retrieval of logs. Currently, the method of indexing logs is used to store operation logs, but this storage method consumes a large amount of storage resources, and the excessive amount of stored data makes subsequent query difficult.

[0003] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0004] Embodiments of this application provide a method for processing operation logs, an electronic device, and a storage medium, so as to at least solve the technical problem of low query efficiency of operation logs in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for processing operation logs is provided, including: during the interaction process between a network device and a network platform, generating a target operation log of the network device based on the interaction data between the network device and the network platform; performing sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; storing at least one log shard based on the generation time of the log data included in at least one log shard.

[0006] According to another aspect of the embodiments of this application, a method for processing operation logs is further provided, including: a cloud server receives a log query request sent by a client, where the log query request carries the identification information of the device to be queried and the time period to be queried; the cloud server obtains a target log shard that matches the identification information from the operation log set, where the log shards in the operation log set are obtained by performing sharding processing on the target operation log based on the identification information of the network device, the network device interacts with the network platform to which the cloud server belongs, and the target operation log is an operation log generated based on the interaction data between the network device and the network platform; the cloud server obtains target log data that matches the time period to be queried from the target log shard, where the log shards in the operation log set are stored based on the generation time of the log data included in the log shards; the cloud server outputs the target log data to the client.

[0007] According to another aspect of the embodiments of the present application, there is also provided a method for processing operation logs, including: during the interaction between the Internet of Things device and the Internet of Things platform, generating the target operation log of the Internet of Things device based on the interaction data between the Internet of Things device and the Internet of Things platform; performing sharding processing on the target operation log based on the device identification information and product identification information of the Internet of Things device to obtain at least one log shard, wherein the device identification information and product identification information corresponding to the log data included in different log shards are different; storing the at least one log shard based on the generation time of the log data included in the at least one log shard.

[0008] According to another aspect of the embodiments of the present application, there is also provided a method for processing operation logs, including: in response to a device interaction instruction acting on the operation interface, displaying the interaction data between the network device and the network platform on the operation interface; in response to a log storage instruction acting on the operation interface, displaying the storage result of the target operation log of the network device on the operation interface, wherein the target operation log is an operation log generated based on the interaction data, the storage result is a result obtained by storing the at least one log shard based on the generation time of the log data included in the at least one log shard, and the at least one log shard is a shard obtained by performing sharding processing on the target operation log based on the identification information of the network device, and the identification information corresponding to the log data included in different log shards is different.

[0009] According to another aspect of the embodiments of the present application, there is also provided a method for processing operation logs, including: obtaining the interaction data between the network device and the network platform by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the interaction data; generating the target operation log of the network device based on the interaction data; performing sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard, wherein the identification information corresponding to the log data included in different log shards is different; storing the at least one log shard based on the generation time of the log data included in the at least one log shard to obtain a storage result; outputting the storage result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the storage result.

[0010] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory storing an executable program; a processor for running the program, wherein when the program runs, it executes the method of any one of the above embodiments.

[0011] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, characterized in that the computer-readable storage medium includes an executable program stored for the method of any one of the above embodiments.

[0012] In an embodiment of the present application, during the interaction between a network device and a network platform, based on the interaction data between the network device and the network platform, a target operation log of the network device is generated; the target operation log is segmented based on the identification information of the network device to obtain at least one log segment, where the identification information corresponding to the log data included in different log segments is different; the at least one log segment is stored based on the generation time of the log data included in the at least one log segment, thereby improving the query efficiency of the target operation log; it is easy to notice that the target operation log can be sampled at a preset sampling rate to reduce the number of operation logs. After segmenting and storing the target operation log through the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and further solving the technical problem of low query efficiency of operation logs in the related art.

[0013] It is easy to notice that the above general description and the following detailed description are only for exemplifying and explaining the present application, and do not constitute a limitation to the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation to the present application. In the drawings:

[0015] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for processing operation logs according to an embodiment of the present application;

[0016] Figure 2 is a structure block diagram of a computing environment according to an embodiment of the present application;

[0017] Figure 3 is a structure block diagram of a service mesh according to an embodiment of the present application;

[0018] Figure 4 is a flowchart of a method for processing operation logs according to Embodiment 1 of the present application;

[0019] Figure 5 is a structural schematic diagram of a process for processing operation logs according to an embodiment of the present application;

[0020] Figure 6 is a schematic diagram of a multi-segment storage mechanism according to an embodiment of the present application;

[0021] Figure 7 is a schematic diagram of a downsampling function according to an embodiment of the present application;

[0022] Figure 8It is a flowchart of a method for processing operation logs according to Embodiment 2 of the present application;

[0023] Figure 9 It is a flowchart of a method for processing operation logs according to Embodiment 3 of the present application;

[0024] Figure 10 It is a flowchart of a method for processing operation logs according to Embodiment 4 of the present application;

[0025] Figure 11 It is a flowchart of a method for processing operation logs according to Embodiment 5 of the present application;

[0026] Figure 12 It is a schematic diagram of a device for processing operation logs according to Embodiment 6 of the present application;

[0027] Figure 13 It is a schematic diagram of a device for processing operation logs according to Embodiment 7 of the present application;

[0028] Figure 14 It is a schematic diagram of a device for processing operation logs according to Embodiment 8 of the present application;

[0029] Figure 15 It is a schematic diagram of a device for processing operation logs according to Embodiment 9 of the present application;

[0030] Figure 16 It is a schematic diagram of a device for processing operation logs according to Embodiment 10 of the present application;

[0031] Figure 17 It is a block diagram of a computer terminal according to an embodiment of the present application. Detailed implementation manners

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] First, some nouns or terms that appear during the description of the embodiments of this application are applicable to the following explanations:

[0035] The Internet of Things platform can be a platform that integrates device management, device access, and up / downstream message subscription and transfer. It supports device access and device data upload in the south direction, and supports the issuance of cloud control instructions in the north direction.

[0036] A product can be a collection of devices, usually a collection of devices with the same function definition. For example, a product refers to products of the same model, and a device is a certain device under that model.

[0037] Device cloud operation logs: There are various types of interaction methods between Internet of Things devices and the Internet of Things platform, such as device login and offline operations, devices sending messages to the Internet of Things platform, the Internet of Things platform transferring messages to customers for a high-performance message queue system (Kafka cluster) for task integration, and the Internet of Things platform sending messages to devices, etc.; all these interactions require corresponding log records to facilitate system analysis, device status analysis, device fault diagnosis, etc.

[0038] Log component (Loki), an open-source technical component positioned for log collection, aggregation storage, and query;

[0039] File component (Minio), an open-source technical component positioned for the storage and query of file objects.

[0040] Currently, with the application of the Internet of Things platform in various industries, the demands for privately deployed Internet of Things platforms are gradually increasing, and miniaturization and low resource requirements are the core requirements for privately deployed Internet of Things platforms. Traditional log processing and analysis solutions based on inverted indexes index log content, resulting in large consumption of computing and storage resources.

[0041] The industry-popular log storage solution uses open-source components such as ElasticSearch (abbreviated as ES), Logstash (abbreviated as LS), and Kibana (abbreviated as KB) for analysis and visualization platforms. Among them, ES is an open-source distributed search engine based on inverted indexes, providing three major functions: collecting, analyzing, and storing data. The indexing mechanism of ES determines that ES is suitable for full-text indexing and searching of document types, but it also determines that its storage overhead is relatively large, which is not suitable for private cloud IoT platforms with low resource requirements. In the scenario of device operation logs in the IoT platform, it faces the problems of sharply increasing storage costs and deteriorating query performance. Moreover, the typical query scenario for device operation logs is to query the operation logs of a specified device, and the full-text indexing ability of ES is not required. Therefore, the ELK solution is not suitable for the scenario of device operation log tasks where a large number of operation logs are continuously generated.

[0042] Some IoT platforms adopt a distributed computing platform (Hadoop) to build a storage and retrieval solution for operation logs. Hadoop is a typical open-source solution for big data computing scenarios, which can make full use of the capabilities of distributed computing and is suitable for large-scale log data analysis and computing scenarios. However, the Hadoop distributed computing platform is also an architecture that consumes a huge amount of resources and has high requirements for storage and computing resources, and is not suitable for private cloud IoT platforms with low resource requirements; moreover, the typical capabilities of Hadoop are for large-scale log data analysis and are not suitable for the typical usage scenarios of device operation log retrieval.

[0043] Based on the log processing system (Flink), a system that can process a large amount of logs in real time can be implemented. By cooperating with other log collection components (such as Logstash) and log storage and retrieval components (such as ES), functions such as log collection, transmission, real-time cleaning, anomaly detection, storage, and retrieval can be realized; the core role of Flink in this is log cleaning, format normalization, etc. This solution is suitable for building a unified log service system for large-scale multi-system platforms. Because of its high resource requirements, it is not suitable for private cloud IoT platforms with low resource requirements; moreover, log format normalization and log cleaning requirements are not the capabilities required by device operation logs.

[0044] This solution is based on open-source log components that index log metadata (meta), and combines the task characteristics of the IoT system to implement a low-resource-consuming and flexible cloud-based operation log system through methods such as multi-level data sharding and automatic downsampling.

[0045] Example 1

[0046] According to an embodiment of the present application, a method for processing operation logs is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0047] The method embodiment provided by the first embodiment of the present application can be executed in a mobile terminal, a computer terminal, or a similar computing device. Figure 1 It is a hardware structural block diagram of a computer terminal (or mobile device) for implementing the method for processing operation logs according to an embodiment of the present application. As Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b,..., 102n in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0048] It should be noted that the above-mentioned one or more processors 102 and / or other data processing circuits can generally be referred to as "data processing circuits" in this article. The data processing circuit can be embodied in whole or in part as software, hardware, firmware, or any arbitrary combination thereof. In addition, the data processing circuit can be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is used for processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0049] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the processing method of the running log in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned processing method of the running log. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0050] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0051] The display can be, for example, a touch-screen liquid crystal display (Liquid Crystal Display, LCD), and this liquid crystal display enables a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0052] Figure 1 The shown hardware structure block diagram can not only be used as an exemplary block diagram of the above computer terminal 10 (or mobile device), but also as an exemplary block diagram of the above server. In an alternative embodiment, Figure 2 is shown in a block diagram using the above Figure 1 shown computer terminal 10 (or mobile device) as a computing node in the computing environment 201 in one embodiment. Figure 2 is a structural block diagram of a computing environment according to an embodiment of the present application, such as Figure 2As shown, the computing environment 201 includes multiple computing nodes (such as servers, shown as 210-1, 210-2, … in the figure) running on a distributed network. Each computing node contains local processing and memory resources, and end users 202 can remotely run applications or store data in the computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3, and 220-4 in the computing environment 201, representing services "A", "D", "E", and "H" respectively.

[0053] End users 202 can provide and access services through a web browser or other software applications on the client side. In some embodiments, the provisioning and / or requests of end users 202 can be provided to the ingress gateway 230. The ingress gateway 230 can include a corresponding proxy to handle the provisioning and / or requests for services (one or more services provided in the computing environment 201).

[0054] Services are provided or deployed according to various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar means. VM-based virtualization can simulate a real computer by initializing a virtual machine and execute programs and applications without directly accessing any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, containers can be launched to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0055] In one embodiment of container-based virtualization, several containers of a service can be assembled into a Pod (e.g., Kubernetes Pod). For example, as Figure 2 shown, service 220-2 can be equipped with one or more Pods 240-1, 240-2, …, 240-N (collectively referred to as Pods). A Pod can include a proxy 245 and one or more containers 242-1, 242-2, …, 242-M (collectively referred to as containers). One or more containers in the Pod handle requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0056] During operation, executing a user request from end user 202 may require invoking one or more services in the computing environment 201, and executing one or more functions of a service may require invoking one or more functions of another service. As Figure 2As shown, service "A" 220-1 receives a user request from end user 202 at the ingress gateway 230. Service "A" 220-1 may invoke service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to perform one or more functions.

[0057] The computing environment described above may be a cloud computing environment where the allocation of resources is managed by a cloud service provider, allowing for the development of functions without concern for implementing, tuning, or scaling servers. This computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be broken down into sets of functions that can scale automatically and independently, rather than scaling a single hardware device to handle potential loads.

[0058] In another alternative embodiment, Figure 3 is shown in block diagram form as an embodiment of a service mesh using the Figure 1 computer terminal 10 (or mobile device) shown above. Figure 3 is a structural block diagram of a service mesh according to an embodiment of the present application, as Figure 3 shown. The service mesh 300 is mainly used to facilitate secure and reliable communication between multiple microservices. A microservice refers to breaking down an application into multiple smaller services or instances and running them on different clusters / machines.

[0059] As Figure 3 shown, the microservices may include application service instance A and application service instance B, and application service instance A and application service instance B form the functional application layer of the service mesh 300. In one implementation, application service instance A runs in the form of a container / process 308 on a machine / workload container group 314 (Pod), and application service instance B runs in the form of a container / process 310 on a machine / workload container group 316 (Pod).

[0060] In one implementation, application service instance A may be a product query service, and application service instance B may be a product order placement service.

[0061] As Figure 3As shown, application service instance A and mesh proxy (sidecar) 303 coexist in machine workload container group 314, and application service instance B and mesh proxy 305 coexist in machine workload container 316. Mesh proxy 303 and mesh proxy 305 form the data plane layer (dataplane) of service mesh 300. Among them, mesh proxy 303 and mesh proxy 305 run in the form of container / process 304 and container / process 306 respectively, and can receive requests 312 for commodity query services. Moreover, two-way communication is possible between mesh proxy 303 and application service instance A, and between mesh proxy 305 and application service instance B. In addition, two-way communication is also possible between mesh proxy 303 and mesh proxy 305.

[0062] In one implementation, the traffic of application service instance A is routed to the appropriate destination through mesh proxy 303, and the network traffic of application service instance B is routed to the appropriate destination through mesh proxy 305. It should be noted that the network traffic mentioned here includes but is not limited to forms such as Hyper Text Transfer Protocol (abbreviated as HTTP), Representational State Transfer (abbreviated as REST), high-performance, general open-source framework (google Remote Procedure Call, gRPC), and open-source in-memory data structure storage system (Redis).

[0063] In one implementation, the function of the extended data plane layer can be achieved by writing a custom filter (Filter) for the proxy (Envoy) in service mesh 300. The service mesh proxy configuration can be used to correctly proxy service traffic in the service mesh, realizing service interconnection and service governance. Mesh proxy 303 and mesh proxy 305 can be configured to perform at least one of the following functions: service discovery, health checking, routing, load balancing, authentication and authorization, and observability.

[0064] As Figure 3 shown, the service mesh 300 also includes a control plane layer. Among them, the control plane layer can be a group of services running in a dedicated namespace, and these services are hosted by the managed control plane component 301 in machine / workload container group (machine / Pod) 302. As Figure 3As shown in the figure, the managed control plane component 301 communicates bidirectionally with the grid agents 303 and 305. The managed control plane component 301 is configured to perform some control and management functions. For example, the managed control plane component 301 receives the telemetry data transmitted by the grid agents 303 and 305, and can further aggregate this telemetry data. For these services, the managed control plane component 301 can also provide user-facing application programming interfaces (APIs) to more easily manipulate network behavior and provide configuration data to the grid agents 303 and 305, etc.

[0065] In the above operating environment, the present application provides a method for processing the operating log as shown in Figure 4 the figure. Figure 4 It is a flowchart of the method for processing the operating log according to Embodiment 1 of the present application. As shown in Figure 4 the figure, the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. Here, the client devices 20 can include, but are not limited to: smart phones, tablet computers, laptop computers, palmtop computers, personal computers, smart home devices, in-vehicle devices, etc. The client device 20 can interact with the user through a graphical user interface. The method includes:

[0066] Step S402, during the interaction between the network device and the network platform, based on the interaction data between the network device and the network platform, generate the target operating log of the network device;

[0067] The above-mentioned network devices include, but are not limited to, Internet of Things gateways, sensors, controllers, tag devices, smart devices, wireless devices, communication devices, and storage devices. Here, the network devices are not limited and can be any network devices.

[0068] The above-mentioned network platform can be an Internet of Things platform, a management platform, etc. Here, it is not limited. Among them, the Internet of Things platform can be a platform integrating functions such as sensors, communication devices, data storage, and analysis. The Internet of Things platform can be used to carry connection and message communication services for a large number of devices.

[0069] The above-mentioned network platform can be network platforms in different fields, such as network platforms in the sales field, production field, or design field. Here, the type of the network platform is not limited.

[0070] Optionally, during the interaction between the network device and the network platform, the first component of the network platform generates the target operation log of the network device based on the interaction data between the network device and the network platform; the second component of the network platform samples the target operation log at a preset sampling rate to obtain a sampled operation log.

[0071] The above-mentioned first component of the network platform can be a functional component. Among them, the functional component can include a data access component, a data flow component, a gateway interface component, a control component, etc., which are not limited here. Different functional components can be used to implement different functions of the network platform.

[0072] The above-mentioned second component of the network platform can be an operation log component. Among them, the operation log component can include a collection component, a storage component, and a retrieval component. The collection component is used to collect the target operation log, the storage component is used to store the target operation log, and the retrieval component is used to retrieve the target operation component.

[0073] The above-mentioned target operation log can be a file or database that records the information generated during the interaction between the network device and the network platform, and can include but are not limited to content such as program startup, events during operation, errors, warnings, and debugging information. Among them, the target operation log is usually used to track the operation status of the program, help developers quickly locate and solve problems, and can also be used to detect the operation status and performance of the device or platform.

[0074] In an optional embodiment, during the interaction between the network device and the network platform, the first component of the network platform can obtain the target operation log based on the interaction data between the network device and the network platform. Among them, the interaction data can refer to the data collected by the network device through sensors or other means, and the collected data can be transmitted to the network platform. The data generated during the transmission process can be the interaction data, and the target operation log can be generated based on the interaction data so that subsequent personnel can detect the interaction situation between the network device and the network platform through the log.

[0075] The above-mentioned preset sampling rate can be a pre-set sampling rate, and this preset sampling rate can be flexibly adjusted according to the actual sampling scenario.

[0076] In an optional embodiment, since the number of generated target operation logs is large, it is necessary to further sample the target operation log to reduce the number of target operation logs and obtain the target operation log. The second component of the network platform can sample the target operation log at a preset sampling rate to obtain a smaller number of target operation logs.

[0077] Step S404: Perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard;

[0078] Among them, the identification information corresponding to the log data included in different log shards is different.

[0079] The identification information of the above network device may include but is not limited to the device identification of the network device and the product identification on the network device. The identification information of the network device may also be other identifications of the network device. The identification information is not limited here.

[0080] In an alternative embodiment, the target running log can be sharded by the identification information of the network device, which can reduce the index tags of the target running log. The log shards containing the same identification information in the target running log can be sharded under the same index tag, so that the target running log corresponding to the network device can be viewed according to the identification information of the network device subsequently, thereby improving the subsequent query efficiency of the target running log.

[0081] Furthermore, the identification information can be further classified. The identification information belonging to the same class can be assigned to the same index tag according to different classification criteria, and then the target running log is sharded based on the index tag, thereby reducing the number of shards of the target running log and improving the subsequent query efficiency of the target running log.

[0082] The identification information corresponding to the same modulus value can be assigned to the same index tag by taking the modulus of the identification information. This is only an example here and is not limited to this method.

[0083] Step S406, store at least one log shard based on the generation time of the log data included in the at least one log shard.

[0084] When generating the target running log, the generation time of the target running log is generally included in the target running log. Therefore, when storing at least one log shard, it is necessary to store at least one log shard according to the generation time of the target running log, so that at least one log shard can display the interaction between the network device and the network platform in chronological order. Storing at least one log shard in chronological order according to the generation time of the log data in the at least one log shard can also facilitate viewing the log data at the corresponding time point in chronological order subsequently.

[0085] The second component can store at least one log shard based on the generation time of the log data included in the at least one log shard.

[0086] This application can be used for the collection, storage, and query of cloud running logs, and involves core technical solutions such as log sharding, automatic downsampling, and specified device log generation. Figure 5It is a schematic structural diagram of a processing process of operation logs according to an embodiment of the present application. As Figure 5 shown, the entire processing process includes an Internet of Things platform, an operation log component, a network device, and a message queue system (Kafka). Among them, the Internet of Things platform includes a multi-task component, including an interface gateway component (Application Programming Interface, abbreviated as API), a console component, an access layer component, and a data flow component. The operation log component includes a collection component, a storage component, and a retrieval component.

[0087] As Figure 5 shown, the operation logs of the Internet of Things platform are printed by each task component of the Internet of Things platform and are used to record the operation behaviors related to Internet of Things devices. For example, when a network device goes online, the access layer component will record information such as the online time and whether it is successful. The message of the network device going online will be recorded by the access layer component with corresponding information such as the topic, time, and payload. The data flow component will record the transfer time, destination, and whether it is successful, etc. The operation log component can print logs in a fixed format. The log information includes time, tenant identification information (Identification, abbreviated as ID), product key, device name, trace ID (traceId), operation type, operation identification (Code), status, etc. Among them, the product key and device name are used as the sharding keys for log data storage.

[0088] Through the above steps, in the process of the network device interacting with the network platform, based on the interaction data between the network device and the network platform, the target operation log of the network device is generated; based on the identification information of the network device, the target operation log is sharded to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; based on the generation time of the log data included in at least one log shard, at least one log shard is stored, thereby improving the query efficiency of the target operation log; it is easy to notice that the target operation log can be sampled according to a preset sampling rate to reduce the number of operation logs. After sharding and storing the target operation log through the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and thus solving the technical problem of the low query efficiency of operation logs in the related art.

[0089] In the above embodiments of the present application, the target running log is segmented based on the identification information of the network device to obtain at least one log segment, including: performing a modulo operation on the identification information to obtain an index label corresponding to the target running log; segmenting the target running log based on the index label to obtain at least one log segment, where the index labels corresponding to the log data included in different log segments are the same.

[0090] The above identification information may be the device name, device address, etc. of the network device. There is no limitation on the identification information here, and it can be any identification information of the network device.

[0091] The above modulo operation may refer to obtaining relevant data of the network device according to the identification information, and the correlation degree between identification information can be increased by reducing the radix of the identification information. For example, different data are included in multiple identification information, but after performing the modulo operation on multiple identification information, the identification information can be limited between 0 and 9. In this way, if there are 100 identification information, they are finally divided into 10 categories, and the index labels corresponding to the 10 categories can be determined. The identification information is assigned to 10 index labels according to the value obtained by taking the modulo of the identification information, thereby reducing the number of segments of the running log and improving the subsequent query efficiency of the log.

[0092] A typical scenario for retrieving the running log in the cloud of the Internet of Things platform is to query the log of a certain device under a certain product for a specified time period. Selecting the product and device name as labels for log data segmentation is beneficial to reducing the scope of log search when retrieving the running log. However, the numbers of both the product and device names are dynamic. For example, there may be more than 100 products, and the number of devices may reach the order of 1 million. If the product identification and device name are directly used for running log segmentation, it will cause the problem of excessive segmentation, which is instead not conducive to the storage and retrieval of the log.

[0093] To solve the above problems, the present application can adopt the method of reducing the radix of the product identification and device name. Instead of directly using the product identification and device name as index labels, the modulo operation is respectively performed on their hash values, and the value obtained by taking the modulo is used as the index label. After reducing the radix, the number of segments of the log can be less than or equal to the product of the two moduli.

[0094] In the above embodiments of the present application, performing a modulo operation on the identification information to obtain an index label corresponding to the target running log includes: performing a hash operation on the identification information to obtain a hash value; determining the modulus of the modulo operation based on the number of network devices; performing a modulo operation on the same hash value based on the modulus to obtain an index label.

[0095] The above modulus can be used to determine the number of segments of the target running log.

[0096] In an alternative embodiment, the product identifier can be hashed to obtain a hash value of the product identifier, and the device name can be hashed to obtain a hash value of the device name. The same hash value can be modulo-processed according to the modulus obtained by the modulo operation, so as to allocate the running logs corresponding to the same hash value to the same index label, thereby improving the subsequent query efficiency.

[0097] For example, the modulus value of the product identifier can be calculated in the way of the hash value of the product identifier % 10, and the obtained value can be 0-9; the modulus value of the device name can be calculated in the way of the hash value of the device name % 50, and the obtained value is 0-49; the number of shards can be controlled within 500.

[0098] In another alternative embodiment, the number of network devices corresponding to the same hash value can be determined, and the modulus can be determined according to the number of network devices, so as to evenly disperse the target running logs in the relevant shards.

[0099] For example, if the number of products in a certain customer environment is less than 10 and the device data is unevenly distributed at the product granularity, the modulus of the product identifier dimension can be set to 2, and the modulus of the device name dimension can be set to 200 to balance the number of devices in each shard; conversely, if the number of products is more than 1000 and the number of devices under each product is less than 1000, the modulus of the product identifier dimension can be set to 20, and the modulus of the device name dimension can be set to 30 to balance the number of devices in each shard.

[0100] Figure 6 It is a schematic diagram of a multi-shard storage mechanism according to an embodiment of the present application. As Figure 6 shown, the target running logs included can be Pk=dacrvvxxz, dn=abczzdfcsz, xxx; Pk=dxxcvvxxz, dn=erszzdfcsz, xxx; Pk=xecrvvxdz, dn=tffszdfcsz, xxx; Pk=aaadvvxxz, dn=osjfndfcsz, xxx; The Pk and dn of the target running logs can be modulo-reduced to obtain Pk_idx=0, dn_idx=15, xxx; Pk_idx=3, dn_idx=23, xxx; Pk_idx=0, dn_idx=15, xxx; Pk_idx=0, dn_idx=37, xxx. The target running logs can be stored through a multi-shard mechanism according to the values of Pk_idx and dn_idx, thereby reducing the number of index labels and improving the query efficiency.

[0101] Figure 6The sharded data can be stored in a file in the form of time-series data. With the existing storage models, the storage solution can be selected freely. If you need to reduce the cost of log storage and improve the storage performance, you can choose a local persistent volume (PV for short) for log storage; if you want higher log data availability, higher log storage and retrieval performance for the system, you can use a distributed object storage server (minio object service) to start multiple log components simultaneously for storage.

[0102] In the above embodiments of the present application, the target running log is sharded based on the identification information of the network device to obtain at least one log shard, including: determining the generation rate of the target running log; in the case where the generation rate is less than the preset rate threshold, sharding the target running log based on the identification information of the network device to obtain at least one log shard.

[0103] The above preset rate threshold can be a preset rate threshold set in advance, and the preset rate threshold can be flexibly adjusted according to the scenario.

[0104] The above generation rate can be the current generation rate of the target running log. Among them, the current generation rate can be determined according to the number of target running logs generated per unit time.

[0105] In an alternative embodiment, the threshold of the highest achievable log printing rate can be evaluated based on the available storage resource capacity and storage time requirements in the running log component, that is, the above preset threshold.

[0106] In the case where the generation rate is less than the preset rate threshold, it can be ensured that the stored log volume does not exceed the storage budget, that is, the target running log can be sharded based on the identification information of the network device to obtain at least one log shard. It should be noted that the storage budget can be determined according to the storage capacity of the storage medium or the actual storage environment.

[0107] In the above embodiments of the present application, in the case where the generation rate is greater than or equal to the preset rate threshold, the method further includes one of the following: determining a sampling rate based on the generation rate and the preset rate threshold, sampling the target running log according to the sampling rate to obtain a sampled running log, and sharding the sampled running log based on the identification information of the network device to obtain at least one log shard; discarding some of the running logs in the target running log to obtain the remaining running logs, and sharding the remaining running logs based on the identification information of the network device to obtain at least one log shard, where the remaining running logs are used to represent the running logs in the target running log except for some of the running logs.

[0108] When the generation rate is greater than or equal to the preset rate threshold, the number of generated target operation logs may exceed the storage budget. Therefore, a log decimation function is required. Through the log decimation function, the preset sampling rate of the target operation logs can be reduced, or the number of target operation logs can be reduced, so as to ensure that the number of sampled operation logs obtained is within the storage budget.

[0109] For example, in an actual storage environment, the storage resources available for logs are 900G. If the project requires retaining the log volume for 30 days, the daily log storage increment shall not be higher than 30G. Based on the estimated log storage consumption verified by the Internet of Things platform system experiment, the highest log printing rate is 1500 logs per second. Among them, the log storage consumption can be that continuously printing 1K logs per second for 1 day occupies 20G of space. The values here are only for illustration, and the specific situation shall be subject to the actual situation.

[0110] The above sampling rate can be a preset sampling rate for executing the log decimation function, and the above sampling rate can also be a sampling rate determined according to the current generation rate and the number of target operation logs. The determination method of the sampling rate is not limited here, and the sampling rate can be flexibly set according to the user's needs.

[0111] In an alternative embodiment, the number of sampled operation logs can be reduced by reducing the sampling rate. By reducing the sampling rate, the number of samples of the target operation logs can be reduced, thereby reducing the number of sampled operation logs obtained.

[0112] The above partial operation logs can be target operation logs with a generation rate per unit time greater than the preset rate threshold, where the unit time can be 1 second, and this is not limited here.

[0113] In another alternative embodiment, the number of sampled operation logs can be reduced by reducing some of the operation logs in the target operation logs. When the number of target operation logs is large, the number of operation logs to be sampled can be reduced by reducing the number of target operation logs. After discarding some of the operation logs in the target operation logs, the remaining operation logs can be sharded according to the identification information of the network device. Since the number of remaining operation logs is small, the number of log shards can be reduced.

[0114] In the above embodiments of the present application, the method further includes one of the following: discarding some of the operation logs according to the generation time of the target operation logs to obtain the remaining operation logs, where the number of the remaining operation logs is the preset number corresponding to the preset rate threshold; discarding some of the operation logs according to the interaction link to which the target operation logs belong to obtain the remaining operation logs, where there are no operation logs in some of the operation logs that belong to the same interaction link as the operation logs in the remaining operation logs.

[0115] The generation time of the above-mentioned target operation log can be used to sort the target operation log in chronological order, so as to determine the partial operation logs that need to be discarded in the target operation log.

[0116] In an alternative embodiment, if the generation rate of the target operation log is greater than the preset rate threshold, it indicates that the number of target operation logs generated at this time is relatively large. If the preset sampling rate remains unchanged and the sampling operation logs are continuously generated at this generation rate, it will result in a relatively large number of target operation logs generated finally, exceeding the storage budget. Therefore, it is necessary to discard the partial operation logs in the target operation log whose generation rate is greater than the preset rate threshold to reduce the number of target operation logs, so as to determine that the sampling operation logs obtained by sampling the target operation log do not exceed the storage budget.

[0117] The above-mentioned same interaction link refers to multiple links on the same device. The partial operation logs belonging to the same device as the target operation log can be determined from the target operation log, so as to maintain the consistency of link sampling, thereby improving the effectiveness of subsequent device problem troubleshooting.

[0118] The consistency of link sampling is used to indicate that the generation process of the target operation log in multiple links on the same device needs to be consistent. Similarly, the discarding process also needs to be consistent. It should be noted that link sampling consistency means that in a distributed system, for the same request or transaction, during the transmission process between multiple nodes and services, the sampled data can be kept consistent. Doing so can help developers and operation and maintenance personnel to be more accurate and convenient when troubleshooting and performance updating in a distributed system. Link sampling consistency can be achieved by using the same strategy on different nodes to ensure that the sampled data in the entire link is consistent.

[0119] In another alternative embodiment, in order to maintain the consistency of link sampling, after determining the partial operation logs from the target operation log, it is necessary to determine the partial operation logs belonging to the same interaction link as the partial operation logs from the target operation log, so as to discard the partial operation logs belonging to the same interaction link simultaneously, thereby maintaining the consistency of link sampling and improving the efficiency of device problem troubleshooting.

[0120] Figure 7 is a schematic diagram of a downsampling function according to an embodiment of the present application, as Figure 7As shown, the preset sampling rate can be updated according to the generation rate of the target operation log and a preset rate threshold. If the generation rate of the target operation log is less than the preset rate threshold, the target operation log can be sampled using the preset sampling rate to obtain a sampled operation log. If the generation rate of the target operation log is greater than or equal to the preset rate threshold, the target operation log is sampled according to the sampling rate to obtain a sampled operation log. A whitelist can be set to retain the target operation logs of the network devices in the whitelist, and downsampling can be performed on the target operation logs with consistent links.

[0121] In the above embodiments of the present application, the method further includes: determining whether the target network device corresponding to the target operation log is a preset device; in the case where the target network device is a preset device, prohibiting the discarding of some operation logs in the target operation log; in the case where the target network device is not a preset device, discarding some operation logs in the target operation log to obtain a remaining operation log.

[0122] The above-mentioned preset device can be a device pre-set in the whitelist, where the target operation logs generated by the preset device will not be discarded. It should be noted that the preset device can be a device that is key-concerned or prone to problems. By obtaining relatively complete target operation logs of the preset device, it is convenient to troubleshoot problems that occur in the preset device.

[0123] In an alternative embodiment, it can be determined whether the target network device corresponding to the target operation log is a preset device through the whitelist. In the case where the target network device is a preset device, it is necessary to retain the target operation log to reduce memory occupancy. In the case where the target network device is not a preset device, some operation logs in the target operation log can be directly discarded.

[0124] In the above embodiments of the present application, the method further includes: determining the storage resource information of the storage space for storing the sampled operation log, and the storage time of the sampled operation log; based on the storage resource information and the storage time, predicting to obtain a preset rate threshold.

[0125] The above-mentioned storage space can be a virtual space for storing the target operation log.

[0126] The above-mentioned storage resource information can be used to represent the storage resource capacity information of the storage space, and the storage space can be managed for storage through the storage resource capacity.

[0127] The above-mentioned storage time of the sampled operation log can be used to represent the unit time for storing the sampled operation log.

[0128] In an alternative embodiment, the preset rate threshold can be predicted by the ratio of the storage resource information to the storage time, or the ratio of the storage resource information to the storage time can be used as the reference rate threshold, and the reference rate threshold can be adjusted in combination with the operating environment of the sampled operation log to obtain the preset rate threshold, so that the obtained preset rate threshold can meet the current operating environment of the sampled operation log.

[0129] In this application, for the query scenario of the operation log in the IoT cloud, the query scenario can be to query the operation log of a specified device within a certain time window. In this application, a hybrid sharding method based on time sharding and IoT task data sharding can be adopted, which can reduce the magnitude of the retrieved log files and improve the query efficiency in the above query scenario.

[0130] This application can also rely on lightweight open-source log components to index the key data of the logs, reducing the requirements for resources such as storage, processors, and memory. It can automatically perform dynamic downsampling, ensure the availability of the log function when the task volume increases, and can evaluate the maximum allowable log printing rate according to the actual storage resource budget capacity and the requirement for log retention time. The system will automatically adjust the log sampling rate according to the original log generation rate to ensure that the actual stored log volume does not exceed the storage resource budget capacity.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0132] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0133] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present application.

[0134] Embodiment 2

[0135] According to an embodiment of the present application, a method for processing operation logs is further provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than this.

[0136] Figure 8 is a flowchart of a method for processing operation logs according to Embodiment 2 of the present application. As Figure 8 shown, the method includes the following steps:

[0137] Step S802, the cloud server receives a log query request sent by the client;

[0138] Among them, the log query request carries the identification information of the device to be queried and the time period to be queried.

[0139] The type of the above cloud server is not limited here. The above client can be the client used by the user.

[0140] The above log query request may include the identification information of the device to be queried and the time period to be queried. Through the identification information of the device to be queried and the time period to be queried, the log shards required by the user can be located.

[0141] Step S804, the cloud server obtains the target log shard that matches the identification information from the operation log set based on the identification information of the device to be queried;

[0142] Among them, the log shards in the operation log set are obtained by sharding the target operation log based on the identification information of the network device. The network device interacts with the network platform to which the cloud server belongs, and the target operation log is an operation log generated based on the interaction data between the network device and the network platform.

[0143] In an alternative embodiment, the target log shard that matches the identification information can be determined by determining the similarity between the identification information and the identification information in the log shards.

[0144] Step S806, the cloud server obtains target log data that matches the to-be-query time period from the target log shard;

[0145] Among them, the log shards in the running log set are stored based on the generation time of the log data included in the log shards.

[0146] Step S808, the cloud server outputs the target log data to the client.

[0147] Through the above steps, the cloud server receives a log query request sent by the client. Among them, the log query request carries the identification information of the device to be queried and the to-be-query time period; the cloud server obtains a target log shard that matches the identification information from the running log set based on the identification information of the device to be queried. Among them, the log shards in the running log set are obtained by sharding the target running log based on the identification information of the network device. The network device interacts with the network platform to which the cloud server belongs, and the target running log is a running log generated based on the interaction data between the network device and the network platform; the cloud server obtains target log data that matches the to-be-query time period from the target log shard. Among them, the log shards in the running log set are stored based on the generation time of the log data included in the log shards; the cloud server outputs the target log data to the client, thereby improving the query efficiency of the target running log; it is easy to note that the target running log can be sampled at a preset sampling rate to reduce the number of running logs. After sharding and storing the target running log through the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and further solving the technical problem of the low query efficiency of the running log in the related art.

[0148] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0149] Embodiment 3

[0150] According to an embodiment of the present application, a method for processing running logs is further provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than this.

[0151] Figure 9It is a flowchart of a method for processing operation logs according to Embodiment 3 of the present application. As Figure 9 shown, the method includes the following steps:

[0152] Step S902, during the interaction between the IoT device and the IoT platform, based on the interaction data between the IoT device and the IoT platform, generate the target operation log of the IoT device;

[0153] The above-mentioned IoT devices may include, but are not limited to, user devices, sensors, memories, etc.

[0154] The above-mentioned IoT platform is used to manage the IoT platform through network connection.

[0155] Step S904, based on the device identification information and product identification information of the IoT device, perform sharding processing on the target operation log to obtain at least one log shard;

[0156] Among them, the device identification information and product identification information corresponding to the log data included in different log shards are different.

[0157] Step S906, based on the generation time of the log data included in at least one log shard, store at least one log shard.

[0158] Through the above steps, during the interaction between the IoT device and the IoT platform, based on the interaction data between the IoT device and the IoT platform, generate the target operation log of the IoT device; based on the device identification information and product identification information of the IoT device, perform sharding processing on the target operation log to obtain at least one log shard, where the device identification information and product identification information corresponding to the log data included in different log shards are different; based on the generation time of the log data included in at least one log shard, store at least one log shard, thereby improving the query efficiency of the target operation log; it is easy to notice that the target operation log can be sampled according to a preset sampling rate to reduce the number of operation logs. After sharding and storing the target operation log through the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and further solving the technical problem of low query efficiency of operation logs in the related art.

[0159] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0160] Embodiment 4

[0161] According to an embodiment of the present application, there is also provided a method for processing operation logs. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than this.

[0162] Figure 10 It is a flowchart of a method for processing operation logs according to Embodiment 4 of the present application. As Figure 10 shown, the method includes the following steps:

[0163] Step S1002, in response to a device interaction instruction acting on the operation interface, display the interaction data between the network device and the network platform on the operation interface;

[0164] The above operation interface can be used to display a log generation instruction. Among them, the operation interface can include a variety of different controls for the user to operate, so as to display the log generation instruction on the operation interface.

[0165] Step S1004, in response to a log storage instruction acting on the operation interface, display the storage result of the target operation log of the network device on the operation interface.

[0166] The above log storage instruction can be an instruction triggered after the target operation log is collected, or an instruction generated by the user according to needs.

[0167] Among them, the target operation log is an operation log generated based on the interaction data, the storage result is the result obtained by storing at least one log shard based on the generation time of the log data included in the at least one log shard, and the at least one log shard is a shard obtained by sharding the target operation log based on the identification information of the network device. The identification information corresponding to the log data included in different log shards is different.

[0168] Through the above steps, in response to a device interaction instruction acting on the operation interface, interaction data between the network device and the network platform is displayed on the operation interface; in response to a log storage instruction acting on the operation interface, the storage result of the target operation log of the network device is displayed on the operation interface, where the target operation log is an operation log generated based on the interaction data, and the storage result is a result obtained by storing at least one log shard based on the generation time of the log data included in the at least one log shard. The at least one log shard is a shard obtained by sharding the target operation log based on the identification information of the network device, and the identification information corresponding to the log data included in different log shards is different, thereby improving the query efficiency of the target operation log; it is easy to notice that the target operation log can be sampled at a preset sampling rate to reduce the number of operation logs. After sharding and storing the target operation log through the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and further solving the technical problem of low query efficiency of operation logs in the related art.

[0169] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0170] Embodiment 5

[0171] According to an embodiment of the present application, a method for processing operation logs is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than this.

[0172] Figure 11 is a flowchart of a method for processing operation logs according to Embodiment 5 of the present application. As Figure 11 shown, the method includes the following steps:

[0173] Step S1102, obtaining interaction data between the network device and the network platform by calling a first interface;

[0174] Among them, the first interface includes a first parameter, and the parameter value of the first parameter includes the interaction data.

[0175] The first interface in the above steps can be an interface for data interaction between the cloud server and the client. The client can pass the interaction data into the interface function as the first parameter of the interface function to achieve the purpose of uploading the interaction data to the cloud server.

[0176] Step S1104, generating a target operation log of the network device based on the interaction data;

[0177] Step S1106: Based on the identification information of the network device, perform sharding processing on the target operation log to obtain at least one log shard;

[0178] Among them, the identification information corresponding to the log data included in different log shards is different.

[0179] Step S1108: Based on the generation time of the log data included in at least one log shard, store at least one log shard to obtain a storage result;

[0180] Step S1110: Output the storage result by calling the second interface.

[0181] Among them, the second interface includes a second parameter, and the parameter value of the second parameter includes the storage result.

[0182] The above-mentioned second interface can be an interface for data interaction between the cloud server and the client. The cloud server can pass the storage result into the interface function as the second parameter of the interface function to achieve the purpose of sending the storage result to the client.

[0183] Through the above steps, obtain the interaction data between the network device and the network platform by calling the first interface. Among them, the first interface includes a first parameter, and the parameter value of the first parameter includes the interaction data; generate the target operation log of the network device based on the interaction data; perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard. Among them, the identification information corresponding to the log data included in different log shards is different; store at least one log shard based on the generation time of the log data included in at least one log shard to obtain a storage result; output the storage result by calling the second interface. Among them, the second interface includes a second parameter, and the parameter value of the second parameter includes the storage result, thereby improving the query efficiency of the target operation log; it is easy to notice that the target operation log can be sampled according to a preset sampling rate to reduce the number of operation logs. After sharding and storing the target operation log through the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and further solving the technical problem of low query efficiency of operation logs in the related art.

[0184] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0185] Embodiment 6

[0186] According to an embodiment of the present application, there is also provided a processing device for operation logs for implementing the above-mentioned processing method of operation logs. Figure 12Schematic diagram of a processing device for operation logs according to Embodiment 6 of the present application, as shown in Figure 12 shown, the device 1200 includes: a generation module 1202, a sharding module 1204, and a storage module 1206.

[0187] Among them, the generation module is used to generate the target operation log of the network device based on the interaction data between the network device and the network platform during the interaction between the network device and the network platform; the sharding module is used to perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; the storage module is used to store at least one log shard based on the generation time of the log data included in at least one log shard.

[0188] It should be noted here that the above generation module 1202, sharding module 1204, and storage module 1206 correspond to steps S402 to S406 in Embodiment 1. The instances and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n), and the above modules can also be part of the device and can run in the computer terminal 10 provided in Embodiment 1.

[0189] In the above embodiments of the present application, the sharding module is further used to perform modulo operation on the identification information to obtain an index label corresponding to the target operation log; perform sharding processing on the target operation log based on the index label to obtain at least one log shard, where the index labels corresponding to the log data included in different log shards are the same.

[0190] In the above embodiments of the present application, the sharding module is further used to perform a hash operation on the identification information to obtain a hash value; determine the modulus of the modulo operation based on the number of network devices; perform modulo operation on the same hash value based on the modulus to obtain an index label.

[0191] In the above embodiments of the present application, the sharding module is further used to determine the generation rate of the target operation log; in the case where the generation rate is less than the preset rate threshold, perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard.

[0192] In the above embodiments of the present application, the device further includes: a determination module and a discard module.

[0193] Among them, the determination module is used to determine the sampling rate based on the generation rate and a preset rate threshold, sample the target operation log according to the sampling rate to obtain a sampled operation log, and perform sharding processing on the sampled operation log based on the identification information of the network device to obtain at least one log shard; the discarding module is used to discard part of the operation log in the target operation log to obtain a remaining operation log, and perform sharding processing on the remaining operation log based on the identification information of the network device to obtain at least one log shard, where the remaining operation log is used to represent the operation log in the target operation log except for part of the operation log.

[0194] In the above embodiments of the present application, the discarding module is further used to discard part of the operation log according to the generation time of the target operation log to obtain a remaining operation log, where the number of the remaining operation logs is a preset number corresponding to the preset rate threshold; the discarding module is further used to discard part of the operation log according to the interaction link to which the target operation log belongs to obtain a remaining operation log, where there is no operation log in the part of the operation log that belongs to the same interaction link as the operation log in the remaining operation log.

[0195] In the above embodiments of the present application, the determination module is further used to determine whether the target network device corresponding to the target operation log is a preset device. If the target network device is a preset device, it is prohibited to discard part of the operation log in the target operation log. If the target network device is not a preset device, part of the operation log in the target operation log is discarded to obtain a remaining operation log.

[0196] In the above embodiments of the present application, the device further includes: a prediction module.

[0197] Among them, the determination module is further used to determine the storage resource information of the storage space for storing the sampled operation log and the storage time of the sampled operation log; the prediction module is further used to predict a preset rate threshold based on the storage resource information and the storage time.

[0198] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0199] Embodiment 7

[0200] According to an embodiment of the present application, there is also provided a processing device for operation logs for implementing the above-mentioned operation log processing method. Figure 13 It is a schematic diagram of a processing device for operation logs according to Embodiment 7 of the present application, as Figure 13 shown. The device 1300 includes: a receiving module 1302, a sharding module 1304, an obtaining module 1306, and an output module 1308.

[0201] Among them, the receiving module is used to receive the log query request sent by the client through the cloud server. The log query request carries the identification information of the device to be queried and the time period to be queried. The sharding module is used to obtain the target log shard that matches the identification information from the running log set through the cloud server based on the identification information of the device to be queried. The log shards in the running log set are obtained by sharding the target running log based on the identification information of the network device. The network device interacts with the network platform to which the cloud server belongs, and the target running log is the running log generated based on the interaction data between the network device and the network platform. The obtaining module is used to obtain the target log data that matches the time period to be queried from the target log shard through the cloud server. The log shards in the running log set are stored based on the generation time of the log data included in the log shard. The output module is used to output the target log data to the client through the cloud server.

[0202] It should be noted here that the above receiving module 1302, sharding module 1304, obtaining module 1306, and output module 1308 correspond to steps S802 to S808 in Embodiment 2. The instances and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n). The above modules can also be part of the device and can run in the computer terminal 10 provided in Embodiment 1.

[0203] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0204] Embodiment 8

[0205] According to an embodiment of the present application, there is also provided a processing device for running logs for implementing the above processing method of running logs. Figure 14 It is a schematic diagram of a processing device for running logs according to Embodiment 8 of the present application, as Figure 14 shown. The device 1400 includes: a generating module 1402, a sharding module 1404, and a storage module 1406.

[0206] Among them, the generation module is used to generate the target operation log of the Internet of Things device based on the interaction data between the Internet of Things device and the Internet of Things platform during the interaction process between the Internet of Things device and the Internet of Things platform; the sharding module is used to perform sharding processing on the target operation log based on the device identification information and product identification information of the Internet of Things device to obtain at least one log shard, where the device identification information and product identification information corresponding to the log data included in different log shards are different; the storage module is used to store at least one log shard based on the generation time of the log data included in at least one log shard.

[0207] It should be noted here that the above generation module 1402, sharding module 1404, and storage module 1406 correspond to steps S902 to S906 in Embodiment 3. The instances and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n). The above modules can also be part of a device and can run in the computer terminal 10 provided in Embodiment 1.

[0208] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0209] Embodiment 9

[0210] According to an embodiment of the present application, there is also provided a processing device for an operation log for implementing the above processing method of the operation log. Figure 15 It is a schematic diagram of a processing device for an operation log according to Embodiment 9 of the present application. As Figure 15 shown, the device 1500 includes: a first display module 1502 and a second display module 1504.

[0211] Among them, the first display module is used to display the interaction data between the network device and the network platform on the operation interface in response to a device interaction instruction acting on the operation interface; the second display module is used to display the storage result of the target operation log of the network device on the operation interface in response to a log storage instruction acting on the operation interface, where the target operation log is an operation log generated based on the interaction data, the storage result is a result obtained by storing at least one log shard based on the generation time of the log data included in at least one log shard, and at least one log shard is a shard obtained by performing sharding processing on the target operation log based on the identification information of the network device, and the identification information corresponding to the log data included in different log shards is different.

[0212] It should be noted that the above first display module 1502 and second display module 1504 correspond to steps S1002 to S1004 in Embodiment 4. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules or units may be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n). The above modules may also be part of a device and may run in the computer terminal 10 provided in Embodiment 1.

[0213] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0214] Embodiment 10

[0215] According to an embodiment of the present application, there is also provided a processing device for a running log for implementing the above processing method of the running log. Figure 16 It is a schematic diagram of a processing device for a running log according to Embodiment 10 of the present application, as Figure 16 shown. The device 1600 includes: an acquisition module 1602, a generation module 1604, a sharding module 1606, a storage module 1608, and an output module 1610.

[0216] The acquisition module is used to obtain the interaction data between the network device and the network platform by calling a first interface. Among them, the first interface includes a first parameter, and the parameter value of the first parameter includes the interaction data; the generation module is used to generate a target running log of the network device based on the interaction data; the sharding module is used to perform sharding processing on the target running log based on the identification information of the network device to obtain at least one log shard. Among them, the identification information corresponding to the log data included in different log shards is different; the storage module is used to store at least one log shard based on the generation time of the log data included in the at least one log shard to obtain a storage result; the output module is used to output the storage result by calling a second interface. Among them, the second interface includes a second parameter, and the parameter value of the second parameter includes the storage result.

[0217] It should be noted that the above-mentioned acquisition module 1602, generation module 1604, sharding module 1606, storage module 1608, and output module 1610 correspond to steps S1102 to S1110 in Embodiment 5. The instances and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the content disclosed in the above-mentioned Embodiment 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n). The above-mentioned modules may also be part of a device and may run in the computer terminal 10 provided in Embodiment 1.

[0218] It should be noted that the preferred implementation schemes involved in the above-mentioned embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0219] Embodiment 11

[0220] An embodiment of the present application may provide a computer terminal, and the computer terminal may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the above-mentioned computer terminal may also be replaced with a terminal device such as a mobile terminal.

[0221] Optionally, in this embodiment, the above-mentioned computer terminal may be located in at least one of multiple network devices in a computer network.

[0222] In this embodiment, the above-mentioned computer terminal may execute program codes of the following steps in the processing method of the operation log: during the interaction between the network device and the network platform, based on the interaction data between the network device and the network platform, generate a target operation log of the network device; perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; store at least one log shard based on the generation time of the log data included in at least one log shard.

[0223] Optionally, Figure 17 is a structural block diagram of a computer terminal according to an embodiment of the present application. As Figure 17 shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 102, a memory 104, a storage controller, and a peripheral interface, where the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0224] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the processing method and device of the running log in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned processing method of the running log. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely disposed relative to the processor, and these remote memories can be connected to the terminal A through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0225] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: during the interaction between the network device and the network platform, based on the interaction data between the network device and the network platform, generate the target running log of the network device; perform sharding processing on the target running log based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; store at least one log shard based on the generation time of the log data included in at least one log shard.

[0226] Optionally, the above processor may further execute the program code of the following steps: perform modulo operation on the identification information to obtain the index label corresponding to the target running log; perform sharding processing on the target running log based on the index label to obtain at least one log shard, where the index labels corresponding to the log data included in different log shards are the same.

[0227] Optionally, the above processor may further execute the program code of the following steps: perform hash operation on the identification information to obtain a hash value; determine the modulus of the modulo operation based on the number of network devices; perform modulo operation on the same hash value based on the modulus to obtain the index label.

[0228] Optionally, the above processor may further execute the program code of the following steps: determine the generation rate of the target running log; in the case where the generation rate is less than the preset rate threshold, perform sharding processing on the target running log based on the identification information of the network device to obtain at least one log shard.

[0229] Optionally, the above-mentioned processor may also execute the program code of the following steps: Determine the sampling rate based on the generation rate and a preset rate threshold, sample the target operation log according to the sampling rate to obtain a sampled operation log, and perform sharding processing on the sampled operation log based on the identification information of the network device to obtain at least one log shard; Discard part of the operation log in the target operation log to obtain a remaining operation log, and perform sharding processing on the remaining operation log based on the identification information of the network device to obtain at least one log shard, where the remaining operation log is used to represent the operation log in the target operation log except for the part of the operation log.

[0230] Optionally, the above-mentioned processor may also execute the program code of the following steps: Discard part of the operation log according to the generation time of the target operation log to obtain a remaining operation log, where the number of the remaining operation logs is a preset number corresponding to the preset rate threshold; Discard part of the operation log according to the interaction link to which the target operation log belongs to obtain a remaining operation log, where there is no operation log in the part of the operation log that belongs to the same interaction link as the operation log in the remaining operation log.

[0231] Optionally, the above-mentioned processor may also execute the program code of the following steps: Determine whether the target network device corresponding to the target operation log is a preset device; In the case where the target network device is a preset device, prohibit discarding part of the operation log in the target operation log; In the case where the target network device is not a preset device, discard part of the operation log in the target operation log to obtain a remaining operation log.

[0232] Optionally, the above-mentioned processor may also execute the program code of the following steps: Determine the storage resource information of the storage space for storing the sampled operation log and the storage time of the sampled operation log; Predict a preset rate threshold based on the storage resource information and the storage time.

[0233] The processor may call the information and application programs stored in the memory through the transmission device to execute the following steps: The cloud server receives a log query request sent by the client, where the log query request carries the identification information of the device to be queried and the time period to be queried; The cloud server obtains a target log shard that matches the identification information from the operation log set based on the identification information of the device to be queried, where the log shards in the operation log set are obtained by performing sharding processing on the target operation log based on the identification information of the network device, the network device interacts with the network platform to which the cloud server belongs, and the target operation log is an operation log generated based on the interaction data between the network device and the network platform; The cloud server obtains target log data that matches the time period to be queried from the target log shard, where the log shards in the operation log set are stored based on the generation time of the log data included in the log shard; The cloud server outputs the target log data to the client.

[0234] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: During the interaction between the Internet of Things device and the Internet of Things platform, based on the interaction data between the Internet of Things device and the Internet of Things platform, generate the target operation log of the Internet of Things device; Based on the device identification information and product identification information of the Internet of Things device, perform fragmentation processing on the target operation log to obtain at least one log fragment, where the device identification information and product identification information corresponding to the log data included in different log fragments are different; Based on the generation time of the log data included in at least one log fragment, store at least one log fragment.

[0235] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: Respond to the device interaction instruction acting on the operation interface, and display the interaction data between the network device and the network platform on the operation interface; Respond to the log storage instruction acting on the operation interface, and display the storage result of the target operation log of the network device on the operation interface, where the target operation log is the operation log generated based on the interaction data, the storage result is the result obtained by storing at least one log fragment based on the generation time of the log data included in at least one log fragment, at least one log fragment is the fragment obtained by performing fragmentation processing on the target operation log based on the identification information of the network device, and the identification information corresponding to the log data included in different log fragments is different.

[0236] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: Obtain the interaction data between the network device and the network platform by calling the first interface, where the first interface includes a first parameter, and the parameter value of the first parameter includes the interaction data; Generate the target operation log of the network device based on the interaction data; Perform fragmentation processing on the target operation log based on the identification information of the network device to obtain at least one log fragment, where the identification information corresponding to the log data included in different log fragments is different; Based on the generation time of the log data included in at least one log fragment, store at least one log fragment to obtain a storage result; Output the storage result by calling the second interface, where the second interface includes a second parameter, and the parameter value of the second parameter includes the storage result.

[0237] By adopting the embodiment of the present application, during the interaction between the network device and the network platform, based on the interaction data between the network device and the network platform, the target operation log of the network device is generated; the target operation log is segmented based on the identification information of the network device to obtain at least one log segment, wherein the identification information corresponding to the log data included in different log segments is different; based on the generation time of the log data included in the at least one log segment, the at least one log segment is stored, thereby improving the query efficiency of the target operation log; it is easy to notice that the target operation log can be sampled at a preset sampling rate to reduce the number of operation logs. After segmenting and storing the target operation log through the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and further solving the technical problem of low query efficiency of operation logs in the related art.

[0238] Those of ordinary skill in the art can understand that Figure 17 the structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and a mobile Internet device (Mobile Internet Devices, MID), a PAD and other terminal devices. Figure 17 It does not limit the structure of the above electronic device. For example, computer terminal A may further include more or fewer components (such as a network interface, a display device, etc.) than those shown in Figure 17 the figure, or have a different configuration from that shown in Figure 17 the figure.

[0239] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware of the terminal device through a program, and the program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disc, etc.

[0240] Embodiment 12

[0241] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the above storage medium can be used to store the program code executed by the processing method of the operation log provided in the first embodiment above.

[0242] Optionally, in this embodiment, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0243] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: during the interaction between the network device and the network platform, generate a target operation log of the network device based on the interaction data between the network device and the network platform; perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; store the at least one log shard based on the generation time of the log data included in the at least one log shard.

[0244] Optionally, the above storage medium is further configured to store program code for performing the following steps: perform modulo operation on the identification information to obtain an index label corresponding to the target operation log; perform sharding processing on the target operation log based on the index label to obtain at least one log shard, where the index labels corresponding to the log data included in different log shards are the same.

[0245] Optionally, the above storage medium is further configured to store program code for performing the following steps: perform a hash operation on the identification information to obtain a hash value; determine the modulus of the modulo operation based on the number of network devices; perform modulo operation on the same hash value based on the modulus to obtain an index label.

[0246] Optionally, the above storage medium is further configured to store program code for performing the following steps: determine the generation rate of the target operation log; in the case where the generation rate is less than a preset rate threshold, perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard.

[0247] Optionally, the above storage medium is further configured to store program code for performing the following steps: determine a sampling rate based on the generation rate and the preset rate threshold, sample the target operation log according to the sampling rate to obtain a sampled operation log, and perform sharding processing on the sampled operation log based on the identification information of the network device to obtain at least one log shard; discard some operation logs in the target operation log to obtain a remaining operation log, and perform sharding processing on the remaining operation log based on the identification information of the network device to obtain at least one log shard, where the remaining operation log is used to represent the operation log in the target operation log except for some operation logs.

[0248] Optionally, the above storage medium is further configured to store program code for performing the following steps: discard some operation logs according to the generation time of the target operation log to obtain a remaining operation log, where the number of the remaining operation logs is a preset number corresponding to the preset rate threshold; discard some operation logs according to the interaction link to which the target operation log belongs to obtain a remaining operation log, where there is no operation log in some operation logs that belongs to the same interaction link as the operation log in the remaining operation log.

[0249] Optionally, the above storage medium is further configured to store program code for performing the following steps: determining whether a target network device corresponding to a target operation log is a preset device; in a case where the target network device is a preset device, prohibiting discarding part of the operation log in the target operation log; in a case where the target network device is not a preset device, discarding part of the operation log in the target operation log to obtain a remaining operation log.

[0250] Optionally, the above storage medium is further configured to store program code for performing the following steps: determining storage resource information of a storage space for storing sampled operation logs and a storage time of the sampled operation logs; predicting a preset rate threshold based on the storage resource information and the storage time.

[0251] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: a cloud server receives a log query request sent by a client, where the log query request carries identification information of a device to be queried and a time period to be queried; the cloud server obtains a target log shard matching the identification information from an operation log set, where the log shards in the operation log set are obtained by performing sharding processing on a target operation log based on the identification information of network devices, the network devices interact with a network platform to which the cloud server belongs, and the target operation log is an operation log generated based on interaction data between the network devices and the network platform; the cloud server obtains target log data matching the time period to be queried from the target log shard, where the log shards in the operation log set are stored based on the generation time of the log data included in the log shards; the cloud server outputs the target log data to the client.

[0252] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: during the interaction between an Internet of Things device and an Internet of Things platform, generating a target operation log of the Internet of Things device based on the interaction data between the Internet of Things device and the Internet of Things platform; performing sharding processing on the target operation log based on the device identification information and product identification information of the Internet of Things device to obtain at least one log shard, where the device identification information and product identification information corresponding to the log data included in different log shards are different; storing the at least one log shard based on the generation time of the log data included in the at least one log shard.

[0253] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: in response to a device interaction instruction acting on the operation interface, display the interaction data between the network device and the network platform on the operation interface; in response to a log storage instruction acting on the operation interface, display the storage result of the target operation log of the network device on the operation interface, where the target operation log is an operation log generated based on the interaction data, and the storage result is a result obtained by storing at least one log shard based on the generation time of the log data included in the at least one log shard. The at least one log shard is a shard obtained by sharding the target operation log based on the identification information of the network device, and the identification information corresponding to the log data included in different log shards is different.

[0254] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtain the interaction data between the network device and the network platform by calling a first interface, where the first interface includes a first parameter, and the parameter value of the first parameter includes the interaction data; generate a target operation log of the network device based on the interaction data; perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; store the at least one log shard based on the generation time of the log data included in the at least one log shard to obtain a storage result; output the storage result by calling a second interface, where the second interface includes a second parameter, and the parameter value of the second parameter includes the storage result.

[0255] By adopting the embodiment of the present application, during the interaction between the network device and the network platform, a target operation log of the network device is generated based on the interaction data between the network device and the network platform; the target operation log is sharded based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; the at least one log shard is stored based on the generation time of the log data included in the at least one log shard, thereby improving the query efficiency of the target operation log. It is easy to notice that the target operation log can be sampled at a preset sampling rate to reduce the number of operation logs. After sharding and storing the target operation log by the identification information of the network device, the subsequent query index can be reduced, thereby improving the query efficiency, and further solving the technical problem of low query efficiency of operation logs in the related art.

[0256] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0257] In the above embodiments of the present application, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0258] In several embodiments provided by this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed among each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0259] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0260] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0261] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this application. The foregoing storage medium includes: USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs and other various media that can store program codes.

[0262] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A method for processing operation logs, characterized in that, Including: During the interaction between the network device and the network platform, based on the interaction data between the network device and the network platform, generate the target operation log of the network device; Based on the identification information of the network device, perform sharding processing on the target operation log to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; Based on the generation time of the log data included in the at least one log shard, store the at least one log shard.

2. The method according to claim 1, characterized in that, Based on the identification information of the network device, performing sharding processing on the target operation log to obtain at least one log shard includes: Perform a modulo operation on the identification information to obtain an index label corresponding to the target operation log; Based on the index label, perform sharding processing on the target operation log to obtain the at least one log shard, where the index labels corresponding to the log data included in different log shards are the same.

3. The method according to claim 2, characterized in that, Performing a modulo operation on the identification information to obtain an index label corresponding to the target operation log includes: Perform a hash operation on the identification information to obtain a hash value; Based on the number of network devices, determine the modulus of the modulo operation; Based on the modulus, perform a modulo operation on the same hash value to obtain the index label.

4. The method according to claim 1, wherein Based on the identification information of the network device, performing sharding processing on the target operation log to obtain at least one log shard includes: Determine the generation rate of the target operation log; When the generation rate is less than a preset rate threshold, based on the identification information of the network device, perform sharding processing on the target operation log to obtain the at least one log shard.

5. The method according to claim 4, characterized in that, When the generation rate is greater than or equal to the preset rate threshold, the method further includes one of the following: Based on the generation rate and the preset rate threshold, determine a sampling rate, sample the target operation log according to the sampling rate to obtain a sampled operation log, and based on the identification information of the network device, perform sharding processing on the sampled operation log to obtain the at least one log shard; Discard some operation logs in the target operation log to obtain a remaining operation log, and based on the identification information of the network device, perform sharding processing on the remaining operation log to obtain the at least one log shard, where the remaining operation log is used to represent the operation log in the target operation log except for the some operation logs.

6. The method according to claim 5, wherein Discarding some operation logs in the target operation log to obtain a remaining operation log includes one of the following: According to the generation time of the target operation log, discard the some operation logs to obtain the remaining operation log, where the number of the remaining operation logs is a preset number corresponding to the preset rate threshold; According to the interaction link to which the target operation log belongs, discard the some operation logs to obtain the remaining operation log, where there is no operation log in the some operation logs that belongs to the same interaction link as the operation logs in the remaining operation log.

7. The method according to claim 6, characterized in that, The method further includes: Determine whether the target network device corresponding to the target operation log is a preset device; In the case where the target network device is the preset device, prohibit discarding the partial operation log in the target operation log; In the case where the target network device is not the preset device, discard the partial operation log in the target operation log to obtain the remaining operation log.

8. The method according to claim 5, characterized in that The method further includes: Determine the storage resource information of the storage space for storing the sampled operation log, and the storage time of the sampled operation log; Based on the storage resource information and the storage time, predict to obtain the preset rate threshold.

9. A method for processing operation logs, characterized in that, It includes: The cloud server receives a log query request sent by the client, where the log query request carries the identification information of the device to be queried and the time period to be queried; The cloud server obtains a target log shard that matches the identification information from the operation log set, where the log shards in the operation log set are obtained by sharding the target operation log based on the identification information of the network device, the network device interacts with the network platform to which the cloud server belongs, and the target operation log is an operation log generated based on the interaction data between the network device and the network platform; The cloud server obtains target log data that matches the time period to be queried from the target log shard, where the log shards in the operation log set are stored based on the generation time of the log data included in the log shards; The cloud server outputs the target log data to the client.

10. A method for processing operation logs, characterized in that, It includes: During the interaction between the IoT device and the IoT platform, generate the target operation log of the IoT device based on the interaction data between the IoT device and the IoT platform; Based on the device identification information and product identification information of the IoT device, perform sharding processing on the target operation log to obtain at least one log shard, where the device identification information and product identification information corresponding to the log data included in different log shards are different; Based on the generation time of the log data included in the at least one log shard, store the at least one log shard.

11. A method for processing operation logs, characterized in that, It includes: In response to a device interaction instruction acting on the operation interface, display the interaction data between the network device and the network platform on the operation interface; In response to a log storage instruction acting on the operation interface, display the storage result of the target operation log of the network device on the operation interface, where the target operation log is an operation log generated based on the interaction data, the storage result is the result obtained by storing the at least one log shard based on the generation time of the log data included in the at least one log shard, the at least one log shard is a shard obtained by sharding the target operation log based on the identification information of the network device, and the identification information corresponding to the log data included in different log shards is different.

12. A method for processing operation logs, characterized in that, It includes: Obtain the interaction data between the network device and the network platform by calling the first interface, where the first interface includes a first parameter, and the parameter value of the first parameter includes the interaction data; Generate the target operation log of the network device based on the interaction data; Perform sharding processing on the target operation log based on the identification information of the network device to obtain at least one log shard, where the identification information corresponding to the log data included in different log shards is different; Store the at least one log shard based on the generation time of the log data included in the at least one log shard to obtain a storage result; Output the storage result by calling the second interface, where the second interface includes a second parameter, and the parameter value of the second parameter includes the storage result.

13. An electronic device, characterized in that, Comprising: A memory storing an executable program; A processor for running the program, where when the program runs, it executes the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, where when the executable program runs, it controls the device where the storage medium is located to execute the method according to any one of claims 1 to 12.

Images